2023-11-24T05:52:47.148 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T05:52:47.148 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --name rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550 --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} --verbose -- /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/orig.config.yaml 2023-11-24T05:52:47.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T05:52:47.338 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} email: null first_in_suite: false job_id: '7465550' kernel: client: sha1: distro flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3305 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_05:41:33 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T05:52:47.338 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa; will attempt to use it 2023-11-24T05:52:47.339 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks 2023-11-24T05:52:47.339 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T05:52:47.341 INFO:teuthology.task.internal:Checking packages... 2023-11-24T05:52:47.374 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T05:52:47.375 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T05:52:47.375 INFO:teuthology.packaging:ref: None 2023-11-24T05:52:47.375 INFO:teuthology.packaging:tag: None 2023-11-24T05:52:47.375 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T05:52:47.375 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T05:52:47.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T05:52:47.597 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T05:52:47.598 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T05:52:47.605 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T05:52:47.605 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T05:52:47.610 INFO:teuthology.task.internal:Saving configuration 2023-11-24T05:52:47.629 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T05:52:47.635 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T05:52:47.665 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:47:46.624557', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T05:52:47.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:47:46.625801', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T05:52:47.695 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T05:52:47.703 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T05:52:47.703 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T05:52:47.703 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T05:52:47.907 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f372cc09b80>, signals=[15]) 2023-11-24T05:52:47.907 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T05:52:47.915 INFO:teuthology.task.internal:Opening connections... 2023-11-24T05:52:47.916 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T05:52:47.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:52:48.034 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T05:52:48.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:52:48.145 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T05:52:48.152 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2023-11-24T05:52:48.172 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2023-11-24T05:52:48.172 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2023-11-24T05:52:48.234 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-24T05:52:48.235 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T05:52:48.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T05:52:48.236 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-24T05:52:48.236 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-24T05:52:48.236 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-24T05:52:48.236 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-24T05:52:48.237 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-11-24T05:52:48.254 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2023-11-24T05:52:48.274 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2023-11-24T05:52:48.275 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-11-24T05:52:48.333 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Red Hat Enterprise Linux" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="8.6 (Ootpa)" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:ID="rhel" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="fedora" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="8.6" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el8" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.redhat.com/" 2023-11-24T05:52:48.334 INFO:teuthology.orchestra.run.smithi183.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-24T05:52:48.335 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T05:52:48.335 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T05:52:48.335 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-24T05:52:48.335 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-24T05:52:48.335 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-24T05:52:48.335 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-24T05:52:48.335 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-11-24T05:52:48.354 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T05:52:48.366 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T05:52:48.371 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T05:52:48.371 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2023-11-24T05:52:48.374 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2023-11-24T05:52:48.391 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T05:52:48.397 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T05:52:48.397 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2023-11-24T05:52:48.437 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2023-11-24T05:52:48.475 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T05:52:48.534 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T05:52:48.551 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T05:52:48.551 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-24T05:52:48.551 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-24T05:52:48.552 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T05:52:48.552 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T05:52:48.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:52:48.570 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2023-11-24T05:52:48.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:52:48.574 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2023-11-24T05:52:48.632 INFO:teuthology.orchestra.run.smithi183.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-24T05:52:48.632 INFO:teuthology.task.kernel:Running kernel on smithi183: 4.18.0-372.9.1.el8.x86_64 2023-11-24T05:52:48.633 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum install -y kernel 2023-11-24T05:52:48.638 INFO:teuthology.orchestra.run.smithi097.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-24T05:52:48.638 INFO:teuthology.task.kernel:Running kernel on smithi097: 4.18.0-372.9.1.el8.x86_64 2023-11-24T05:52:48.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum install -y kernel 2023-11-24T05:52:49.801 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-11-24T05:52:49.802 INFO:teuthology.orchestra.run.smithi183.stdout:Unable to read consumer identity 2023-11-24T05:52:49.932 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-11-24T05:52:49.933 INFO:teuthology.orchestra.run.smithi097.stdout:Unable to read consumer identity 2023-11-24T05:52:51.450 INFO:teuthology.orchestra.run.smithi097.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-11-24T05:52:51.458 INFO:teuthology.orchestra.run.smithi183.stdout:Copr repo for python3-asyncssh owned by ceph 9.2 kB/s | 3.5 kB 00:00 2023-11-24T05:52:52.234 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-11-24T05:52:52.551 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-11-24T05:52:55.701 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 428 kB/s | 24 kB 00:00 2023-11-24T05:52:56.109 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 382 kB/s | 24 kB 00:00 2023-11-24T05:52:56.411 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-24T05:52:56.474 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-11-24T05:52:56.475 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2023-11-24T05:52:56.476 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-11-24T05:52:56.604 DEBUG:teuthology.orchestra.run.smithi097:> echo no | sudo yum reinstall kernel || true 2023-11-24T05:52:56.811 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-24T05:52:56.862 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2023-11-24T05:52:56.863 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2023-11-24T05:52:56.863 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2023-11-24T05:52:56.933 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-11-24T05:52:56.933 INFO:teuthology.orchestra.run.smithi097.stdout:Unable to read consumer identity 2023-11-24T05:52:56.998 DEBUG:teuthology.orchestra.run.smithi183:> echo no | sudo yum reinstall kernel || true 2023-11-24T05:52:57.197 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 05:52:55 AM UTC. 2023-11-24T05:52:57.236 INFO:teuthology.orchestra.run.smithi097.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T05:52:57.274 INFO:teuthology.orchestra.run.smithi097.stderr:Error: No packages marked for reinstall. 2023-11-24T05:52:57.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum reinstall -y kernel || true 2023-11-24T05:52:57.368 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-11-24T05:52:57.368 INFO:teuthology.orchestra.run.smithi183.stdout:Unable to read consumer identity 2023-11-24T05:52:57.632 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:01 ago on Fri 24 Nov 2023 05:52:56 AM UTC. 2023-11-24T05:52:57.648 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-11-24T05:52:57.648 INFO:teuthology.orchestra.run.smithi097.stdout:Unable to read consumer identity 2023-11-24T05:52:57.672 INFO:teuthology.orchestra.run.smithi183.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T05:52:57.711 INFO:teuthology.orchestra.run.smithi183.stderr:Error: No packages marked for reinstall. 2023-11-24T05:52:57.758 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum reinstall -y kernel || true 2023-11-24T05:52:57.906 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 05:52:55 AM UTC. 2023-11-24T05:52:57.952 INFO:teuthology.orchestra.run.smithi097.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T05:52:57.979 INFO:teuthology.orchestra.run.smithi097.stderr:Error: No packages marked for reinstall. 2023-11-24T05:52:58.026 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2023-11-24T05:52:58.088 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-11-24T05:52:58.089 INFO:teuthology.orchestra.run.smithi183.stdout:Unable to read consumer identity 2023-11-24T05:52:58.106 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-24T05:52:58.107 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-24T05:52:58.107 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-24T05:52:58.107 DEBUG:teuthology.parallel:result is None 2023-11-24T05:52:58.343 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 05:52:56 AM UTC. 2023-11-24T05:52:58.383 INFO:teuthology.orchestra.run.smithi183.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T05:52:58.405 INFO:teuthology.orchestra.run.smithi183.stderr:Error: No packages marked for reinstall. 2023-11-24T05:52:58.452 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2023-11-24T05:52:58.533 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-24T05:52:58.533 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-24T05:52:58.533 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-24T05:52:58.533 DEBUG:teuthology.parallel:result is None 2023-11-24T05:52:58.533 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T05:52:58.541 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T05:52:58.541 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T05:52:58.544 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T05:52:58.561 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T05:52:58.570 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T05:52:58.580 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T05:52:58.580 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T05:52:58.603 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T05:52:58.633 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T05:52:58.641 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T05:52:58.641 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T05:52:58.670 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T05:52:58.697 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T05:52:58.718 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T05:52:58.724 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T05:52:58.736 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T05:52:58.738 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T05:52:58.745 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T05:52:58.745 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T05:52:58.747 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T05:52:58.787 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T05:52:58.797 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T05:52:58.797 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T05:52:58.815 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T05:52:58.844 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T05:52:58.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T05:52:58.985 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T05:52:59.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T05:52:59.112 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T05:52:59.112 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T05:52:59.180 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T05:52:59.222 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T05:52:59.270 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T05:52:59.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T05:52:59.395 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T05:52:59.395 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T05:52:59.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2023-11-24T05:52:59.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2023-11-24T05:52:59.518 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T05:52:59.557 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T05:52:59.991 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T05:53:00.000 INFO:teuthology.task.internal:Starting timer... 2023-11-24T05:53:00.000 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T05:53:00.023 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T05:53:00.034 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-24T05:53:00.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2023-11-24T05:53:00.099 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2023-11-24T05:53:00.100 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2023-11-24T05:53:00.167 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2023-11-24T05:53:00.169 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T05:53:00.169 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2023-11-24T05:53:00.196 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2023-11-24T05:53:00.197 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2023-11-24T05:53:00.229 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2023-11-24T05:53:00.231 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2023-11-24T05:53:00.239 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T05:53:00.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:53:00.268 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T05:53:00.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:53:00.298 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T05:53:00.298 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2023-11-24T05:53:00.334 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2023-11-24T05:53:00.369 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T05:53:00.379 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-24T05:53:00.428 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-24T05:53:00.858 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T05:53:00.949 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T05:53:00.952 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T06:05:30.468 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2023-11-24T06:05:30.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-24T06:05:30.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:05:30.538 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-11-24T06:05:30.606 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-24T06:05:30.607 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-11-24T06:05:30.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:05:30.678 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-11-24T06:05:30.748 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-11-24T06:05:30.749 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T06:05:30.760 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T06:05:30.760 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:05:30.760 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:05:30.762 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:05:30.763 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:05:30.791 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:05:30.805 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:05:30.825 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:05:30.839 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:05:30.864 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2023-11-24T06:05:30.876 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2023-11-24T06:05:30.890 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:05:30.903 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:05:30.911 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2023-11-24T06:05:30.923 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2023-11-24T06:05:30.937 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:05:30.951 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:05:30.984 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-11-24T06:05:30.987 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:05:30.987 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-11-24T06:05:30.987 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:05:30.988 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:05:30.988 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:05:30.988 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:05:31.027 INFO:teuthology.orchestra.run.smithi183.stderr:bash: ntpq: command not found 2023-11-24T06:05:31.031 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:05:31.031 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2023-11-24T06:05:31.031 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:05:31.031 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:05:31.031 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:05:31.031 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:05:31.032 INFO:teuthology.run_tasks:Running task install... 2023-11-24T06:05:31.040 DEBUG:teuthology.task.install:project ceph 2023-11-24T06:05:31.040 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T06:05:31.040 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T06:05:31.040 INFO:teuthology.task.install:Using flavor: default 2023-11-24T06:05:31.046 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T06:05:31.046 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T06:05:31.046 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:05:31.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:05:31.048 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:05:31.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:05:31.418 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/ 2023-11-24T06:05:31.418 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:05:31.423 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/ 2023-11-24T06:05:31.423 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:05:31.529 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:05:31.529 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:05:31.530 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:05:31.536 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:05:31.536 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:05:31.536 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:05:31.558 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:05:31.558 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:05:31.564 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:05:31.565 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:05:31.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:05:31.634 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:05:31.704 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:05:31.713 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:05:31.769 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2023-11-24T06:05:31.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2023-11-24T06:05:31.779 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2023-11-24T06:05:31.781 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2023-11-24T06:05:32.140 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-11-24T06:05:32.152 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-11-24T06:05:32.391 INFO:teuthology.orchestra.run.smithi183.stdout:44 files removed 2023-11-24T06:05:32.407 INFO:teuthology.orchestra.run.smithi097.stdout:44 files removed 2023-11-24T06:05:32.437 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:05:32.454 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:05:32.766 INFO:teuthology.orchestra.run.smithi183.stdout:Updating Subscription Management repositories. 2023-11-24T06:05:32.783 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-11-24T06:05:33.547 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 192 kB/s | 81 kB 00:00 2023-11-24T06:05:33.774 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 122 kB/s | 81 kB 00:00 2023-11-24T06:05:34.033 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 42 kB/s | 9.5 kB 00:00 2023-11-24T06:05:34.164 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 16 kB/s | 9.5 kB 00:00 2023-11-24T06:05:34.626 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 2.6 kB/s | 1.5 kB 00:00 2023-11-24T06:05:34.790 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 2.5 kB/s | 1.5 kB 00:00 2023-11-24T06:05:34.868 INFO:teuthology.orchestra.run.smithi183.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-11-24T06:05:35.041 INFO:teuthology.orchestra.run.smithi097.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-11-24T06:05:37.103 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 16 MB 00:02 2023-11-24T06:05:40.708 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 333 kB/s | 24 kB 00:00 2023-11-24T06:05:41.631 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-11-24T06:05:44.611 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 1.6 MB/s | 16 MB 00:09 2023-11-24T06:05:48.159 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 395 kB/s | 24 kB 00:00 2023-11-24T06:05:49.044 INFO:teuthology.orchestra.run.smithi183.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-11-24T06:05:49.720 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-11-24T06:05:56.970 INFO:teuthology.orchestra.run.smithi183.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 60 MB/s | 53 MB 00:00 2023-11-24T06:06:00.570 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T06:06:00.571 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T06:06:00.585 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-24T06:06:00.586 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-24T06:06:00.587 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-24T06:06:00.588 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-24T06:06:00.590 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-24T06:06:00.594 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-24T06:06:00.595 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-24T06:06:00.726 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-11-24T06:06:00.738 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-11-24T06:06:00.738 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2023-11-24T06:06:00.738 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-11-24T06:06:00.738 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.4 k 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 5.4 M 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 920 k 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 171 k 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 1.6 M 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 133 k 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 1.8 M 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 7.4 M 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 50 k 2023-11-24T06:06:00.739 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 14 M 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 49 M 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 262 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 407 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 262 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 18 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 31 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 839 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 139 k 2023-11-24T06:06:00.740 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 197 k 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 371 k 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 364 k 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 107 k 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 94 k 2023-11-24T06:06:00.741 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.6 M 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 181 k 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.8 M 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.7 M 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 23 M 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 28 k 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 2.3 M 2023-11-24T06:06:00.742 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 243 k 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 4.6 M 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 18 M 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 15 k 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 24 k 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-24T06:06:00.743 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 183 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 517 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.9 M 2023-11-24T06:06:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-24T06:06:00.745 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 45 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 128 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-24T06:06:00.746 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-24T06:06:00.747 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T06:06:00.748 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-24T06:06:00.749 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling module streams: 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql 10 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-11-24T06:06:00.750 INFO:teuthology.orchestra.run.smithi097.stdout:Install 128 Packages 2023-11-24T06:06:00.751 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 3 Packages 2023-11-24T06:06:00.751 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:06:00.751 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 209 M 2023-11-24T06:06:00.751 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-11-24T06:06:01.178 INFO:teuthology.orchestra.run.smithi097.stdout:(1/131): ceph-18.0.0-7484.ga13ea5ff.el8.x86_64. 51 kB/s | 6.4 kB 00:00 2023-11-24T06:06:01.428 INFO:teuthology.orchestra.run.smithi097.stdout:(2/131): ceph-fuse-18.0.0-7484.ga13ea5ff.el8.x8 3.6 MB/s | 920 kB 00:00 2023-11-24T06:06:01.495 INFO:teuthology.orchestra.run.smithi097.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 2.5 MB/s | 171 kB 00:00 2023-11-24T06:06:01.854 INFO:teuthology.orchestra.run.smithi097.stdout:(4/131): ceph-mds-18.0.0-7484.ga13ea5ff.el8.x86 6.5 MB/s | 2.3 MB 00:00 2023-11-24T06:06:01.921 INFO:teuthology.orchestra.run.smithi097.stdout:(5/131): ceph-base-18.0.0-7484.ga13ea5ff.el8.x8 6.2 MB/s | 5.4 MB 00:00 2023-11-24T06:06:02.129 INFO:teuthology.orchestra.run.smithi097.stdout:(6/131): ceph-mgr-18.0.0-7484.ga13ea5ff.el8.x86 5.9 MB/s | 1.6 MB 00:00 2023-11-24T06:06:02.480 INFO:teuthology.orchestra.run.smithi097.stdout:(7/131): ceph-mon-18.0.0-7484.ga13ea5ff.el8.x86 8.3 MB/s | 4.6 MB 00:00 2023-11-24T06:06:03.480 INFO:teuthology.orchestra.run.smithi097.stdout:(8/131): ceph-common-18.0.0-7484.ga13ea5ff.el8. 9.3 MB/s | 23 MB 00:02 2023-11-24T06:06:03.547 INFO:teuthology.orchestra.run.smithi097.stdout:(9/131): ceph-selinux-18.0.0-7484.ga13ea5ff.el8 365 kB/s | 24 kB 00:00 2023-11-24T06:06:04.130 INFO:teuthology.orchestra.run.smithi097.stdout:(10/131): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 8.4 MB/s | 14 MB 00:01 2023-11-24T06:06:04.239 INFO:teuthology.orchestra.run.smithi097.stdout:(11/131): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.4 MB/s | 262 kB 00:00 2023-11-24T06:06:04.406 INFO:teuthology.orchestra.run.smithi097.stdout:(12/131): ceph-osd-18.0.0-7484.ga13ea5ff.el8.x8 7.9 MB/s | 18 MB 00:02 2023-11-24T06:06:04.431 INFO:teuthology.orchestra.run.smithi097.stdout:(13/131): libcephfs-devel-18.0.0-7484.ga13ea5ff 162 kB/s | 31 kB 00:00 2023-11-24T06:06:04.498 INFO:teuthology.orchestra.run.smithi097.stdout:(14/131): libcephsqlite-18.0.0-7484.ga13ea5ff.e 2.7 MB/s | 183 kB 00:00 2023-11-24T06:06:04.523 INFO:teuthology.orchestra.run.smithi097.stdout:(15/131): libcephfs2-18.0.0-7484.ga13ea5ff.el8. 7.0 MB/s | 839 kB 00:00 2023-11-24T06:06:04.557 INFO:teuthology.orchestra.run.smithi097.stdout:(16/131): librados-devel-18.0.0-7484.ga13ea5ff. 2.3 MB/s | 139 kB 00:00 2023-11-24T06:06:04.607 INFO:teuthology.orchestra.run.smithi097.stdout:(17/131): libradosstriper1-18.0.0-7484.ga13ea5f 6.0 MB/s | 517 kB 00:00 2023-11-24T06:06:04.707 INFO:teuthology.orchestra.run.smithi097.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.ga1 447 kB/s | 45 kB 00:00 2023-11-24T06:06:04.783 INFO:teuthology.orchestra.run.smithi097.stdout:(19/131): python3-ceph-common-18.0.0-7484.ga13e 1.7 MB/s | 128 kB 00:00 2023-11-24T06:06:04.891 INFO:teuthology.orchestra.run.smithi097.stdout:(20/131): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.8 MB/s | 197 kB 00:00 2023-11-24T06:06:04.983 INFO:teuthology.orchestra.run.smithi097.stdout:(21/131): python3-rados-18.0.0-7484.ga13ea5ff.e 4.0 MB/s | 371 kB 00:00 2023-11-24T06:06:05.083 INFO:teuthology.orchestra.run.smithi097.stdout:(22/131): python3-rbd-18.0.0-7484.ga13ea5ff.el8 3.6 MB/s | 364 kB 00:00 2023-11-24T06:06:05.192 INFO:teuthology.orchestra.run.smithi097.stdout:(23/131): python3-rgw-18.0.0-7484.ga13ea5ff.el8 984 kB/s | 107 kB 00:00 2023-11-24T06:06:05.326 INFO:teuthology.orchestra.run.smithi097.stdout:(24/131): librgw2-18.0.0-7484.ga13ea5ff.el8.x86 9.0 MB/s | 6.9 MB 00:00 2023-11-24T06:06:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:(25/131): rbd-fuse-18.0.0-7484.ga13ea5ff.el8.x8 590 kB/s | 94 kB 00:00 2023-11-24T06:06:05.435 INFO:teuthology.orchestra.run.smithi097.stdout:(26/131): rbd-nbd-18.0.0-7484.ga13ea5ff.el8.x86 2.1 MB/s | 181 kB 00:00 2023-11-24T06:06:05.502 INFO:teuthology.orchestra.run.smithi097.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7484.g 419 kB/s | 28 kB 00:00 2023-11-24T06:06:05.585 INFO:teuthology.orchestra.run.smithi097.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.6 MB/s | 133 kB 00:00 2023-11-24T06:06:05.736 INFO:teuthology.orchestra.run.smithi097.stdout:(29/131): rbd-mirror-18.0.0-7484.ga13ea5ff.el8. 8.8 MB/s | 3.6 MB 00:00 2023-11-24T06:06:05.886 INFO:teuthology.orchestra.run.smithi097.stdout:(30/131): ceph-mgr-dashboard-18.0.0-7484.ga13ea 5.9 MB/s | 1.8 MB 00:00 2023-11-24T06:06:05.986 INFO:teuthology.orchestra.run.smithi097.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.ga1 2.4 MB/s | 243 kB 00:00 2023-11-24T06:06:06.070 INFO:teuthology.orchestra.run.smithi097.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 601 kB/s | 50 kB 00:00 2023-11-24T06:06:06.154 INFO:teuthology.orchestra.run.smithi097.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7484.ga 180 kB/s | 15 kB 00:00 2023-11-24T06:06:06.245 INFO:teuthology.orchestra.run.smithi097.stdout:(34/131): ceph-volume-18.0.0-7484.ga13ea5ff.el8 2.8 MB/s | 262 kB 00:00 2023-11-24T06:06:06.354 INFO:teuthology.orchestra.run.smithi097.stdout:(35/131): cephadm-18.0.0-7484.ga13ea5ff.el8.noa 3.7 MB/s | 407 kB 00:00 2023-11-24T06:06:06.421 INFO:teuthology.orchestra.run.smithi097.stdout:(36/131): cephfs-top-18.0.0-7484.ga13ea5ff.el8. 272 kB/s | 18 kB 00:00 2023-11-24T06:06:06.546 INFO:teuthology.orchestra.run.smithi097.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 9.1 MB/s | 7.4 MB 00:00 2023-11-24T06:06:07.070 INFO:teuthology.orchestra.run.smithi097.stdout:(38/131): ceph-test-18.0.0-7484.ga13ea5ff.el8.x 14 MB/s | 49 MB 00:03 2023-11-24T06:06:07.239 INFO:teuthology.orchestra.run.smithi097.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 85 kB/s | 59 kB 00:00 2023-11-24T06:06:07.315 INFO:teuthology.orchestra.run.smithi097.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 310 kB/s | 75 kB 00:00 2023-11-24T06:06:07.381 INFO:teuthology.orchestra.run.smithi097.stdout:(41/131): gperftools-libs-2.7-9.el8.x86_64.rpm 319 kB/s | 306 kB 00:00 2023-11-24T06:06:07.398 INFO:teuthology.orchestra.run.smithi097.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 523 kB/s | 44 kB 00:00 2023-11-24T06:06:07.448 INFO:teuthology.orchestra.run.smithi097.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 661 kB/s | 33 kB 00:00 2023-11-24T06:06:07.502 INFO:teuthology.orchestra.run.smithi097.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 326 kB/s | 16 kB 00:00 2023-11-24T06:06:07.523 INFO:teuthology.orchestra.run.smithi097.stdout:(45/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-11-24T06:06:07.557 INFO:teuthology.orchestra.run.smithi097.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.5 MB/s | 476 kB 00:00 2023-11-24T06:06:07.665 INFO:teuthology.orchestra.run.smithi097.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 750 kB/s | 81 kB 00:00 2023-11-24T06:06:07.691 INFO:teuthology.orchestra.run.smithi097.stdout:(48/131): python3-cheroot-8.5.2-1.el8.noarch.rp 898 kB/s | 173 kB 00:00 2023-11-24T06:06:07.724 INFO:teuthology.orchestra.run.smithi097.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-11-24T06:06:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 300 kB/s | 18 kB 00:00 2023-11-24T06:06:07.791 INFO:teuthology.orchestra.run.smithi097.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.4 MB/s | 384 kB 00:00 2023-11-24T06:06:07.817 INFO:teuthology.orchestra.run.smithi097.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 727 kB/s | 49 kB 00:00 2023-11-24T06:06:07.867 INFO:teuthology.orchestra.run.smithi097.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 779 kB/s | 59 kB 00:00 2023-11-24T06:06:07.909 INFO:teuthology.orchestra.run.smithi097.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 654 kB/s | 60 kB 00:00 2023-11-24T06:06:07.968 INFO:teuthology.orchestra.run.smithi097.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 268 kB/s | 16 kB 00:00 2023-11-24T06:06:08.035 INFO:teuthology.orchestra.run.smithi097.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 505 kB/s | 34 kB 00:00 2023-11-24T06:06:08.102 INFO:teuthology.orchestra.run.smithi097.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.2 MB/s | 283 kB 00:00 2023-11-24T06:06:08.160 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T06:06:08.161 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T06:06:08.175 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-24T06:06:08.176 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-24T06:06:08.178 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-24T06:06:08.179 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-24T06:06:08.181 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-24T06:06:08.184 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-24T06:06:08.185 INFO:teuthology.orchestra.run.smithi097.stdout:(58/131): python3-rsa-4.9-2.el8.noarch.rpm 772 kB/s | 64 kB 00:00 2023-11-24T06:06:08.186 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-24T06:06:08.219 INFO:teuthology.orchestra.run.smithi097.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.0 MB/s | 196 kB 00:00 2023-11-24T06:06:08.261 INFO:teuthology.orchestra.run.smithi097.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2023-11-24T06:06:08.295 INFO:teuthology.orchestra.run.smithi097.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 318 kB/s | 24 kB 00:00 2023-11-24T06:06:08.317 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout:=========================================================================================================================== 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout:=========================================================================================================================== 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.4 k 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 5.4 M 2023-11-24T06:06:08.330 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 920 k 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 171 k 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 1.6 M 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 133 k 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 1.8 M 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 7.4 M 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 50 k 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 14 M 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 49 M 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 262 k 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 407 k 2023-11-24T06:06:08.331 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 262 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 18 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 31 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 839 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 139 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-24T06:06:08.332 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 197 k 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 371 k 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 364 k 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 107 k 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 94 k 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.6 M 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 181 k 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.8 M 2023-11-24T06:06:08.333 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.7 M 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 23 M 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 28 k 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 2.3 M 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 243 k 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 4.6 M 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 18 M 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 15 k 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 24 k 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-24T06:06:08.334 INFO:teuthology.orchestra.run.smithi183.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 183 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 517 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.9 M 2023-11-24T06:06:08.335 INFO:teuthology.orchestra.run.smithi183.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-24T06:06:08.336 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 45 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 128 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-24T06:06:08.337 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-24T06:06:08.338 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-24T06:06:08.339 INFO:teuthology.orchestra.run.smithi183.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-24T06:06:08.340 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-24T06:06:08.341 INFO:teuthology.orchestra.run.smithi183.stdout:Enabling module streams: 2023-11-24T06:06:08.341 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql 10 2023-11-24T06:06:08.341 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:06:08.341 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2023-11-24T06:06:08.341 INFO:teuthology.orchestra.run.smithi183.stdout:=========================================================================================================================== 2023-11-24T06:06:08.341 INFO:teuthology.orchestra.run.smithi183.stdout:Install 128 Packages 2023-11-24T06:06:08.341 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 3 Packages 2023-11-24T06:06:08.341 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:06:08.342 INFO:teuthology.orchestra.run.smithi097.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-11-24T06:06:08.342 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 209 M 2023-11-24T06:06:08.342 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2023-11-24T06:06:08.370 INFO:teuthology.orchestra.run.smithi097.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-11-24T06:06:08.571 INFO:teuthology.orchestra.run.smithi097.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.1 MB/s | 255 kB 00:00 2023-11-24T06:06:08.596 INFO:teuthology.orchestra.run.smithi097.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.1 MB/s | 251 kB 00:00 2023-11-24T06:06:08.611 INFO:teuthology.orchestra.run.smithi183.stdout:(1/131): ceph-18.0.0-7484.ga13ea5ff.el8.x86_64. 54 kB/s | 6.4 kB 00:00 2023-11-24T06:06:08.630 INFO:teuthology.orchestra.run.smithi097.stdout:(66/131): python3-kubernetes-11.0.0-6.el8.noarc 1.3 MB/s | 1.2 MB 00:00 2023-11-24T06:06:08.655 INFO:teuthology.orchestra.run.smithi097.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 733 kB/s | 61 kB 00:00 2023-11-24T06:06:08.697 INFO:teuthology.orchestra.run.smithi097.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 343 kB/s | 23 kB 00:00 2023-11-24T06:06:08.723 INFO:teuthology.orchestra.run.smithi097.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 678 kB/s | 85 kB 00:00 2023-11-24T06:06:08.764 INFO:teuthology.orchestra.run.smithi097.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.0 MB/s | 70 kB 00:00 2023-11-24T06:06:08.789 INFO:teuthology.orchestra.run.smithi097.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 704 kB/s | 47 kB 00:00 2023-11-24T06:06:08.839 INFO:teuthology.orchestra.run.smithi097.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 960 kB/s | 43 kB 00:00 2023-11-24T06:06:08.864 INFO:teuthology.orchestra.run.smithi097.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2023-11-24T06:06:08.890 INFO:teuthology.orchestra.run.smithi097.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-24T06:06:08.915 INFO:teuthology.orchestra.run.smithi097.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 959 kB/s | 48 kB 00:00 2023-11-24T06:06:08.940 INFO:teuthology.orchestra.run.smithi097.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-24T06:06:08.965 INFO:teuthology.orchestra.run.smithi097.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 851 kB/s | 43 kB 00:00 2023-11-24T06:06:08.982 INFO:teuthology.orchestra.run.smithi097.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-11-24T06:06:09.008 INFO:teuthology.orchestra.run.smithi097.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.3 MB/s | 229 kB 00:00 2023-11-24T06:06:09.058 INFO:teuthology.orchestra.run.smithi097.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-11-24T06:06:09.075 INFO:teuthology.orchestra.run.smithi097.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 279 kB/s | 19 kB 00:00 2023-11-24T06:06:09.100 INFO:teuthology.orchestra.run.smithi097.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-11-24T06:06:09.134 INFO:teuthology.orchestra.run.smithi097.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-11-24T06:06:09.151 INFO:teuthology.orchestra.run.smithi097.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 354 kB/s | 18 kB 00:00 2023-11-24T06:06:09.193 INFO:teuthology.orchestra.run.smithi097.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-11-24T06:06:09.218 INFO:teuthology.orchestra.run.smithi097.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-11-24T06:06:09.252 INFO:teuthology.orchestra.run.smithi097.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.1 MB/s | 72 kB 00:00 2023-11-24T06:06:09.311 INFO:teuthology.orchestra.run.smithi097.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 20 MB/s | 2.3 MB 00:00 2023-11-24T06:06:09.336 INFO:teuthology.orchestra.run.smithi097.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 991 kB/s | 83 kB 00:00 2023-11-24T06:06:09.361 INFO:teuthology.orchestra.run.smithi097.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-11-24T06:06:09.387 INFO:teuthology.orchestra.run.smithi097.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-11-24T06:06:09.411 INFO:teuthology.orchestra.run.smithi097.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-11-24T06:06:09.437 INFO:teuthology.orchestra.run.smithi097.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 772 kB/s | 39 kB 00:00 2023-11-24T06:06:09.470 INFO:teuthology.orchestra.run.smithi097.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-11-24T06:06:09.528 INFO:teuthology.orchestra.run.smithi097.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.9 MB/s | 171 kB 00:00 2023-11-24T06:06:09.545 INFO:teuthology.orchestra.run.smithi183.stdout:(2/131): ceph-fuse-18.0.0-7484.ga13ea5ff.el8.x8 986 kB/s | 920 kB 00:00 2023-11-24T06:06:09.563 INFO:teuthology.orchestra.run.smithi097.stdout:(96/131): NetworkManager-initscripts-updown-1.3 1.5 MB/s | 138 kB 00:00 2023-11-24T06:06:09.597 INFO:teuthology.orchestra.run.smithi097.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 3.2 MB/s | 110 kB 00:00 2023-11-24T06:06:09.622 INFO:teuthology.orchestra.run.smithi097.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-11-24T06:06:09.663 INFO:teuthology.orchestra.run.smithi097.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-24T06:06:09.705 INFO:teuthology.orchestra.run.smithi097.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-24T06:06:09.712 INFO:teuthology.orchestra.run.smithi183.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 1.0 MB/s | 171 kB 00:00 2023-11-24T06:06:09.731 INFO:teuthology.orchestra.run.smithi097.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.5 MB/s | 39 kB 00:00 2023-11-24T06:06:09.773 INFO:teuthology.orchestra.run.smithi097.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 16 MB/s | 688 kB 00:00 2023-11-24T06:06:09.815 INFO:teuthology.orchestra.run.smithi097.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-11-24T06:06:09.848 INFO:teuthology.orchestra.run.smithi097.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-11-24T06:06:10.015 INFO:teuthology.orchestra.run.smithi097.stdout:(105/131): python3-scipy-1.0.0-21.module+el8.5. 28 MB/s | 14 MB 00:00 2023-11-24T06:06:10.041 INFO:teuthology.orchestra.run.smithi097.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 977 kB/s | 187 kB 00:00 2023-11-24T06:06:10.074 INFO:teuthology.orchestra.run.smithi097.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-24T06:06:10.116 INFO:teuthology.orchestra.run.smithi097.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 640 kB/s | 27 kB 00:00 2023-11-24T06:06:10.158 INFO:teuthology.orchestra.run.smithi097.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-24T06:06:10.200 INFO:teuthology.orchestra.run.smithi097.stdout:(110/131): thrift-0.13.0-2.el8.x86_64.rpm 1.1 MB/s | 1.7 MB 00:01 2023-11-24T06:06:10.225 INFO:teuthology.orchestra.run.smithi097.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 656 kB/s | 44 kB 00:00 2023-11-24T06:06:10.301 INFO:teuthology.orchestra.run.smithi097.stdout:(112/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-11-24T06:06:10.334 INFO:teuthology.orchestra.run.smithi097.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 4.3 MB/s | 457 kB 00:00 2023-11-24T06:06:10.368 INFO:teuthology.orchestra.run.smithi097.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 3.3 MB/s | 225 kB 00:00 2023-11-24T06:06:10.393 INFO:teuthology.orchestra.run.smithi097.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-11-24T06:06:10.418 INFO:teuthology.orchestra.run.smithi097.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-24T06:06:10.444 INFO:teuthology.orchestra.run.smithi097.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-11-24T06:06:10.502 INFO:teuthology.orchestra.run.smithi097.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.2 MB/s | 103 kB 00:00 2023-11-24T06:06:10.536 INFO:teuthology.orchestra.run.smithi097.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-11-24T06:06:10.578 INFO:teuthology.orchestra.run.smithi097.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-24T06:06:10.636 INFO:teuthology.orchestra.run.smithi097.stdout:(121/131): python3-numpy-1.14.3-10.el8.x86_64.r 8.6 MB/s | 3.7 MB 00:00 2023-11-24T06:06:10.678 INFO:teuthology.orchestra.run.smithi097.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-24T06:06:10.720 INFO:teuthology.orchestra.run.smithi097.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-24T06:06:10.787 INFO:teuthology.orchestra.run.smithi097.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 17 MB/s | 4.9 MB 00:00 2023-11-24T06:06:10.838 INFO:teuthology.orchestra.run.smithi097.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-11-24T06:06:10.896 INFO:teuthology.orchestra.run.smithi183.stdout:(4/131): ceph-base-18.0.0-7484.ga13ea5ff.el8.x8 2.2 MB/s | 5.4 MB 00:02 2023-11-24T06:06:10.896 INFO:teuthology.orchestra.run.smithi097.stdout:(126/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-11-24T06:06:10.937 INFO:teuthology.orchestra.run.smithi183.stdout:(5/131): ceph-mds-18.0.0-7484.ga13ea5ff.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-11-24T06:06:10.946 INFO:teuthology.orchestra.run.smithi097.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-11-24T06:06:11.047 INFO:teuthology.orchestra.run.smithi097.stdout:(128/131): postgresql-server-10.21-2.module+el8 16 MB/s | 5.1 MB 00:00 2023-11-24T06:06:11.097 INFO:teuthology.orchestra.run.smithi097.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-24T06:06:11.113 INFO:teuthology.orchestra.run.smithi183.stdout:(6/131): ceph-mgr-18.0.0-7484.ga13ea5ff.el8.x86 7.5 MB/s | 1.6 MB 00:00 2023-11-24T06:06:11.539 INFO:teuthology.orchestra.run.smithi097.stdout:(130/131): librados2-18.0.0-7484.ga13ea5ff.el8. 5.9 MB/s | 3.8 MB 00:00 2023-11-24T06:06:11.622 INFO:teuthology.orchestra.run.smithi097.stdout:(131/131): librbd1-18.0.0-7484.ga13ea5ff.el8.x8 5.5 MB/s | 3.7 MB 00:00 2023-11-24T06:06:11.626 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:06:11.626 INFO:teuthology.orchestra.run.smithi097.stdout:Total 19 MB/s | 209 MB 00:10 2023-11-24T06:06:12.469 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-11-24T06:06:12.488 INFO:teuthology.orchestra.run.smithi183.stdout:(7/131): ceph-osd-18.0.0-7484.ga13ea5ff.el8.x86 13 MB/s | 18 MB 00:01 2023-11-24T06:06:12.572 INFO:teuthology.orchestra.run.smithi183.stdout:(8/131): ceph-mon-18.0.0-7484.ga13ea5ff.el8.x86 2.8 MB/s | 4.6 MB 00:01 2023-11-24T06:06:12.590 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-11-24T06:06:12.590 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-11-24T06:06:12.655 INFO:teuthology.orchestra.run.smithi183.stdout:(9/131): ceph-selinux-18.0.0-7484.ga13ea5ff.el8 293 kB/s | 24 kB 00:00 2023-11-24T06:06:13.222 INFO:teuthology.orchestra.run.smithi183.stdout:(10/131): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 19 MB/s | 14 MB 00:00 2023-11-24T06:06:13.307 INFO:teuthology.orchestra.run.smithi183.stdout:(11/131): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 3.0 MB/s | 262 kB 00:00 2023-11-24T06:06:13.356 INFO:teuthology.orchestra.run.smithi183.stdout:(12/131): libcephfs-devel-18.0.0-7484.ga13ea5ff 637 kB/s | 31 kB 00:00 2023-11-24T06:06:13.431 INFO:teuthology.orchestra.run.smithi183.stdout:(13/131): libcephfs2-18.0.0-7484.ga13ea5ff.el8. 11 MB/s | 839 kB 00:00 2023-11-24T06:06:13.482 INFO:teuthology.orchestra.run.smithi183.stdout:(14/131): libcephsqlite-18.0.0-7484.ga13ea5ff.e 3.6 MB/s | 183 kB 00:00 2023-11-24T06:06:13.532 INFO:teuthology.orchestra.run.smithi183.stdout:(15/131): librados-devel-18.0.0-7484.ga13ea5ff. 2.7 MB/s | 139 kB 00:00 2023-11-24T06:06:13.599 INFO:teuthology.orchestra.run.smithi183.stdout:(16/131): libradosstriper1-18.0.0-7484.ga13ea5f 7.6 MB/s | 517 kB 00:00 2023-11-24T06:06:13.723 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-11-24T06:06:13.727 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-11-24T06:06:13.983 INFO:teuthology.orchestra.run.smithi183.stdout:(17/131): librgw2-18.0.0-7484.ga13ea5ff.el8.x86 18 MB/s | 6.9 MB 00:00 2023-11-24T06:06:14.054 INFO:teuthology.orchestra.run.smithi183.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.ga1 634 kB/s | 45 kB 00:00 2023-11-24T06:06:14.108 INFO:teuthology.orchestra.run.smithi183.stdout:(19/131): python3-ceph-common-18.0.0-7484.ga13e 2.3 MB/s | 128 kB 00:00 2023-11-24T06:06:14.358 INFO:teuthology.orchestra.run.smithi183.stdout:(20/131): ceph-common-18.0.0-7484.ga13ea5ff.el8 3.8 MB/s | 23 MB 00:05 2023-11-24T06:06:14.383 INFO:teuthology.orchestra.run.smithi183.stdout:(21/131): python3-cephfs-18.0.0-7484.ga13ea5ff. 717 kB/s | 197 kB 00:00 2023-11-24T06:06:14.459 INFO:teuthology.orchestra.run.smithi183.stdout:(22/131): python3-rbd-18.0.0-7484.ga13ea5ff.el8 4.8 MB/s | 364 kB 00:00 2023-11-24T06:06:14.493 INFO:teuthology.orchestra.run.smithi183.stdout:(23/131): python3-rados-18.0.0-7484.ga13ea5ff.e 2.7 MB/s | 371 kB 00:00 2023-11-24T06:06:14.518 INFO:teuthology.orchestra.run.smithi183.stdout:(24/131): python3-rgw-18.0.0-7484.ga13ea5ff.el8 1.8 MB/s | 107 kB 00:00 2023-11-24T06:06:14.543 INFO:teuthology.orchestra.run.smithi183.stdout:(25/131): rbd-fuse-18.0.0-7484.ga13ea5ff.el8.x8 1.8 MB/s | 94 kB 00:00 2023-11-24T06:06:14.585 INFO:teuthology.orchestra.run.smithi183.stdout:(26/131): rbd-nbd-18.0.0-7484.ga13ea5ff.el8.x86 4.3 MB/s | 181 kB 00:00 2023-11-24T06:06:14.669 INFO:teuthology.orchestra.run.smithi183.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7484.g 335 kB/s | 28 kB 00:00 2023-11-24T06:06:14.736 INFO:teuthology.orchestra.run.smithi183.stdout:(28/131): rbd-mirror-18.0.0-7484.ga13ea5ff.el8. 16 MB/s | 3.6 MB 00:00 2023-11-24T06:06:14.761 INFO:teuthology.orchestra.run.smithi183.stdout:(29/131): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 1.4 MB/s | 133 kB 00:00 2023-11-24T06:06:14.861 INFO:teuthology.orchestra.run.smithi183.stdout:(30/131): ceph-mgr-dashboard-18.0.0-7484.ga13ea 14 MB/s | 1.8 MB 00:00 2023-11-24T06:06:14.911 INFO:teuthology.orchestra.run.smithi183.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.ga1 4.8 MB/s | 243 kB 00:00 2023-11-24T06:06:14.962 INFO:teuthology.orchestra.run.smithi183.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 1.0 MB/s | 50 kB 00:00 2023-11-24T06:06:15.004 INFO:teuthology.orchestra.run.smithi183.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7484.ga 359 kB/s | 15 kB 00:00 2023-11-24T06:06:15.063 INFO:teuthology.orchestra.run.smithi183.stdout:(34/131): ceph-volume-18.0.0-7484.ga13ea5ff.el8 4.4 MB/s | 262 kB 00:00 2023-11-24T06:06:15.121 INFO:teuthology.orchestra.run.smithi183.stdout:(35/131): cephadm-18.0.0-7484.ga13ea5ff.el8.noa 6.8 MB/s | 407 kB 00:00 2023-11-24T06:06:15.188 INFO:teuthology.orchestra.run.smithi183.stdout:(36/131): cephfs-top-18.0.0-7484.ga13ea5ff.el8. 272 kB/s | 18 kB 00:00 2023-11-24T06:06:15.463 INFO:teuthology.orchestra.run.smithi183.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-11-24T06:06:15.556 INFO:teuthology.orchestra.run.smithi183.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 642 kB/s | 59 kB 00:00 2023-11-24T06:06:15.614 INFO:teuthology.orchestra.run.smithi183.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-11-24T06:06:15.723 INFO:teuthology.orchestra.run.smithi183.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2023-11-24T06:06:15.790 INFO:teuthology.orchestra.run.smithi183.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 656 kB/s | 44 kB 00:00 2023-11-24T06:06:15.873 INFO:teuthology.orchestra.run.smithi183.stdout:(42/131): ceph-mgr-diskprediction-local-18.0.0- 6.7 MB/s | 7.4 MB 00:01 2023-11-24T06:06:15.948 INFO:teuthology.orchestra.run.smithi183.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-11-24T06:06:16.024 INFO:teuthology.orchestra.run.smithi183.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 217 kB/s | 16 kB 00:00 2023-11-24T06:06:16.066 INFO:teuthology.orchestra.run.smithi183.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 172 kB/s | 33 kB 00:00 2023-11-24T06:06:16.099 INFO:teuthology.orchestra.run.smithi183.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-11-24T06:06:16.175 INFO:teuthology.orchestra.run.smithi183.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2023-11-24T06:06:16.250 INFO:teuthology.orchestra.run.smithi183.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-11-24T06:06:16.309 INFO:teuthology.orchestra.run.smithi183.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.5 MB/s | 384 kB 00:00 2023-11-24T06:06:16.334 INFO:teuthology.orchestra.run.smithi183.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-11-24T06:06:16.409 INFO:teuthology.orchestra.run.smithi183.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 650 kB/s | 49 kB 00:00 2023-11-24T06:06:16.485 INFO:teuthology.orchestra.run.smithi183.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 782 kB/s | 59 kB 00:00 2023-11-24T06:06:16.577 INFO:teuthology.orchestra.run.smithi183.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 4.4 MB/s | 1.2 MB 00:00 2023-11-24T06:06:16.593 INFO:teuthology.orchestra.run.smithi183.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 556 kB/s | 60 kB 00:00 2023-11-24T06:06:16.685 INFO:teuthology.orchestra.run.smithi183.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-11-24T06:06:16.702 INFO:teuthology.orchestra.run.smithi183.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-11-24T06:06:16.753 INFO:teuthology.orchestra.run.smithi183.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 503 kB/s | 34 kB 00:00 2023-11-24T06:06:16.786 INFO:teuthology.orchestra.run.smithi183.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.3 MB/s | 196 kB 00:00 2023-11-24T06:06:16.828 INFO:teuthology.orchestra.run.smithi183.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 855 kB/s | 64 kB 00:00 2023-11-24T06:06:16.870 INFO:teuthology.orchestra.run.smithi183.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 224 kB/s | 19 kB 00:00 2023-11-24T06:06:16.912 INFO:teuthology.orchestra.run.smithi183.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 286 kB/s | 24 kB 00:00 2023-11-24T06:06:16.954 INFO:teuthology.orchestra.run.smithi183.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 348 kB/s | 29 kB 00:00 2023-11-24T06:06:17.163 INFO:teuthology.orchestra.run.smithi183.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 110 kB/s | 27 kB 00:00 2023-11-24T06:06:17.238 INFO:teuthology.orchestra.run.smithi183.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.3 MB/s | 251 kB 00:00 2023-11-24T06:06:17.313 INFO:teuthology.orchestra.run.smithi183.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 818 kB/s | 61 kB 00:00 2023-11-24T06:06:17.414 INFO:teuthology.orchestra.run.smithi183.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2023-11-24T06:06:17.439 INFO:teuthology.orchestra.run.smithi183.stdout:(67/131): python3-waitress-1.4.3-1.el8.noarch.r 527 kB/s | 255 kB 00:00 2023-11-24T06:06:17.489 INFO:teuthology.orchestra.run.smithi183.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-11-24T06:06:17.515 INFO:teuthology.orchestra.run.smithi183.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-24T06:06:17.840 INFO:teuthology.orchestra.run.smithi183.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 42 MB/s | 14 MB 00:00 2023-11-24T06:06:17.882 INFO:teuthology.orchestra.run.smithi183.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-11-24T06:06:17.924 INFO:teuthology.orchestra.run.smithi183.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-24T06:06:17.966 INFO:teuthology.orchestra.run.smithi183.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-11-24T06:06:18.008 INFO:teuthology.orchestra.run.smithi183.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 3.1 MB/s | 1.7 MB 00:00 2023-11-24T06:06:18.041 INFO:teuthology.orchestra.run.smithi183.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-11-24T06:06:18.092 INFO:teuthology.orchestra.run.smithi183.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 464 kB/s | 39 kB 00:00 2023-11-24T06:06:18.125 INFO:teuthology.orchestra.run.smithi183.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 8.6 MB/s | 688 kB 00:00 2023-11-24T06:06:18.159 INFO:teuthology.orchestra.run.smithi183.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-11-24T06:06:18.217 INFO:teuthology.orchestra.run.smithi183.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2023-11-24T06:06:18.243 INFO:teuthology.orchestra.run.smithi183.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-11-24T06:06:18.276 INFO:teuthology.orchestra.run.smithi183.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-24T06:06:18.310 INFO:teuthology.orchestra.run.smithi183.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 797 kB/s | 27 kB 00:00 2023-11-24T06:06:18.352 INFO:teuthology.orchestra.run.smithi183.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-24T06:06:18.427 INFO:teuthology.orchestra.run.smithi183.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-11-24T06:06:18.453 INFO:teuthology.orchestra.run.smithi183.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 439 kB/s | 44 kB 00:00 2023-11-24T06:06:18.495 INFO:teuthology.orchestra.run.smithi183.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-11-24T06:06:18.528 INFO:teuthology.orchestra.run.smithi183.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 6.6 MB/s | 225 kB 00:00 2023-11-24T06:06:18.603 INFO:teuthology.orchestra.run.smithi183.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 21 MB/s | 3.7 MB 00:00 2023-11-24T06:06:18.620 INFO:teuthology.orchestra.run.smithi183.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-11-24T06:06:18.646 INFO:teuthology.orchestra.run.smithi183.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.9 MB/s | 126 kB 00:00 2023-11-24T06:06:18.671 INFO:teuthology.orchestra.run.smithi183.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 396 kB/s | 20 kB 00:00 2023-11-24T06:06:18.688 INFO:teuthology.orchestra.run.smithi183.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-11-24T06:06:18.722 INFO:teuthology.orchestra.run.smithi183.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 14 MB/s | 709 kB 00:00 2023-11-24T06:06:18.722 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-11-24T06:06:18.764 INFO:teuthology.orchestra.run.smithi183.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-24T06:06:18.856 INFO:teuthology.orchestra.run.smithi183.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 29 MB/s | 4.9 MB 00:00 2023-11-24T06:06:18.894 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-24T06:06:18.897 INFO:teuthology.orchestra.run.smithi183.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-24T06:06:18.990 INFO:teuthology.orchestra.run.smithi183.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-11-24T06:06:19.015 INFO:teuthology.orchestra.run.smithi183.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-11-24T06:06:19.070 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-24T06:06:19.073 INFO:teuthology.orchestra.run.smithi183.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-11-24T06:06:19.165 INFO:teuthology.orchestra.run.smithi183.stdout:(100/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-11-24T06:06:19.207 INFO:teuthology.orchestra.run.smithi183.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-11-24T06:06:19.232 INFO:teuthology.orchestra.run.smithi183.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 700 kB/s | 47 kB 00:00 2023-11-24T06:06:19.249 INFO:teuthology.orchestra.run.smithi183.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 4.6 MB/s | 200 kB 00:00 2023-11-24T06:06:19.255 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T06:06:19.306 INFO:teuthology.orchestra.run.smithi183.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 580 kB/s | 43 kB 00:00 2023-11-24T06:06:19.342 INFO:teuthology.orchestra.run.smithi183.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-11-24T06:06:19.367 INFO:teuthology.orchestra.run.smithi183.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 795 kB/s | 48 kB 00:00 2023-11-24T06:06:19.376 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T06:06:19.392 INFO:teuthology.orchestra.run.smithi183.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-24T06:06:19.418 INFO:teuthology.orchestra.run.smithi183.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-11-24T06:06:19.443 INFO:teuthology.orchestra.run.smithi183.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-24T06:06:19.468 INFO:teuthology.orchestra.run.smithi183.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-11-24T06:06:19.502 INFO:teuthology.orchestra.run.smithi183.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-11-24T06:06:19.527 INFO:teuthology.orchestra.run.smithi183.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 320 kB/s | 19 kB 00:00 2023-11-24T06:06:19.552 INFO:teuthology.orchestra.run.smithi183.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-24T06:06:19.586 INFO:teuthology.orchestra.run.smithi183.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-11-24T06:06:19.596 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T06:06:19.611 INFO:teuthology.orchestra.run.smithi183.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-11-24T06:06:19.676 INFO:teuthology.orchestra.run.smithi183.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.6 MB/s | 605 kB 00:00 2023-11-24T06:06:19.712 INFO:teuthology.orchestra.run.smithi183.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-11-24T06:06:19.745 INFO:teuthology.orchestra.run.smithi183.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 2.1 MB/s | 72 kB 00:00 2023-11-24T06:06:19.804 INFO:teuthology.orchestra.run.smithi183.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 18 MB/s | 2.3 MB 00:00 2023-11-24T06:06:19.830 INFO:teuthology.orchestra.run.smithi183.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 985 kB/s | 83 kB 00:00 2023-11-24T06:06:19.846 INFO:teuthology.orchestra.run.smithi183.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-11-24T06:06:19.872 INFO:teuthology.orchestra.run.smithi183.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-11-24T06:06:19.889 INFO:teuthology.orchestra.run.smithi183.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-11-24T06:06:19.906 INFO:teuthology.orchestra.run.smithi183.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-11-24T06:06:19.939 INFO:teuthology.orchestra.run.smithi183.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 12 MB/s | 644 kB 00:00 2023-11-24T06:06:19.964 INFO:teuthology.orchestra.run.smithi183.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-11-24T06:06:19.990 INFO:teuthology.orchestra.run.smithi183.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-24T06:06:20.031 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T06:06:20.305 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T06:06:20.315 INFO:teuthology.orchestra.run.smithi183.stdout:(128/131): librados2-18.0.0-7484.ga13ea5ff.el8. 11 MB/s | 3.8 MB 00:00 2023-11-24T06:06:20.374 INFO:teuthology.orchestra.run.smithi183.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-24T06:06:20.632 INFO:teuthology.orchestra.run.smithi183.stdout:(130/131): librbd1-18.0.0-7484.ga13ea5ff.el8.x8 5.8 MB/s | 3.7 MB 00:00 2023-11-24T06:06:20.703 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T06:06:20.888 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:06:21.002 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:06:21.213 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:06:21.311 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:06:21.468 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:06:21.680 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-24T06:06:21.824 INFO:teuthology.orchestra.run.smithi183.stdout:(131/131): ceph-test-18.0.0-7484.ga13ea5ff.el8. 5.4 MB/s | 49 MB 00:09 2023-11-24T06:06:21.828 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:06:21.828 INFO:teuthology.orchestra.run.smithi183.stdout:Total 16 MB/s | 209 MB 00:13 2023-11-24T06:06:21.908 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-24T06:06:22.140 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-24T06:06:22.539 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-24T06:06:22.679 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2023-11-24T06:06:22.798 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2023-11-24T06:06:22.799 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2023-11-24T06:06:22.803 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-24T06:06:23.263 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T06:06:23.372 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T06:06:23.530 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T06:06:23.700 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T06:06:23.884 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-24T06:06:23.931 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2023-11-24T06:06:23.935 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2023-11-24T06:06:24.036 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-24T06:06:24.219 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-24T06:06:24.337 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T06:06:24.483 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T06:06:24.654 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T06:06:25.267 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T06:06:25.446 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T06:06:26.058 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T06:06:26.229 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T06:06:26.920 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T06:06:27.190 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-24T06:06:27.352 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-24T06:06:27.589 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-24T06:06:27.735 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 26/134 2023-11-24T06:06:27.856 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 27/134 2023-11-24T06:06:28.108 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 28/134 2023-11-24T06:06:28.287 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T06:06:29.411 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2023-11-24T06:06:29.601 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-24T06:06:29.779 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-24T06:06:29.976 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-24T06:06:30.033 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T06:06:30.171 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-24T06:06:30.289 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-24T06:06:30.512 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-24T06:06:30.902 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-24T06:06:31.028 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-24T06:06:31.186 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:06:31.342 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-24T06:06:31.613 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:06:31.622 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-24T06:06:31.811 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:06:31.824 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-24T06:06:31.929 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:06:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-24T06:06:32.145 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:06:32.214 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-24T06:06:32.238 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:06:32.409 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 36/134 2023-11-24T06:06:32.512 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T06:06:32.568 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 37/134 2023-11-24T06:06:32.710 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-24T06:06:32.732 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-24T06:06:32.879 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-24T06:06:32.939 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-24T06:06:33.053 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T06:06:33.126 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T06:06:33.325 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-24T06:06:33.338 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-24T06:06:33.548 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-24T06:06:33.601 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-24T06:06:33.686 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T06:06:33.755 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T06:06:33.794 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T06:06:33.920 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T06:06:33.973 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-24T06:06:34.094 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T06:06:34.132 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-24T06:06:34.183 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T06:06:34.321 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-24T06:06:34.338 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-24T06:06:34.474 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-24T06:06:34.487 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-24T06:06:34.638 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-24T06:06:34.644 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-24T06:06:34.822 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-24T06:06:34.999 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-24T06:06:35.172 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-24T06:06:35.242 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-24T06:06:35.436 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-24T06:06:35.544 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-24T06:06:35.846 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-24T06:06:36.017 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-24T06:06:36.047 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-24T06:06:36.244 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-24T06:06:36.323 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-24T06:06:36.532 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-24T06:06:36.860 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-24T06:06:36.935 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-24T06:06:37.086 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-24T06:06:37.233 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-24T06:06:37.312 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-24T06:06:37.436 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-24T06:06:37.499 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-24T06:06:37.709 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-24T06:06:37.832 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-24T06:06:37.872 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-24T06:06:38.034 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-24T06:06:38.159 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-24T06:06:38.236 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-24T06:06:38.423 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-24T06:06:38.498 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-24T06:06:38.667 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 26/134 2023-11-24T06:06:38.668 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-24T06:06:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 27/134 2023-11-24T06:06:38.920 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-24T06:06:39.060 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-24T06:06:39.280 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-24T06:06:39.732 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 28/134 2023-11-24T06:06:39.841 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-24T06:06:40.017 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-24T06:06:40.073 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-24T06:06:40.198 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-24T06:06:40.339 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-24T06:06:40.361 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-24T06:06:40.520 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-24T06:06:40.593 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-24T06:06:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-24T06:06:40.785 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-24T06:06:40.921 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-24T06:06:41.088 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-24T06:06:41.216 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-24T06:06:41.432 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-24T06:06:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-24T06:06:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-24T06:06:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-24T06:06:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-24T06:06:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-24T06:06:42.538 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-24T06:06:42.716 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-24T06:06:42.877 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-24T06:06:43.071 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-24T06:06:43.278 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-24T06:06:43.449 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 36/134 2023-11-24T06:06:43.622 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 37/134 2023-11-24T06:06:43.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-24T06:06:43.803 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-24T06:06:43.985 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-24T06:06:44.081 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-24T06:06:44.145 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-24T06:06:44.284 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-24T06:06:44.288 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-24T06:06:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-24T06:06:44.602 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-24T06:06:44.743 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-24T06:06:44.853 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-24T06:06:45.095 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-24T06:06:45.121 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-24T06:06:45.387 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-24T06:06:45.628 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-24T06:06:45.840 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-24T06:06:45.936 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-24T06:06:46.039 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-24T06:06:46.101 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T06:06:46.167 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T06:06:46.327 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 93/134 2023-11-24T06:06:46.368 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-24T06:06:46.666 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-24T06:06:46.843 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-24T06:06:47.097 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-24T06:06:47.258 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-24T06:06:47.445 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-24T06:06:47.762 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-24T06:06:47.996 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-24T06:06:48.246 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-24T06:06:48.540 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-24T06:06:48.716 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-24T06:06:48.911 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-24T06:06:49.108 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-24T06:06:49.184 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-24T06:06:49.435 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-24T06:06:49.612 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-24T06:06:49.767 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-24T06:06:49.845 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-24T06:06:50.081 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-24T06:06:50.113 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:06:50.251 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-24T06:06:50.434 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-24T06:06:50.460 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:06:50.714 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-24T06:06:51.024 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-24T06:06:51.055 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:06:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-24T06:06:51.372 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T06:06:51.429 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-24T06:06:51.453 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T06:06:51.584 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T06:06:51.620 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-24T06:06:51.818 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-24T06:06:52.030 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-24T06:06:52.220 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-24T06:06:52.411 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-24T06:06:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-24T06:06:52.790 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-24T06:06:53.003 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-24T06:06:53.172 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-24T06:06:53.488 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-24T06:06:53.749 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-24T06:06:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-24T06:06:54.163 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-24T06:06:55.239 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-24T06:06:55.502 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-24T06:06:55.757 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-24T06:06:56.032 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-24T06:06:56.298 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-24T06:06:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-24T06:06:57.314 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-24T06:06:57.515 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T06:06:57.597 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T06:06:57.783 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 93/134 2023-11-24T06:07:01.798 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:07:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:07:02.634 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T06:07:02.928 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T06:07:03.010 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T06:07:03.198 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T06:07:10.839 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /run 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-11-24T06:07:10.840 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:11.215 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T06:07:12.808 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T06:07:12.986 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T06:07:13.210 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T06:07:13.348 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T06:07:13.804 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T06:07:14.048 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T06:07:14.065 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T06:07:15.751 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T06:07:15.963 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T06:07:16.080 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 102/134 2023-11-24T06:07:16.255 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 103/134 2023-11-24T06:07:16.472 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T06:07:16.692 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T06:07:16.968 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T06:07:17.929 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T06:07:18.081 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T06:07:18.171 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T06:07:18.545 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 107/134 2023-11-24T06:07:18.702 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T06:07:18.739 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T06:07:18.947 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T06:07:18.958 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T06:07:20.275 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 110/134 2023-11-24T06:07:20.516 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T06:07:20.549 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T06:07:20.697 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T06:07:20.739 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T06:07:20.905 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T06:07:21.264 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T06:07:21.472 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T06:07:22.139 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T06:07:22.139 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2023-11-24T06:07:22.139 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2023-11-24T06:07:22.139 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /dev 2023-11-24T06:07:22.139 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /run 2023-11-24T06:07:22.140 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2023-11-24T06:07:22.140 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2023-11-24T06:07:22.140 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2023-11-24T06:07:22.140 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2023-11-24T06:07:22.140 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /dev 2023-11-24T06:07:22.140 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:22.511 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T06:07:24.115 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T06:07:24.299 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T06:07:24.529 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T06:07:24.677 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T06:07:25.133 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T06:07:25.335 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T06:07:25.352 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T06:07:26.147 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T06:07:26.991 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 115/134 2023-11-24T06:07:27.138 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T06:07:27.330 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-24T06:07:27.359 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T06:07:27.484 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 102/134 2023-11-24T06:07:27.502 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 117/134 2023-11-24T06:07:27.675 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 103/134 2023-11-24T06:07:27.774 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-24T06:07:27.885 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T06:07:27.921 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T06:07:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T06:07:28.112 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T06:07:28.333 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-24T06:07:28.431 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T06:07:28.525 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 121/134 2023-11-24T06:07:28.672 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T06:07:29.390 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T06:07:29.593 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T06:07:29.683 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T06:07:30.081 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 107/134 2023-11-24T06:07:30.306 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T06:07:30.343 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T06:07:30.509 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T06:07:30.520 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T06:07:30.933 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:07:31.179 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:07:31.196 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:07:31.346 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 124/134 2023-11-24T06:07:31.637 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 125/134 2023-11-24T06:07:31.838 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 110/134 2023-11-24T06:07:31.922 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 126/134 2023-11-24T06:07:32.036 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T06:07:32.070 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T06:07:32.251 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T06:07:32.293 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T06:07:32.294 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-24T06:07:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T06:07:32.512 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T06:07:32.591 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T06:07:32.785 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T06:07:32.812 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T06:07:32.825 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T06:07:32.984 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T06:07:33.012 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T06:07:33.020 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T06:07:33.164 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-24T06:07:33.164 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:07:33.398 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:07:33.398 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T06:07:33.625 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T06:07:33.819 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:07:34.342 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 1/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 3/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 4/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 10/134 2023-11-24T06:07:34.343 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 11/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 12/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 13/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 14/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 15/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 16/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 17/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 18/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 19/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 21/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 22/134 2023-11-24T06:07:34.344 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 23/134 2023-11-24T06:07:34.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 24/134 2023-11-24T06:07:34.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 25/134 2023-11-24T06:07:34.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 26/134 2023-11-24T06:07:34.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 27/134 2023-11-24T06:07:34.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 28/134 2023-11-24T06:07:34.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/134 2023-11-24T06:07:34.345 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 30/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 31/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 34/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 36/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 37/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 38/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-24T06:07:34.354 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-24T06:07:34.355 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-24T06:07:34.356 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-24T06:07:34.357 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-24T06:07:34.358 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-24T06:07:34.359 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-24T06:07:34.360 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-24T06:07:34.361 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 129/134 2023-11-24T06:07:34.362 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-24T06:07:34.362 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 131/134 2023-11-24T06:07:34.362 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:07:34.362 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-24T06:07:36.917 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:07:36.917 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:47 ago on Fri 24 Nov 2023 06:05:49 AM UTC. 2023-11-24T06:07:37.673 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T06:07:37.986 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 115/134 2023-11-24T06:07:38.166 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-24T06:07:38.356 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-24T06:07:39.241 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-24T06:07:39.570 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-24T06:07:39.583 INFO:teuthology.orchestra.run.smithi097.stdout:Installed products updated. 2023-11-24T06:07:39.789 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 119/134 2023-11-24T06:07:39.860 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:39.860 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2023-11-24T06:07:39.861 INFO:teuthology.orchestra.run.smithi097.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-24T06:07:39.861 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.861 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.861 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2023-11-24T06:07:39.861 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-24T06:07:39.861 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.1-7.el8.noarch 2023-11-24T06:07:39.862 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.0.4-10.el8.x86_64 2023-11-24T06:07:39.862 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.862 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.862 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.862 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.862 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.863 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.863 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.863 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.863 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.863 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.863 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.864 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.864 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.864 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.864 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.864 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.864 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.865 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.865 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.865 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.865 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:39.866 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-24T06:07:39.866 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.1-9.el8.x86_64 2023-11-24T06:07:39.866 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-24T06:07:39.866 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-24T06:07:39.866 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-3.el8.x86_64 2023-11-24T06:07:39.866 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-24T06:07:39.866 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-24T06:07:39.866 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-24T06:07:39.867 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.867 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.867 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.867 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-24T06:07:39.867 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-24T06:07:39.867 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-24T06:07:39.868 INFO:teuthology.orchestra.run.smithi097.stdout: libpq-13.5-1.el8.x86_64 2023-11-24T06:07:39.868 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-24T06:07:39.868 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-24T06:07:39.868 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.868 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.868 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-24T06:07:39.868 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-24T06:07:39.869 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-24T06:07:39.870 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-24T06:07:39.871 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-2.6-1.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-24T06:07:39.872 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-24T06:07:39.873 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-24T06:07:39.874 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-24T06:07:39.875 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-24T06:07:39.875 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:39.875 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-11-24T06:07:40.020 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-24T06:07:40.080 DEBUG:teuthology.parallel:result is None 2023-11-24T06:07:40.178 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 121/134 2023-11-24T06:07:40.309 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T06:07:42.544 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:07:42.785 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:07:42.801 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T06:07:42.983 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 124/134 2023-11-24T06:07:43.257 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 125/134 2023-11-24T06:07:43.600 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 126/134 2023-11-24T06:07:43.950 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-24T06:07:44.124 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-24T06:07:44.132 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-24T06:07:44.300 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-24T06:07:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-24T06:07:44.584 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-24T06:07:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-24T06:07:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-24T06:07:44.836 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:07:45.010 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:07:45.010 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T06:07:45.253 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T06:07:45.447 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:07:45.987 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 1/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 3/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 4/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 10/134 2023-11-24T06:07:45.988 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 11/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 12/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 13/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 14/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 15/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 16/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 17/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 18/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 19/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 21/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 22/134 2023-11-24T06:07:45.989 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 23/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 24/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 25/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 26/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 27/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 28/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 30/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 31/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/134 2023-11-24T06:07:45.990 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 34/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 36/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 37/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 38/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-24T06:07:45.991 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-24T06:07:45.992 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-24T06:07:45.993 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-24T06:07:45.994 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-24T06:07:45.995 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-24T06:07:45.996 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-24T06:07:45.997 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-24T06:07:45.998 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-24T06:07:45.999 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 129/134 2023-11-24T06:07:45.999 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-24T06:07:45.999 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 131/134 2023-11-24T06:07:45.999 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T06:07:45.999 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-24T06:07:48.564 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T06:07:48.564 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:52 ago on Fri 24 Nov 2023 06:05:56 AM UTC. 2023-11-24T06:07:51.124 INFO:teuthology.orchestra.run.smithi183.stdout:Installed products updated. 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.1-7.el8.noarch 2023-11-24T06:07:51.395 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.0.4-10.el8.x86_64 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.396 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.397 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.1-9.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: jq-1.6-3.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.398 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.399 INFO:teuthology.orchestra.run.smithi183.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-24T06:07:51.399 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-24T06:07:51.399 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-24T06:07:51.399 INFO:teuthology.orchestra.run.smithi183.stdout: libpq-13.5-1.el8.x86_64 2023-11-24T06:07:51.399 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-24T06:07:51.399 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-24T06:07:51.399 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-24T06:07:51.401 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T06:07:51.402 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-24T06:07:51.403 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-24T06:07:51.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-2.6-1.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-24T06:07:51.405 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-24T06:07:51.406 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-24T06:07:51.407 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-24T06:07:51.408 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-24T06:07:51.408 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-24T06:07:51.408 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-24T06:07:51.408 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:51.408 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2023-11-24T06:07:51.689 DEBUG:teuthology.parallel:result is None 2023-11-24T06:07:51.689 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:07:52.029 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:07:52.051 INFO:teuthology.orchestra.run.smithi097.stdout:18.0.0-7484.ga13ea5ff.el8 2023-11-24T06:07:52.051 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el8 2023-11-24T06:07:52.051 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:07:52.052 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:07:52.378 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:07:52.400 INFO:teuthology.orchestra.run.smithi183.stdout:18.0.0-7484.ga13ea5ff.el8 2023-11-24T06:07:52.400 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el8 2023-11-24T06:07:52.400 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:07:52.401 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T06:07:52.401 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:52.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:07:52.443 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:07:52.444 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:07:52.484 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T06:07:52.485 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:52.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:07:52.511 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:07:52.578 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:07:52.578 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:07:52.605 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:07:52.671 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T06:07:52.671 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:52.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:07:52.697 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:07:52.762 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:07:52.763 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:07:52.789 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:07:52.856 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T06:07:52.856 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:52.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:07:52.882 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:07:52.948 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:07:52.948 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:07:52.974 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:07:53.040 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T06:07:53.122 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T06:07:53.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2023-11-24T06:07:53.124 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /var/log/ceph 2023-11-24T06:07:53.150 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T06:07:53.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:07:53.191 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:07:53.221 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T06:07:53.221 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:07:53.264 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:07:53.295 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T06:07:53.295 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T06:07:53.295 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T06:07:53.296 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:53.296 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:07:53.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:07:53.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:07:53.629 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:07:53.629 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:07:53.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:07:53.788 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:07:53.819 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T06:07:53.819 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T06:07:53.819 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}', 'email': None, 'first_in_suite': False, 'job_id': '7465550', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3305, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi183.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_05:41:33', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T06:07:53.820 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:07:53.822 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:07:53.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:07:53.903 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:07:53.936 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:53.936 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:07:53.983 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:07:53.983 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2023-11-24T06:07:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:07:54.040 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:07:54.040 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 389566 Links: 1 2023-11-24T06:07:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:07:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:07:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-11-24 06:07:34.041637717 +0000 2023-11-24T06:07:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-11-24 06:04:51.731175794 +0000 2023-11-24T06:07:54.041 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-11-24 06:04:51.731175794 +0000 2023-11-24T06:07:54.041 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-11-24T06:07:54.041 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:07:54.105 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-11-24T06:07:54.106 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-11-24T06:07:54.106 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000251157 s, 2.0 MB/s 2023-11-24T06:07:54.107 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:07:54.164 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2023-11-24T06:07:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:07:54.221 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:07:54.221 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 390886 Links: 1 2023-11-24T06:07:54.222 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:07:54.222 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:07:54.222 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-11-24 06:07:34.042637700 +0000 2023-11-24T06:07:54.222 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-11-24 06:04:52.249166941 +0000 2023-11-24T06:07:54.222 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-11-24 06:04:52.249166941 +0000 2023-11-24T06:07:54.222 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-11-24T06:07:54.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:07:54.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-11-24T06:07:54.286 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-11-24T06:07:54.287 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000212086 s, 2.4 MB/s 2023-11-24T06:07:54.288 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:07:54.344 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2023-11-24T06:07:54.402 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:07:54.402 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:07:54.402 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 391811 Links: 1 2023-11-24T06:07:54.402 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:07:54.403 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:07:54.403 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-11-24 06:07:34.042637700 +0000 2023-11-24T06:07:54.403 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-11-24 06:04:52.753158327 +0000 2023-11-24T06:07:54.403 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-11-24 06:04:52.753158327 +0000 2023-11-24T06:07:54.403 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-11-24T06:07:54.403 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:07:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-11-24T06:07:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-11-24T06:07:54.467 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000222831 s, 2.3 MB/s 2023-11-24T06:07:54.468 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:07:54.525 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2023-11-24T06:07:54.582 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:07:54.582 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:07:54.582 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 392329 Links: 1 2023-11-24T06:07:54.583 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:07:54.583 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:07:54.583 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-11-24 06:07:34.042637700 +0000 2023-11-24T06:07:54.583 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-11-24 06:04:53.246149901 +0000 2023-11-24T06:07:54.583 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-11-24 06:04:53.246149901 +0000 2023-11-24T06:07:54.583 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-11-24T06:07:54.583 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:07:54.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-11-24T06:07:54.647 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-11-24T06:07:54.647 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000172767 s, 3.0 MB/s 2023-11-24T06:07:54.648 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:07:54.705 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:07:54.706 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:07:54.706 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:07:54.722 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:07:54.722 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 388991 Links: 1 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-11-24 06:07:45.677407033 +0000 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-11-24 06:05:00.853142855 +0000 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-11-24 06:05:00.853142855 +0000 2023-11-24T06:07:54.779 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-11-24T06:07:54.780 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:07:54.844 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-11-24T06:07:54.844 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-11-24T06:07:54.844 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000254307 s, 2.0 MB/s 2023-11-24T06:07:54.845 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:07:54.903 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2023-11-24T06:07:54.961 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:07:54.961 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:07:54.962 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 386758 Links: 1 2023-11-24T06:07:54.962 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:07:54.962 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:07:54.962 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-11-24 06:07:45.677407033 +0000 2023-11-24T06:07:54.962 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-11-24 06:05:01.367128642 +0000 2023-11-24T06:07:54.962 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-11-24 06:05:01.367128642 +0000 2023-11-24T06:07:54.962 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-11-24T06:07:54.962 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:07:55.026 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-11-24T06:07:55.027 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-11-24T06:07:55.027 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000247235 s, 2.1 MB/s 2023-11-24T06:07:55.028 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:07:55.085 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2023-11-24T06:07:55.142 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:07:55.143 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:07:55.143 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 390507 Links: 1 2023-11-24T06:07:55.143 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:07:55.143 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:07:55.143 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-11-24 06:07:45.677407033 +0000 2023-11-24T06:07:55.143 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-11-24 06:05:01.865114872 +0000 2023-11-24T06:07:55.143 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-11-24 06:05:01.865114872 +0000 2023-11-24T06:07:55.143 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-11-24T06:07:55.143 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:07:55.208 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-11-24T06:07:55.208 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-11-24T06:07:55.209 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000223649 s, 2.3 MB/s 2023-11-24T06:07:55.210 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:07:55.267 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2023-11-24T06:07:55.325 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:07:55.325 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:07:55.325 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 6h/6d Inode: 393384 Links: 1 2023-11-24T06:07:55.325 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:07:55.325 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:07:55.325 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-11-24 06:07:45.677407033 +0000 2023-11-24T06:07:55.326 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-11-24 06:05:02.381100603 +0000 2023-11-24T06:07:55.326 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-11-24 06:05:02.381100603 +0000 2023-11-24T06:07:55.326 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-11-24T06:07:55.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:07:55.391 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-11-24T06:07:55.392 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-11-24T06:07:55.392 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000240877 s, 2.1 MB/s 2023-11-24T06:07:55.393 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:07:55.450 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:07:55.451 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:07:55.451 INFO:tasks.ceph:Generating config... 2023-11-24T06:07:55.452 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T06:07:55.452 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T06:07:55.452 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T06:07:55.452 INFO:tasks.ceph:[client] pid file = 2023-11-24T06:07:55.452 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T06:07:55.452 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T06:07:55.453 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T06:07:55.453 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T06:07:55.454 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T06:07:55.454 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T06:07:55.454 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T06:07:55.454 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T06:07:55.454 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T06:07:55.454 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T06:07:55.454 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T06:07:55.454 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T06:07:55.454 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T06:07:55.454 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T06:07:55.454 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T06:07:55.455 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T06:07:55.456 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T06:07:55.456 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T06:07:55.456 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T06:07:55.456 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T06:07:55.456 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T06:07:55.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T06:07:55.663 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T06:07:55.666 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T06:07:55.712 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:07:55.739 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.97'), ('mon.b', '172.21.15.183'), ('mon.c', '[v2:172.21.15.183:3301,v1:172.21.15.183:6790]')] 2023-11-24T06:07:55.740 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.97,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T06:07:55.741 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:55.741 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:07:55.795 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.97 --add b 172.21.15.183 --addv c '[v2:172.21.15.183:3301,v1:172.21.15.183:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:07:55.884 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:07:55.884 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: generated fsid d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9 2023-11-24T06:07:55.884 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = pacific 2023-11-24T06:07:55.884 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 0 2023-11-24T06:07:55.884 INFO:teuthology.orchestra.run.smithi097.stdout:fsid d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9 2023-11-24T06:07:55.884 INFO:teuthology.orchestra.run.smithi097.stdout:last_changed 2023-11-24T06:07:55.882091+0000 2023-11-24T06:07:55.884 INFO:teuthology.orchestra.run.smithi097.stdout:created 2023-11-24T06:07:55.882091+0000 2023-11-24T06:07:55.885 INFO:teuthology.orchestra.run.smithi097.stdout:min_mon_release 16 (pacific) 2023-11-24T06:07:55.885 INFO:teuthology.orchestra.run.smithi097.stdout:election_strategy: 1 2023-11-24T06:07:55.885 INFO:teuthology.orchestra.run.smithi097.stdout:0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2023-11-24T06:07:55.885 INFO:teuthology.orchestra.run.smithi097.stdout:1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2023-11-24T06:07:55.885 INFO:teuthology.orchestra.run.smithi097.stdout:2: [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] mon.c 2023-11-24T06:07:55.885 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T06:07:55.886 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:07:55.902 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9... 2023-11-24T06:07:55.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:07:55.946 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:07:55.989 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-11-24T06:07:55.990 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-11-24T06:07:55.990 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:07:55.990 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-11-24T06:07:55.990 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.990 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-11-24T06:07:55.991 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.991 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-11-24T06:07:55.991 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.991 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-11-24T06:07:55.991 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-11-24T06:07:55.991 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.991 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-11-24T06:07:55.991 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:07:55.992 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-11-24T06:07:55.993 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.97,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]" 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on skipped message = False 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-11-24T06:07:55.994 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2023-11-24T06:07:55.995 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:07:55.996 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-11-24T06:07:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-11-24T06:07:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-11-24T06:07:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-11-24T06:07:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-11-24T06:07:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-11-24T06:07:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-11-24T06:07:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-11-24T06:07:55.998 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: debug finisher = 20 2023-11-24T06:07:55.999 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-11-24T06:07:56.000 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-11-24T06:07:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:07:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:07:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:07:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:07:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:07:56.001 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:07:56.002 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-11-24T06:07:56.003 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-11-24T06:07:56.003 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-11-24T06:07:56.003 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-11-24T06:07:56.003 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:07:56.003 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:07:56.003 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-11-24T06:07:56.003 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-11-24T06:07:56.004 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-11-24T06:07:56.005 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-11-24T06:07:56.006 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-11-24T06:07:56.006 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-11-24T06:07:56.006 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-11-24T06:07:56.007 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2023-11-24T06:07:56.007 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2023-11-24T06:07:56.008 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2023-11-24T06:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2023-11-24T06:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2023-11-24T06:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2023-11-24T06:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.009 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:07:56.010 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:07:56.010 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.010 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2023-11-24T06:07:56.010 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.010 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2023-11-24T06:07:56.010 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:07:56.011 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.012 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2023-11-24T06:07:56.012 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.012 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2023-11-24T06:07:56.012 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.012 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2023-11-24T06:07:56.012 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2023-11-24T06:07:56.012 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2023-11-24T06:07:56.012 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = "172.21.15.97,172.21.15.183,[v2:172.21.15.183:3301,v1:172.21.15.183:6790]" 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on skipped message = False 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:07:56.013 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.014 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2023-11-24T06:07:56.014 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2023-11-24T06:07:56.014 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.014 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2023-11-24T06:07:56.014 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2023-11-24T06:07:56.014 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2023-11-24T06:07:56.014 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug trim objects = true 2023-11-24T06:07:56.014 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2023-11-24T06:07:56.015 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: bdev async discard = True 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: bdev enable discard = True 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore allocator = bitmap 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 1/20 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 1/20 2023-11-24T06:07:56.016 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 20 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 4/10 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: osd op complaint time = 180 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2023-11-24T06:07:56.017 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: debug client = 20 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: debug finisher = 20 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-11-24T06:07:56.018 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:07:56.019 INFO:teuthology.orchestra.run.smithi183.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: # 1m isn't quite enough 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: mon op complaint time = 120 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2023-11-24T06:07:56.020 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: client mount timeout = 600 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: debug client = 20 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = "" 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: rados mon op timeout = 900 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout: rados osd op timeout = 900 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2023-11-24T06:07:56.021 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.b] 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.c] 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.a] 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.c] 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.b] 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.d] 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout:[mds] 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout: debug mds = 20 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout: debug mds balancer = 20 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout: mds debug frag = True 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout: mds debug scatterstat = True 2023-11-24T06:07:56.022 INFO:teuthology.orchestra.run.smithi183.stdout: mds op complaint time = 180 2023-11-24T06:07:56.023 INFO:teuthology.orchestra.run.smithi183.stdout: mds verify scatter = True 2023-11-24T06:07:56.023 INFO:teuthology.orchestra.run.smithi183.stdout: osd op complaint time = 180 2023-11-24T06:07:56.023 INFO:teuthology.orchestra.run.smithi183.stdout: rados mon op timeout = 900 2023-11-24T06:07:56.023 INFO:teuthology.orchestra.run.smithi183.stdout: rados osd op timeout = 900 2023-11-24T06:07:56.023 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T06:07:56.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T06:07:56.105 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T06:07:56.106 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:56.106 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T06:07:56.124 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:56.124 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T06:07:56.181 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:07:56.181 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:56.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:07:56.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:07:56.258 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:07:56.259 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:07:56.317 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:07:56.317 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:07:56.318 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:07:56.318 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:07:56.354 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:07:56.354 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:07:56.410 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T06:07:56.410 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T06:07:56.410 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:07:56.466 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:07:56.469 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:07:56.536 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:07:56.539 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T06:07:56.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:07:56.593 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:07:56.595 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T06:07:56.621 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:07:56.713 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:07:56.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T06:07:56.742 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:07:56.797 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:07:56.800 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T06:07:56.825 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:07:56.919 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:07:56.922 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T06:07:56.947 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T06:07:56.948 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T06:07:56.991 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T06:07:57.004 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T06:07:57.047 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T06:07:57.061 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T06:07:57.105 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T06:07:57.118 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T06:07:57.163 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T06:07:57.176 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T06:07:57.177 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:07:57.177 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T06:07:57.204 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:07:57.204 INFO:tasks.ceph:role: osd.0 2023-11-24T06:07:57.205 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:07:57.205 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:07:57.278 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:07:57.278 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:07:57.279 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:07:57.279 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:07:57.279 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:07:57.279 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:07:57.279 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:07:57.279 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:07:57.279 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:07:57.279 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:07:57.727 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-11-24T06:07:57.728 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-11-24T06:07:57.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T06:07:57.915 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T06:07:57.947 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T06:07:58.013 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:07:58.013 INFO:tasks.ceph:role: osd.1 2023-11-24T06:07:58.014 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:07:58.014 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:07:58.088 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:07:58.507 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-11-24T06:07:58.509 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-11-24T06:07:58.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T06:07:58.617 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T06:07:58.647 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T06:07:58.713 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:07:58.714 INFO:tasks.ceph:role: osd.2 2023-11-24T06:07:58.714 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:07:58.714 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:07:58.788 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:07:58.788 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:07:58.788 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:07:58.788 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:07:58.788 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:07:58.788 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:07:58.789 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:07:58.789 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:07:58.789 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:07:58.789 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:07:59.241 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-11-24T06:07:59.242 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-11-24T06:07:59.242 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T06:07:59.393 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T06:07:59.424 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T06:07:59.489 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:07:59.490 INFO:tasks.ceph:role: osd.3 2023-11-24T06:07:59.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:07:59.490 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:07:59.563 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:07:59.563 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:07:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:07:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:07:59.564 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:07:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:07:59.564 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:07:59.564 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:07:59.564 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:07:59.564 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:08:00.004 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-11-24T06:08:00.006 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-11-24T06:08:00.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T06:08:00.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T06:08:00.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:00.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:00.339+0000 7f991384c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T06:08:00.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:00.339+0000 7f991384c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T06:08:00.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:00.339+0000 7f991384c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:08:00.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:00.339+0000 7f991384c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:08:00.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:00.339+0000 7f991384c640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:08:00.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:00.340+0000 7f991384c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T06:08:03.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T06:08:03.248 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:03.332+0000 7f2f3d35f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T06:08:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:03.332+0000 7f2f3d35f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T06:08:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:03.332+0000 7f2f3d35f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:08:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:03.332+0000 7f2f3d35f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:08:03.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:03.332+0000 7f2f3d35f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:08:03.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:03.333+0000 7f2f3d35f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T06:08:06.202 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T06:08:06.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:06.320+0000 7f43c03d0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T06:08:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:06.321+0000 7f43c03d0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T06:08:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:06.321+0000 7f43c03d0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:08:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:06.321+0000 7f43c03d0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:08:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:06.321+0000 7f43c03d0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:08:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:06.321+0000 7f43c03d0640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T06:08:09.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T06:08:09.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:09.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:09.331+0000 7f1375cad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T06:08:09.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:09.331+0000 7f1375cad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T06:08:09.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:09.331+0000 7f1375cad640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:08:09.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:09.331+0000 7f1375cad640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:08:09.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:09.331+0000 7f1375cad640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:08:09.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:09.332+0000 7f1375cad640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T06:08:12.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T06:08:12.227 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T06:08:12.261 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:08:12.261 INFO:tasks.ceph:role: osd.4 2023-11-24T06:08:12.261 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:08:12.261 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:08:12.337 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:08:12.338 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:08:12.764 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-11-24T06:08:12.766 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-11-24T06:08:12.766 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T06:08:12.950 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T06:08:12.985 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T06:08:13.052 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:08:13.052 INFO:tasks.ceph:role: osd.5 2023-11-24T06:08:13.052 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:08:13.052 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:08:13.125 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:08:13.126 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:08:13.126 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:08:13.126 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:08:13.126 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:08:13.126 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:08:13.127 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:08:13.127 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:08:13.127 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:08:13.127 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:08:13.554 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-11-24T06:08:13.556 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-11-24T06:08:13.557 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T06:08:13.671 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T06:08:13.700 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T06:08:13.766 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:08:13.767 INFO:tasks.ceph:role: osd.6 2023-11-24T06:08:13.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:08:13.767 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:08:13.840 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:08:13.841 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:08:14.293 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-11-24T06:08:14.295 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-11-24T06:08:14.295 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T06:08:14.458 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T06:08:14.488 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T06:08:14.555 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:08:14.555 INFO:tasks.ceph:role: osd.7 2023-11-24T06:08:14.555 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:08:14.555 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:08:14.630 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:08:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:08:14.630 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:08:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T06:08:14.631 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:08:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:08:14.631 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:08:14.631 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T06:08:14.631 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:08:14.631 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:08:15.060 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-11-24T06:08:15.062 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-11-24T06:08:15.062 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T06:08:15.234 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T06:08:15.268 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:15.369 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:15.365+0000 7f098b6e7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T06:08:15.369 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:15.365+0000 7f098b6e7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T06:08:15.369 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:15.365+0000 7f098b6e7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:08:15.369 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:15.365+0000 7f098b6e7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:08:15.369 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:15.365+0000 7f098b6e7640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:08:15.369 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:15.366+0000 7f098b6e7640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T06:08:18.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T06:08:18.279 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:18.370 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:18.366+0000 7f0edb906640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T06:08:18.370 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:18.366+0000 7f0edb906640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T06:08:18.370 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:18.366+0000 7f0edb906640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:08:18.370 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:18.366+0000 7f0edb906640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:08:18.370 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:18.366+0000 7f0edb906640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:08:18.371 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:18.367+0000 7f0edb906640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T06:08:21.227 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T06:08:21.259 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:21.352 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:21.348+0000 7fea69d96640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T06:08:21.352 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:21.348+0000 7fea69d96640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T06:08:21.352 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:21.348+0000 7fea69d96640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:08:21.352 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:21.348+0000 7fea69d96640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:08:21.352 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:21.348+0000 7fea69d96640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:08:21.352 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:21.348+0000 7fea69d96640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T06:08:23.821 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:08:23.823 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:08:24.217 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T06:08:24.247 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:24.338 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:24.334+0000 7f6706fd8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T06:08:24.338 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:24.334+0000 7f6706fd8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T06:08:24.338 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:24.334+0000 7f6706fd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:08:24.338 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:24.334+0000 7f6706fd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:08:24.338 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:24.334+0000 7f6706fd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:08:24.338 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:24.335+0000 7f6706fd8640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T06:08:27.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T06:08:27.231 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T06:08:27.231 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:27.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T06:08:27.263 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:27.263 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T06:08:27.332 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:27.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T06:08:27.402 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:27.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T06:08:27.471 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:27.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T06:08:27.540 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:27.540 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T06:08:27.608 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:27.608 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T06:08:27.675 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:27.675 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T06:08:27.706 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:27.706 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T06:08:27.777 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:27.777 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T06:08:27.847 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:27.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T06:08:27.916 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:27.916 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T06:08:27.984 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:27.985 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T06:08:28.052 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:28.053 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T06:08:28.120 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:28.120 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T06:08:28.139 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:28.139 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T06:08:28.197 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:28.197 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T06:08:28.214 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:28.214 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T06:08:28.273 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T06:08:28.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:08:28.275 DEBUG:teuthology.orchestra.run.smithi183:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBla7myGxAAHEPHxXHfBzCeqi2nLjhK6g== 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlijU/IxAAp5/+zEd6ER/JFqum0ptk0Q== 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlGtxpKhAAiQm5ore3YYETyG+sIKzEcw== 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDAPWBlwipJFBAAGLCD8wGcYoSy4yM4RXb0/Q== 2023-11-24T06:08:28.316 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDDPWBlgW7fExAAQuUSGJn+ZfHSMwSi5r1pQg== 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDGPWBl3E8uExAAJKjzbRFyYie6dmnRrP7VVw== 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDJPWBl5gnRExAAbKbgF6DcdTSv8M84GXyONw== 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlOezbHxAAlNbpLHDzRHvk2SXFvQoTXQ== 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlLftpLxAAH+Ew2GQv73YtKLzZJjSBew== 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBl05iyNhAAHZYZ5pGghTwe55n84iXXQw== 2023-11-24T06:08:28.317 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDPPWBlmCfaFRAAsIQWHUnlgDQcRXLndlUZKg== 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDSPWBlPxfqFRAARyMzgivfTfgmFxZRjp54JQ== 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDVPWBl8cjTFBAAsNfkNY4gGVTJwh+mMtADgg== 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDYPWBljh8EFBAA4nVPoyEXigIDTUB6enXsXA== 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA== 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout:[client.2] 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC9PWBlJRe4AhAAGQVWmqQr/urD7AefOfmoZQ== 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2023-11-24T06:08:28.318 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC9PWBlOXAsBhAABOT6CW2RVj7Iv5XMPvQTgQ== 2023-11-24T06:08:28.319 INFO:teuthology.orchestra.run.smithi097.stdout:[client.3] 2023-11-24T06:08:28.319 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC9PWBlwX2YCRAAIM9eB39aohf1xUdEvHr/tA== 2023-11-24T06:08:28.338 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.y] 2023-11-24T06:08:28.338 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC8PWBla7myGxAAHEPHxXHfBzCeqi2nLjhK6g== 2023-11-24T06:08:28.338 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.a] 2023-11-24T06:08:28.338 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC8PWBlijU/IxAAp5/+zEd6ER/JFqum0ptk0Q== 2023-11-24T06:08:28.338 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.c] 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC8PWBlGtxpKhAAiQm5ore3YYETyG+sIKzEcw== 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.0] 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDAPWBlwipJFBAAGLCD8wGcYoSy4yM4RXb0/Q== 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.1] 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDDPWBlgW7fExAAQuUSGJn+ZfHSMwSi5r1pQg== 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.2] 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDGPWBl3E8uExAAJKjzbRFyYie6dmnRrP7VVw== 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.3] 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDJPWBl5gnRExAAbKbgF6DcdTSv8M84GXyONw== 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.x] 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC8PWBlOezbHxAAlNbpLHDzRHvk2SXFvQoTXQ== 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.b] 2023-11-24T06:08:28.339 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC8PWBlLftpLxAAH+Ew2GQv73YtKLzZJjSBew== 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout:[mds.d] 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC8PWBl05iyNhAAHZYZ5pGghTwe55n84iXXQw== 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.4] 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDPPWBlmCfaFRAAsIQWHUnlgDQcRXLndlUZKg== 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.5] 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDSPWBlPxfqFRAARyMzgivfTfgmFxZRjp54JQ== 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.6] 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDVPWBl8cjTFBAAsNfkNY4gGVTJwh+mMtADgg== 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.7] 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDYPWBljh8EFBAA4nVPoyEXigIDTUB6enXsXA== 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout:[client.0] 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA== 2023-11-24T06:08:28.340 INFO:teuthology.orchestra.run.smithi183.stdout:[client.2] 2023-11-24T06:08:28.341 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC9PWBlJRe4AhAAGQVWmqQr/urD7AefOfmoZQ== 2023-11-24T06:08:28.341 INFO:teuthology.orchestra.run.smithi183.stdout:[client.1] 2023-11-24T06:08:28.341 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC9PWBlOXAsBhAABOT6CW2RVj7Iv5XMPvQTgQ== 2023-11-24T06:08:28.341 INFO:teuthology.orchestra.run.smithi183.stdout:[client.3] 2023-11-24T06:08:28.341 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQC9PWBlwX2YCRAAIM9eB39aohf1xUdEvHr/tA== 2023-11-24T06:08:28.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:08:28.362 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:08:28.436 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:08:28.438 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:08:28.494 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:08:28.496 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:08:28.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.554 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.606 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.608 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.663 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.718 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:08:28.773 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:08:28.825 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:08:28.827 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:08:28.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:08:28.885 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:08:28.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.943 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:28.998 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:29.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:29.054 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:29.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:29.109 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:08:29.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:08:31.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:08:31.661 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:08:31.664 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:08:31.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:08:31.725 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:08:31.783 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:08:31.785 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:08:31.841 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T06:08:31.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T06:08:31.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:08:31.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T06:08:31.991 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T06:08:32.017 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:08:32.115 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T06:08:32.141 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T06:08:32.207 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:08:32.305 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T06:08:32.331 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:32.334 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:08:32.388 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T06:08:32.389 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T06:08:32.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T06:08:32.392 INFO:tasks.ceph.mon.a:Started 2023-11-24T06:08:32.392 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T06:08:32.392 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T06:08:32.432 INFO:tasks.ceph.mon.b:Started 2023-11-24T06:08:32.432 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T06:08:32.432 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T06:08:32.434 INFO:tasks.ceph.mon.c:Started 2023-11-24T06:08:32.434 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T06:08:32.434 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T06:08:32.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T06:08:32.436 INFO:tasks.ceph.mgr.y:Started 2023-11-24T06:08:32.436 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T06:08:32.437 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T06:08:32.438 INFO:tasks.ceph.mgr.x:Started 2023-11-24T06:08:32.439 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T06:08:32.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T06:08:32.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.557+0000 7fbc50fe2700 1 Processor -- start 2023-11-24T06:08:32.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.557+0000 7fbc50fe2700 1 -- start start 2023-11-24T06:08:32.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc50fe2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 0x7fbc4c0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:32.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc50fe2700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 0x7fbc4c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:32.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc50fe2700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c0b2a60 con 0x7fbc4c0cfea0 2023-11-24T06:08:32.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc50fe2700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c15e970 con 0x7fbc4c06e930 2023-11-24T06:08:32.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc50fe2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c15ef30 con 0x7fbc4c15ac40 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc49d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 0x7fbc4c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc49d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 0x7fbc4c15f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44342/0 (socket says 172.21.15.97:44342) 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc49d9b700 1 -- 172.21.15.97:0/2547834960 learned_addr learned my addr 172.21.15.97:0/2547834960 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc49d9b700 1 -- 172.21.15.97:0/2547834960 >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 msgr2=0x7fbc4c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc49d9b700 1 -- 172.21.15.97:0/2547834960 >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 msgr2=0x7fbc4c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc49d9b700 1 --2- 172.21.15.97:0/2547834960 >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 0x7fbc4c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc4a59c700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 0x7fbc4c0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.558+0000 7fbc49d9b700 1 --2- 172.21.15.97:0/2547834960 >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 0x7fbc4c15f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.559+0000 7fbc4a59c700 1 -- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 msgr2=0x7fbc4c0b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.559+0000 7fbc4a59c700 1 -- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 msgr2=0x7fbc4c0b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:08:32.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.559+0000 7fbc4a59c700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 0x7fbc4c0b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:08:32.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.559+0000 7fbc4a59c700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 0x7fbc4c0b42d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:08:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.748+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527734543 0 0) 0x7fbc4c0b2a60 con 0x7fbc4c0cfea0 2023-11-24T06:08:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.748+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc340035b0 con 0x7fbc4c0cfea0 2023-11-24T06:08:32.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610675060 0 0) 0x7fbc340035b0 con 0x7fbc4c0cfea0 2023-11-24T06:08:32.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 msgr2=0x7fbc4c0b42d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:08:32.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 0x7fbc4c0b42d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 msgr2=0x7fbc4c15f3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 --2- 172.21.15.97:0/2547834960 >> v2:172.21.15.97:3300/0 conn(0x7fbc4c06e930 0x7fbc4c15f3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c15f900 con 0x7fbc4c0cfea0 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fbc440019f0 con 0x7fbc4c0cfea0 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 >> v1:172.21.15.183:6789/0 conn(0x7fbc4c0cfea0 legacy=0x7fbc4c0ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc34003a00 0x7fbc34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc34004850 0x7fbc34004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 0x7fbc34005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c15e970 con 0x7fbc34003a00 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c15ef30 con 0x7fbc34004850 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3400bb70 con 0x7fbc4c15ac40 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4a59c700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc34003a00 0x7fbc34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:32.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc49d9b700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc34004850 0x7fbc34004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:32.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.752+0000 7fbc4ad9d700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 0x7fbc34005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:32.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4a59c700 1 -- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 msgr2=0x7fbc34005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:32.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4a59c700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc4c15ac40 0x7fbc34005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4a59c700 1 -- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc34004850 msgr2=0x7fbc34004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:32.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4a59c700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc34004850 0x7fbc34004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4a59c700 1 -- 172.21.15.97:0/2547834960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc340035b0 con 0x7fbc34003a00 2023-11-24T06:08:32.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc49d9b700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc34004850 0x7fbc34004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:32.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4a59c700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc34003a00 0x7fbc34003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009860 tx=0x7fbc34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cf0a9eca4c3e43 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000c070 con 0x7fbc34003a00 2023-11-24T06:08:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc40004020 con 0x7fbc34003a00 2023-11-24T06:08:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.753+0000 7fbc4959a700 1 -- 172.21.15.97:0/2547834960 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400043d0 con 0x7fbc34003a00 2023-11-24T06:08:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.754+0000 7fbc50fe2700 1 -- 172.21.15.97:0/2547834960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc34003a00 msgr2=0x7fbc34003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.754+0000 7fbc50fe2700 1 --2- 172.21.15.97:0/2547834960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc34003a00 0x7fbc34003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009860 tx=0x7fbc34005880 comp rx=0 tx=0).stop 2023-11-24T06:08:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.754+0000 7fbc4ad9d700 1 -- 172.21.15.97:0/2547834960 reap_dead start 2023-11-24T06:08:32.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.754+0000 7fbc50fe2700 1 -- 172.21.15.97:0/2547834960 shutdown_connections 2023-11-24T06:08:32.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.754+0000 7fbc50fe2700 1 -- 172.21.15.97:0/2547834960 >> 172.21.15.97:0/2547834960 conn(0x7fbc4c06b410 msgr2=0x7fbc4c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:32.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.754+0000 7fbc50fe2700 1 -- 172.21.15.97:0/2547834960 shutdown_connections 2023-11-24T06:08:32.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.754+0000 7fbc50fe2700 1 -- 172.21.15.97:0/2547834960 wait complete. 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.755+0000 7fbc50fe2700 1 Processor -- start 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.755+0000 7fbc50fe2700 1 -- start start 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc50fe2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc34003a00 0x7fbc4c14a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc50fe2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c0c9440 0x7fbc4c159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc50fe2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc4c159960 0x7fbc4c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc50fe2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c14ab30 con 0x7fbc4c0c9440 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc50fe2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c159f60 con 0x7fbc4c159960 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc50fe2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c15a240 con 0x7fbc34003a00 2023-11-24T06:08:32.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc49d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c0c9440 0x7fbc4c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc49d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c0c9440 0x7fbc4c159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44376/0 (socket says 172.21.15.97:44376) 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc49d9b700 1 -- 172.21.15.97:0/3877769251 learned_addr learned my addr 172.21.15.97:0/3877769251 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc4a59c700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc34003a00 0x7fbc4c14a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc4ad9d700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc4c159960 0x7fbc4c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc49d9b700 1 -- 172.21.15.97:0/3877769251 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc34003a00 msgr2=0x7fbc4c14a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.756+0000 7fbc49d9b700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc34003a00 0x7fbc4c14a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc49d9b700 1 -- 172.21.15.97:0/3877769251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc4c159960 msgr2=0x7fbc4c156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc49d9b700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc4c159960 0x7fbc4c156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc49d9b700 1 -- 172.21.15.97:0/3877769251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c15c4a0 con 0x7fbc4c0c9440 2023-11-24T06:08:32.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc49d9b700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c0c9440 0x7fbc4c159420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c011ad0 tx=0x7fbc3c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc3affd700 1 -- 172.21.15.97:0/3877769251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c010da0 con 0x7fbc4c0c9440 2023-11-24T06:08:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc3affd700 1 -- 172.21.15.97:0/3877769251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c004020 con 0x7fbc4c0c9440 2023-11-24T06:08:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc3affd700 1 -- 172.21.15.97:0/3877769251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c0043b0 con 0x7fbc4c0c9440 2023-11-24T06:08:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c15c780 con 0x7fbc4c0c9440 2023-11-24T06:08:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c151830 con 0x7fbc4c0c9440 2023-11-24T06:08:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc3affd700 1 -- 172.21.15.97:0/3877769251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbc3c0039a0 con 0x7fbc4c0c9440 2023-11-24T06:08:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.757+0000 7fbc3affd700 1 -- 172.21.15.97:0/3877769251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbc3c023950 con 0x7fbc4c0c9440 2023-11-24T06:08:32.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.759+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c068480 con 0x7fbc4c0c9440 2023-11-24T06:08:32.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.763+0000 7fbc3affd700 1 -- 172.21.15.97:0/3877769251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fbc3c01e020 con 0x7fbc4c0c9440 2023-11-24T06:08:32.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.839+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbc4c0ccaa0 con 0x7fbc4c0c9440 2023-11-24T06:08:32.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.841+0000 7fbc3affd700 1 -- 172.21.15.97:0/3877769251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbc3c003e30 con 0x7fbc4c0c9440 2023-11-24T06:08:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c0c9440 msgr2=0x7fbc4c159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c0c9440 0x7fbc4c159420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c011ad0 tx=0x7fbc3c011de0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 shutdown_connections 2023-11-24T06:08:32.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc4c159960 0x7fbc4c156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc34003a00 0x7fbc4c14a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 --2- 172.21.15.97:0/3877769251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc4c0c9440 0x7fbc4c159420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:32.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 >> 172.21.15.97:0/3877769251 conn(0x7fbc4c06b410 msgr2=0x7fbc4c071830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:32.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 shutdown_connections 2023-11-24T06:08:32.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:32.842+0000 7fbc50fe2700 1 -- 172.21.15.97:0/3877769251 wait complete. 2023-11-24T06:08:32.847 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2023-11-24T06:08:32.857 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T06:08:32.857 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T06:08:32.858 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:32.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T06:08:32.873 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:32.871+0000 7f828fef3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:08:32.875 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:32.873+0000 7fca4b0b1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:08:32.883 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:32.883 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T06:08:32.938 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:32.935+0000 7f828fef3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:08:32.944 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:32.941+0000 7fca4b0b1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:08:32.950 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:32.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T06:08:33.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:33.011+0000 7f828fef3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:08:33.015 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:08:33.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T06:08:33.025 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:33.022+0000 7fca4b0b1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:08:33.050 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:33.050 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T06:08:33.076 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:33.077 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T06:08:33.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:33.093+0000 7f828fef3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:08:33.108 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:33.106+0000 7fca4b0b1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:08:33.120 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:33.120 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T06:08:33.187 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:08:33.187 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T06:08:33.244 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:33.242+0000 7f828fef3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:08:33.249 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:33.247+0000 7fca4b0b1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:08:33.254 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77 0 2023-11-24T06:08:33.416 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.413+0000 7fbad9672700 1 Processor -- start 2023-11-24T06:08:33.416 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.413+0000 7fbad9672700 1 -- start start 2023-11-24T06:08:33.418 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad9672700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad40746a0 0x7fbad40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.418 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad9672700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbad40afc90 0x7fbad406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.418 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad9672700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad40b3c60 con 0x7fbad415e160 2023-11-24T06:08:33.418 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad9672700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad40b4500 con 0x7fbad40afc90 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad9672700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad40b58c0 con 0x7fbad40746a0 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad37fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbad40afc90 0x7fbad406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad2ffd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fbad415e160 0x7fbad4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:34608/0 (socket says 172.21.15.183:34608) 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad2ffd700 1 -- 172.21.15.183:0/3564197481 learned_addr learned my addr 172.21.15.183:0/3564197481 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.414+0000 7fbad27fc700 1 --2- 172.21.15.183:0/3564197481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad40746a0 0x7fbad40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad37fe700 1 -- 172.21.15.183:0/3564197481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad40746a0 msgr2=0x7fbad40b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad37fe700 1 --2- 172.21.15.183:0/3564197481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad40746a0 0x7fbad40b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad37fe700 1 -- 172.21.15.183:0/3564197481 >> v1:172.21.15.183:6789/0 conn(0x7fbad415e160 legacy=0x7fbad4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad37fe700 1 -- 172.21.15.183:0/3564197481 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad40b0140 con 0x7fbad40afc90 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad1ffb700 1 -- 172.21.15.183:0/3564197481 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205599623 0 0) 0x7fbad40b3c60 con 0x7fbad415e160 2023-11-24T06:08:33.419 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad37fe700 1 --2- 172.21.15.183:0/3564197481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad406d930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009d90 tx=0x7fbacc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=322eda44fb3d1db8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad1ffb700 1 -- 172.21.15.183:0/3564197481 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00e070 con 0x7fbad40afc90 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad1ffb700 1 -- 172.21.15.183:0/3564197481 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00bb20 con 0x7fbad40afc90 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad1ffb700 1 -- 172.21.15.183:0/3564197481 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc012430 con 0x7fbad40afc90 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad9672700 1 -- 172.21.15.183:0/3564197481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 msgr2=0x7fbad406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad9672700 1 --2- 172.21.15.183:0/3564197481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad406d930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009d90 tx=0x7fbacc00b330 comp rx=0 tx=0).stop 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad9672700 1 -- 172.21.15.183:0/3564197481 shutdown_connections 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad9672700 1 --2- 172.21.15.183:0/3564197481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad406d930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad9672700 1 --2- 172.21.15.183:0/3564197481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad40746a0 0x7fbad40b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.415+0000 7fbad9672700 1 -- 172.21.15.183:0/3564197481 >> 172.21.15.183:0/3564197481 conn(0x7fbad40d0bf0 msgr2=0x7fbad40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.416+0000 7fbad9672700 1 -- 172.21.15.183:0/3564197481 shutdown_connections 2023-11-24T06:08:33.420 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.416+0000 7fbad9672700 1 -- 172.21.15.183:0/3564197481 wait complete. 2023-11-24T06:08:33.423 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.417+0000 7fbad9672700 1 Processor -- start 2023-11-24T06:08:33.423 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.417+0000 7fbad9672700 1 -- start start 2023-11-24T06:08:33.423 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad9672700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40746a0 0x7fbad4146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.423 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad9672700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad4142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.423 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad9672700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad415e160 0x7fbad41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad9672700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad41431b0 con 0x7fbad40746a0 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad9672700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad4160df0 con 0x7fbad40afc90 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad9672700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad41610d0 con 0x7fbad415e160 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad27fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad4142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad27fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad4142c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36152/0 (socket says 172.21.15.183:36152) 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad27fc700 1 -- 172.21.15.183:0/3668961997 learned_addr learned my addr 172.21.15.183:0/3668961997 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad37fe700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad415e160 0x7fbad41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.418+0000 7fbad2ffd700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40746a0 0x7fbad4146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad27fc700 1 -- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad415e160 msgr2=0x7fbad41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad27fc700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad415e160 0x7fbad41604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad27fc700 1 -- 172.21.15.183:0/3668961997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40746a0 msgr2=0x7fbad4146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.424 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad27fc700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40746a0 0x7fbad4146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad27fc700 1 -- 172.21.15.183:0/3668961997 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc009a70 con 0x7fbad40afc90 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad37fe700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad415e160 0x7fbad41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad27fc700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad4142c70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbac800e9b0 tx=0x7fbac800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbabb7fe700 1 -- 172.21.15.183:0/3668961997 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8009710 con 0x7fbad40afc90 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbabb7fe700 1 -- 172.21.15.183:0/3668961997 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac800ee50 con 0x7fbad40afc90 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad41613b0 con 0x7fbad40afc90 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbabb7fe700 1 -- 172.21.15.183:0/3668961997 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac800ca50 con 0x7fbad40afc90 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad4143420 con 0x7fbad40afc90 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbabb7fe700 1 -- 172.21.15.183:0/3668961997 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbac8005a20 con 0x7fbad40afc90 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.419+0000 7fbabb7fe700 1 -- 172.21.15.183:0/3668961997 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbac80137b0 con 0x7fbad40afc90 2023-11-24T06:08:33.425 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.420+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad40ce3e0 con 0x7fbad40afc90 2023-11-24T06:08:33.426 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.422+0000 7fbabb7fe700 1 -- 172.21.15.183:0/3668961997 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fbad40ce3e0 con 0x7fbad40afc90 2023-11-24T06:08:33.508 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.505+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77", "id": 0} v 0) v1 -- 0x7fbad40ce7c0 con 0x7fbad40afc90 2023-11-24T06:08:33.511 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.509+0000 7fbabb7fe700 1 -- 172.21.15.183:0/3668961997 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbad40ce7c0 con 0x7fbad40afc90 2023-11-24T06:08:33.511 INFO:teuthology.orchestra.run.smithi183.stdout:0 2023-11-24T06:08:33.512 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.509+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 msgr2=0x7fbad4142c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.512 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.509+0000 7fbad9672700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad4142c70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbac800e9b0 tx=0x7fbac800ecc0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.513 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.509+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 shutdown_connections 2023-11-24T06:08:33.513 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.509+0000 7fbad9672700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbad415e160 0x7fbad41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.513 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.509+0000 7fbad9672700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40746a0 0x7fbad4146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.513 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.509+0000 7fbad9672700 1 --2- 172.21.15.183:0/3668961997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbad40afc90 0x7fbad4142c70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.513 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.509+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 >> 172.21.15.183:0/3668961997 conn(0x7fbad40d0bf0 msgr2=0x7fbad4162450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:33.513 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.510+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 shutdown_connections 2023-11-24T06:08:33.513 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.510+0000 7fbad9672700 1 -- 172.21.15.183:0/3668961997 wait complete. 2023-11-24T06:08:33.523 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 377f83d5-5100-48d5-8a3d-a78fcdd2175d 1 2023-11-24T06:08:33.638 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:33.635+0000 7fca4b0b1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:08:33.644 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.642+0000 7f63ed82e700 1 Processor -- start 2023-11-24T06:08:33.644 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.642+0000 7f63ed82e700 1 -- start start 2023-11-24T06:08:33.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.642+0000 7f63ed82e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f63e80b4310 0x7f63e806bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.642+0000 7f63ed82e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f63e80ce430 0x7f63e80ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.642+0000 7f63ed82e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e80cedc0 con 0x7f63e815e080 2023-11-24T06:08:33.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.642+0000 7f63ed82e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e80d1200 con 0x7f63e80b4310 2023-11-24T06:08:33.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.642+0000 7f63ed82e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e815e460 con 0x7f63e80ce430 2023-11-24T06:08:33.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e77fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f63e80ce430 0x7f63e80ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.647 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e6ffd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f63e815e080 0x7f63e80d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:34620/0 (socket says 172.21.15.183:34620) 2023-11-24T06:08:33.648 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e6ffd700 1 -- 172.21.15.183:0/552971386 learned_addr learned my addr 172.21.15.183:0/552971386 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-11-24T06:08:33.648 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e67fc700 1 --2- 172.21.15.183:0/552971386 >> v2:172.21.15.97:3300/0 conn(0x7f63e80b4310 0x7f63e806bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.648 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e77fe700 1 -- 172.21.15.183:0/552971386 >> v2:172.21.15.97:3300/0 conn(0x7f63e80b4310 msgr2=0x7f63e806bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:33.648 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e77fe700 1 --2- 172.21.15.183:0/552971386 >> v2:172.21.15.97:3300/0 conn(0x7f63e80b4310 0x7f63e806bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.648 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e77fe700 1 -- 172.21.15.183:0/552971386 >> v1:172.21.15.183:6789/0 conn(0x7f63e815e080 legacy=0x7f63e80d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e77fe700 1 -- 172.21.15.183:0/552971386 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e806ba50 con 0x7f63e80ce430 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e5ffb700 1 -- 172.21.15.183:0/552971386 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182829720 0 0) 0x7f63d8001110 con 0x7f63e815e080 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e77fe700 1 --2- 172.21.15.183:0/552971386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80ce430 0x7f63e80ce880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f63e806e1c0 tx=0x7f63e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2168a7a7b60bd27c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e5ffb700 1 -- 172.21.15.183:0/552971386 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e000e070 con 0x7f63e80ce430 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e5ffb700 1 -- 172.21.15.183:0/552971386 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e000bb20 con 0x7f63e80ce430 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.643+0000 7f63e5ffb700 1 -- 172.21.15.183:0/552971386 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0012430 con 0x7f63e80ce430 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.644+0000 7f63ed82e700 1 -- 172.21.15.183:0/552971386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80ce430 msgr2=0x7f63e80ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.644+0000 7f63ed82e700 1 --2- 172.21.15.183:0/552971386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80ce430 0x7f63e80ce880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f63e806e1c0 tx=0x7f63e000b330 comp rx=0 tx=0).stop 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.644+0000 7f63ed82e700 1 -- 172.21.15.183:0/552971386 shutdown_connections 2023-11-24T06:08:33.649 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.644+0000 7f63ed82e700 1 --2- 172.21.15.183:0/552971386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80ce430 0x7f63e80ce880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.650 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.644+0000 7f63ed82e700 1 --2- 172.21.15.183:0/552971386 >> v2:172.21.15.97:3300/0 conn(0x7f63e80b4310 0x7f63e806bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.650 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.644+0000 7f63ed82e700 1 -- 172.21.15.183:0/552971386 >> 172.21.15.183:0/552971386 conn(0x7f63e80b3780 msgr2=0x7f63e80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:33.650 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.644+0000 7f63ed82e700 1 -- 172.21.15.183:0/552971386 shutdown_connections 2023-11-24T06:08:33.650 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.644+0000 7f63ed82e700 1 -- 172.21.15.183:0/552971386 wait complete. 2023-11-24T06:08:33.652 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.645+0000 7f63ed82e700 1 Processor -- start 2023-11-24T06:08:33.652 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.645+0000 7f63ed82e700 1 -- start start 2023-11-24T06:08:33.652 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63ed82e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80b4310 0x7f63e80ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.652 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63ed82e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e80ce430 0x7f63e80b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.652 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63ed82e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e815e080 0x7f63e80b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.652 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63ed82e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e80ca7f0 con 0x7f63e80ce430 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63ed82e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e80caed0 con 0x7f63e80b4310 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63ed82e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e80cb100 con 0x7f63e815e080 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80b4310 0x7f63e80ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e815e080 0x7f63e80b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80b4310 0x7f63e80ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36162/0 (socket says 172.21.15.183:36162) 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 -- 172.21.15.183:0/3280407480 learned_addr learned my addr 172.21.15.183:0/3280407480 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e67fc700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e80ce430 0x7f63e80b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 -- 172.21.15.183:0/3280407480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e815e080 msgr2=0x7f63e80b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e815e080 0x7f63e80b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 -- 172.21.15.183:0/3280407480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e80ce430 msgr2=0x7f63e80b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.653 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e80ce430 0x7f63e80b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.654 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 -- 172.21.15.183:0/3280407480 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0009a70 con 0x7f63e80b4310 2023-11-24T06:08:33.654 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e77fe700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e815e080 0x7f63e80b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:33.654 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.646+0000 7f63e6ffd700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80b4310 0x7f63e80ca990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c580 tx=0x7f63d800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:33.654 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.647+0000 7f63cffff700 1 -- 172.21.15.183:0/3280407480 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8004c80 con 0x7f63e80b4310 2023-11-24T06:08:33.654 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.647+0000 7f63cffff700 1 -- 172.21.15.183:0/3280407480 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d8004e00 con 0x7f63e80b4310 2023-11-24T06:08:33.654 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.647+0000 7f63cffff700 1 -- 172.21.15.183:0/3280407480 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d8007310 con 0x7f63e80b4310 2023-11-24T06:08:33.654 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.647+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e80b5700 con 0x7f63e80b4310 2023-11-24T06:08:33.654 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.647+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e80b5cc0 con 0x7f63e80b4310 2023-11-24T06:08:33.655 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.647+0000 7f63cffff700 1 -- 172.21.15.183:0/3280407480 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f63d801d3f0 con 0x7f63e80b4310 2023-11-24T06:08:33.655 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.647+0000 7f63cffff700 1 -- 172.21.15.183:0/3280407480 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f63d8013770 con 0x7f63e80b4310 2023-11-24T06:08:33.655 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.648+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e80b6030 con 0x7f63e80b4310 2023-11-24T06:08:33.655 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.650+0000 7f63cffff700 1 -- 172.21.15.183:0/3280407480 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f63e80cedc0 con 0x7f63e80b4310 2023-11-24T06:08:33.678 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:33.676+0000 7f828fef3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:08:33.736 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.733+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "377f83d5-5100-48d5-8a3d-a78fcdd2175d", "id": 1} v 0) v1 -- 0x7f63e80b5560 con 0x7f63e80b4310 2023-11-24T06:08:33.741 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.739+0000 7f63cffff700 1 -- 172.21.15.183:0/3280407480 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "377f83d5-5100-48d5-8a3d-a78fcdd2175d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f63e80b5560 con 0x7f63e80b4310 2023-11-24T06:08:33.742 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80b4310 msgr2=0x7f63e80ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80b4310 0x7f63e80ca990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c580 tx=0x7f63d800c890 comp rx=0 tx=0).stop 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 shutdown_connections 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f63e815e080 0x7f63e80b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63e80b4310 0x7f63e80ca990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 --2- 172.21.15.183:0/3280407480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63e80ce430 0x7f63e80b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 >> 172.21.15.183:0/3280407480 conn(0x7f63e80b3780 msgr2=0x7f63e806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 shutdown_connections 2023-11-24T06:08:33.743 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.740+0000 7f63ed82e700 1 -- 172.21.15.183:0/3280407480 wait complete. 2023-11-24T06:08:33.756 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 06d31208-d031-4318-ab2a-50873f3d20c4 2 2023-11-24T06:08:33.760 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:33.758+0000 7fca4b0b1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:08:33.802 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:33.800+0000 7f828fef3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:08:33.839 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:33.836+0000 7fca4b0b1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:08:33.879 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.877+0000 7f099f92b700 1 Processor -- start 2023-11-24T06:08:33.879 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.877+0000 7f099f92b700 1 -- start start 2023-11-24T06:08:33.881 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099f92b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f09980cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.881 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099f92b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09980d0140 con 0x7f099815bf50 2023-11-24T06:08:33.881 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099f92b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09980d03e0 con 0x7f09980c6c30 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099f92b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099815c330 con 0x7f09980cf110 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099dec8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f09980cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099cec6700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f09980c6c30 0x7f09980cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:34632/0 (socket says 172.21.15.183:34632) 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099cec6700 1 -- 172.21.15.183:0/3313777742 learned_addr learned my addr 172.21.15.183:0/3313777742 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f098ffff700 1 -- 172.21.15.183:0/3313777742 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905467606 0 0) 0x7f09980d03e0 con 0x7f09980c6c30 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099dec8700 1 -- 172.21.15.183:0/3313777742 >> v1:172.21.15.183:6789/0 conn(0x7f09980c6c30 legacy=0x7f09980cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099dec8700 1 -- 172.21.15.183:0/3313777742 >> v1:172.21.15.97:6789/0 conn(0x7f099815bf50 legacy=0x7f09980ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099dec8700 1 -- 172.21.15.183:0/3313777742 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09980d06c0 con 0x7f09980cf110 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f099dec8700 1 --2- 172.21.15.183:0/3313777742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f09980cd6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0990009dc0 tx=0x7f099000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2857ede2a82657a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f098ffff700 1 -- 172.21.15.183:0/3313777742 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099000e070 con 0x7f09980cf110 2023-11-24T06:08:33.882 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f098ffff700 1 -- 172.21.15.183:0/3313777742 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099000bbd0 con 0x7f09980cf110 2023-11-24T06:08:33.883 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.878+0000 7f098ffff700 1 -- 172.21.15.183:0/3313777742 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990012430 con 0x7f09980cf110 2023-11-24T06:08:33.883 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.879+0000 7f099f92b700 1 -- 172.21.15.183:0/3313777742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 msgr2=0x7f09980cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.883 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.879+0000 7f099f92b700 1 --2- 172.21.15.183:0/3313777742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f09980cd6d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0990009dc0 tx=0x7f099000b330 comp rx=0 tx=0).stop 2023-11-24T06:08:33.883 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.879+0000 7f099f92b700 1 -- 172.21.15.183:0/3313777742 shutdown_connections 2023-11-24T06:08:33.883 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.879+0000 7f099f92b700 1 --2- 172.21.15.183:0/3313777742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f09980cd6d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.883 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.879+0000 7f099f92b700 1 -- 172.21.15.183:0/3313777742 >> 172.21.15.183:0/3313777742 conn(0x7f09980b3780 msgr2=0x7f09980b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:33.883 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.879+0000 7f099f92b700 1 -- 172.21.15.183:0/3313777742 shutdown_connections 2023-11-24T06:08:33.883 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.879+0000 7f099f92b700 1 -- 172.21.15.183:0/3313777742 wait complete. 2023-11-24T06:08:33.884 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.881+0000 7f099f92b700 1 Processor -- start 2023-11-24T06:08:33.884 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.881+0000 7f099f92b700 1 -- start start 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099f92b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980c6c30 0x7f099814a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099f92b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f099814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099f92b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f099815bf50 0x7f099814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099f92b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099814d890 con 0x7f099815bf50 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099f92b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099814cf60 con 0x7f09980c6c30 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099f92b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099814d240 con 0x7f09980cf110 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099cec6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f099814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099d6c7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980c6c30 0x7f099814a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099d6c7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980c6c30 0x7f099814a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36174/0 (socket says 172.21.15.183:36174) 2023-11-24T06:08:33.886 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099cec6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f099814f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:46836/0 (socket says 172.21.15.183:46836) 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099d6c7700 1 -- 172.21.15.183:0/2967386711 learned_addr learned my addr 172.21.15.183:0/2967386711 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099dec8700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f099815bf50 0x7f099814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099cec6700 1 -- 172.21.15.183:0/2967386711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980c6c30 msgr2=0x7f099814a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099cec6700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980c6c30 0x7f099814a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099cec6700 1 -- 172.21.15.183:0/2967386711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f099815bf50 msgr2=0x7f099814f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099cec6700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f099815bf50 0x7f099814f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099cec6700 1 -- 172.21.15.183:0/2967386711 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990009a70 con 0x7f09980cf110 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.882+0000 7f099d6c7700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980c6c30 0x7f099814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.883+0000 7f099cec6700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f099814f3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f098800c470 tx=0x7f098800c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.883+0000 7f098dffb700 1 -- 172.21.15.183:0/2967386711 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988004c80 con 0x7f09980cf110 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.883+0000 7f098dffb700 1 -- 172.21.15.183:0/2967386711 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988004de0 con 0x7f09980cf110 2023-11-24T06:08:33.887 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.883+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099814d520 con 0x7f09980cf110 2023-11-24T06:08:33.888 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.883+0000 7f098dffb700 1 -- 172.21.15.183:0/2967386711 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988007310 con 0x7f09980cf110 2023-11-24T06:08:33.888 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.883+0000 7f098dffb700 1 -- 172.21.15.183:0/2967386711 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f09880133c0 con 0x7f09980cf110 2023-11-24T06:08:33.888 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.883+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09981506b0 con 0x7f09980cf110 2023-11-24T06:08:33.888 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.883+0000 7f098dffb700 1 -- 172.21.15.183:0/2967386711 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0988013cc0 con 0x7f09980cf110 2023-11-24T06:08:33.888 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.884+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0998075350 con 0x7f09980cf110 2023-11-24T06:08:33.888 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.886+0000 7f098dffb700 1 -- 172.21.15.183:0/2967386711 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f09880138a0 con 0x7f09980cf110 2023-11-24T06:08:33.891 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:33.889+0000 7f828fef3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:08:33.919 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:33.916+0000 7fca4b0b1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:08:33.974 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.972+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "06d31208-d031-4318-ab2a-50873f3d20c4", "id": 2} v 0) v1 -- 0x7f0998068fb0 con 0x7f09980cf110 2023-11-24T06:08:33.974 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:33.972+0000 7f828fef3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:08:33.979 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.977+0000 7f098dffb700 1 -- 172.21.15.183:0/2967386711 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "06d31208-d031-4318-ab2a-50873f3d20c4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0988013b00 con 0x7f09980cf110 2023-11-24T06:08:33.979 INFO:teuthology.orchestra.run.smithi183.stdout:2 2023-11-24T06:08:33.980 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 msgr2=0x7f099814f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:33.980 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f099814f3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f098800c470 tx=0x7f098800c780 comp rx=0 tx=0).stop 2023-11-24T06:08:33.981 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 shutdown_connections 2023-11-24T06:08:33.981 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f099815bf50 0x7f099814f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.981 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980c6c30 0x7f099814a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.981 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 --2- 172.21.15.183:0/2967386711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf110 0x7f099814f3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:33.981 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 >> 172.21.15.183:0/2967386711 conn(0x7f09980b3780 msgr2=0x7f09980c7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:33.981 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 shutdown_connections 2023-11-24T06:08:33.981 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:33.978+0000 7f099f92b700 1 -- 172.21.15.183:0/2967386711 wait complete. 2023-11-24T06:08:33.992 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new fa9af3f9-93c3-42e0-9d84-25d7a254ad66 3 2023-11-24T06:08:34.116 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.113+0000 7fe8370fe700 1 Processor -- start 2023-11-24T06:08:34.116 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.113+0000 7fe8370fe700 1 -- start start 2023-11-24T06:08:34.118 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe8370fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe8300b49a0 0x7fe83015b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe8370fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe83006b0f0 0x7fe83006b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe8370fe700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83015e9f0 con 0x7fe83006e370 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe8370fe700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83015ec90 con 0x7fe8300b49a0 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe8370fe700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83015ef70 con 0x7fe83006b0f0 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe83569b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe83006b0f0 0x7fe83006b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe82ffff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fe83006e370 0x7fe8300b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:34646/0 (socket says 172.21.15.183:34646) 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe83569b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe83006b0f0 0x7fe83006b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36184/0 (socket says 172.21.15.183:36184) 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.114+0000 7fe82ffff700 1 -- 172.21.15.183:0/1009301704 learned_addr learned my addr 172.21.15.183:0/1009301704 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe834e9a700 1 --2- 172.21.15.183:0/1009301704 >> v2:172.21.15.97:3300/0 conn(0x7fe8300b49a0 0x7fe83015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe83569b700 1 -- 172.21.15.183:0/1009301704 >> v2:172.21.15.97:3300/0 conn(0x7fe8300b49a0 msgr2=0x7fe83015b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe83569b700 1 --2- 172.21.15.183:0/1009301704 >> v2:172.21.15.97:3300/0 conn(0x7fe8300b49a0 0x7fe83015b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.119 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe83569b700 1 -- 172.21.15.183:0/1009301704 >> v1:172.21.15.183:6789/0 conn(0x7fe83006e370 legacy=0x7fe8300b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe82f7fe700 1 -- 172.21.15.183:0/1009301704 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976528683 0 0) 0x7fe83015e9f0 con 0x7fe83006e370 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe83569b700 1 -- 172.21.15.183:0/1009301704 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83015f250 con 0x7fe83006b0f0 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe83569b700 1 --2- 172.21.15.183:0/1009301704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83006b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe83015c3b0 tx=0x7fe82800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=307aa2b744b7a492 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe82f7fe700 1 -- 172.21.15.183:0/1009301704 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800e070 con 0x7fe83006b0f0 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe82f7fe700 1 -- 172.21.15.183:0/1009301704 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82800bb20 con 0x7fe83006b0f0 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.115+0000 7fe82f7fe700 1 -- 172.21.15.183:0/1009301704 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828012430 con 0x7fe83006b0f0 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.116+0000 7fe8370fe700 1 -- 172.21.15.183:0/1009301704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 msgr2=0x7fe83006b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.116+0000 7fe8370fe700 1 --2- 172.21.15.183:0/1009301704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83006b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe83015c3b0 tx=0x7fe82800b330 comp rx=0 tx=0).stop 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.116+0000 7fe8370fe700 1 -- 172.21.15.183:0/1009301704 shutdown_connections 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.116+0000 7fe8370fe700 1 --2- 172.21.15.183:0/1009301704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83006b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.120 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.116+0000 7fe8370fe700 1 --2- 172.21.15.183:0/1009301704 >> v2:172.21.15.97:3300/0 conn(0x7fe8300b49a0 0x7fe83015b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.121 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.116+0000 7fe8370fe700 1 -- 172.21.15.183:0/1009301704 >> 172.21.15.183:0/1009301704 conn(0x7fe8300b28f0 msgr2=0x7fe8300b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:34.121 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.116+0000 7fe8370fe700 1 -- 172.21.15.183:0/1009301704 shutdown_connections 2023-11-24T06:08:34.121 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.116+0000 7fe8370fe700 1 -- 172.21.15.183:0/1009301704 wait complete. 2023-11-24T06:08:34.123 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.117+0000 7fe8370fe700 1 Processor -- start 2023-11-24T06:08:34.123 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.117+0000 7fe8370fe700 1 -- start start 2023-11-24T06:08:34.123 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe8370fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.123 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe8370fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83006e370 0x7fe830154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe8370fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8300b49a0 0x7fe8301553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe8370fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830156240 con 0x7fe83006e370 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe8370fe700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300b1210 con 0x7fe83006b0f0 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe8370fe700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300b14f0 con 0x7fe8300b49a0 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe834e9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe834e9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83014fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36186/0 (socket says 172.21.15.183:36186) 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe83569b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8300b49a0 0x7fe8301553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe834e9a700 1 -- 172.21.15.183:0/3145826141 learned_addr learned my addr 172.21.15.183:0/3145826141 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe82ffff700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83006e370 0x7fe830154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe834e9a700 1 -- 172.21.15.183:0/3145826141 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8300b49a0 msgr2=0x7fe8301553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.124 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe834e9a700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8300b49a0 0x7fe8301553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe834e9a700 1 -- 172.21.15.183:0/3145826141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83006e370 msgr2=0x7fe830154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe834e9a700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83006e370 0x7fe830154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.118+0000 7fe834e9a700 1 -- 172.21.15.183:0/3145826141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe828009a70 con 0x7fe83006b0f0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe83569b700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8300b49a0 0x7fe8301553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe834e9a700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83014fb30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe82400ed70 tx=0x7fe82400d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe82d7fa700 1 -- 172.21.15.183:0/3145826141 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82400dd90 con 0x7fe83006b0f0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe82d7fa700 1 -- 172.21.15.183:0/3145826141 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe824010440 con 0x7fe83006b0f0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8300b17d0 con 0x7fe83006b0f0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe82d7fa700 1 -- 172.21.15.183:0/3145826141 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8240107d0 con 0x7fe83006b0f0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8300af340 con 0x7fe83006b0f0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe82d7fa700 1 -- 172.21.15.183:0/3145826141 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe824005d20 con 0x7fe83006b0f0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.119+0000 7fe82d7fa700 1 -- 172.21.15.183:0/3145826141 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe824006730 con 0x7fe83006b0f0 2023-11-24T06:08:34.125 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.120+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe830068fb0 con 0x7fe83006b0f0 2023-11-24T06:08:34.126 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.122+0000 7fe82d7fa700 1 -- 172.21.15.183:0/3145826141 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe824036ce0 con 0x7fe83006b0f0 2023-11-24T06:08:34.209 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.206+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fa9af3f9-93c3-42e0-9d84-25d7a254ad66", "id": 3} v 0) v1 -- 0x7fe8301566b0 con 0x7fe83006b0f0 2023-11-24T06:08:34.214 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.212+0000 7fe82d7fa700 1 -- 172.21.15.183:0/3145826141 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa9af3f9-93c3-42e0-9d84-25d7a254ad66", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe8301566b0 con 0x7fe83006b0f0 2023-11-24T06:08:34.214 INFO:teuthology.orchestra.run.smithi183.stdout:3 2023-11-24T06:08:34.215 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.212+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 msgr2=0x7fe83014fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.215 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.212+0000 7fe8370fe700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83014fb30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe82400ed70 tx=0x7fe82400d630 comp rx=0 tx=0).stop 2023-11-24T06:08:34.215 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.213+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 shutdown_connections 2023-11-24T06:08:34.215 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.213+0000 7fe8370fe700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe8300b49a0 0x7fe8301553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.216 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.213+0000 7fe8370fe700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe83006b0f0 0x7fe83014fb30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.216 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.213+0000 7fe8370fe700 1 --2- 172.21.15.183:0/3145826141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe83006e370 0x7fe830154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.216 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.213+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 >> 172.21.15.183:0/3145826141 conn(0x7fe8300b28f0 msgr2=0x7fe830074820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:34.216 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.213+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 shutdown_connections 2023-11-24T06:08:34.216 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.213+0000 7fe8370fe700 1 -- 172.21.15.183:0/3145826141 wait complete. 2023-11-24T06:08:34.227 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 2efd95ef-8292-4722-b41f-210931316c9f 4 2023-11-24T06:08:34.347 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.344+0000 7fba993a3700 1 Processor -- start 2023-11-24T06:08:34.347 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.344+0000 7fba993a3700 1 -- start start 2023-11-24T06:08:34.349 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba993a3700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba9415bf50 0x7fba940ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.349 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba993a3700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fba940ce830 0x7fba940cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.349 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba993a3700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba940d0140 con 0x7fba940cf1f0 2023-11-24T06:08:34.349 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba993a3700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9415c330 con 0x7fba9415bf50 2023-11-24T06:08:34.349 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba993a3700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba940d03e0 con 0x7fba940ce830 2023-11-24T06:08:34.349 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba927fc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fba940ce830 0x7fba940cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba937fe700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fba940cf1f0 0x7fba940c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:34660/0 (socket says 172.21.15.183:34660) 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba937fe700 1 -- 172.21.15.183:0/3094670316 learned_addr learned my addr 172.21.15.183:0/3094670316 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba92ffd700 1 --2- 172.21.15.183:0/3094670316 >> v2:172.21.15.97:3300/0 conn(0x7fba9415bf50 0x7fba940ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.345+0000 7fba91ffb700 1 -- 172.21.15.183:0/3094670316 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847515914 0 0) 0x7fba940d0140 con 0x7fba940cf1f0 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba91ffb700 1 -- 172.21.15.183:0/3094670316 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba7c0035b0 con 0x7fba940cf1f0 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba927fc700 1 -- 172.21.15.183:0/3094670316 >> v2:172.21.15.97:3300/0 conn(0x7fba9415bf50 msgr2=0x7fba940ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba927fc700 1 --2- 172.21.15.183:0/3094670316 >> v2:172.21.15.97:3300/0 conn(0x7fba9415bf50 0x7fba940ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba927fc700 1 -- 172.21.15.183:0/3094670316 >> v1:172.21.15.183:6789/0 conn(0x7fba940cf1f0 legacy=0x7fba940c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba927fc700 1 -- 172.21.15.183:0/3094670316 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba940d06c0 con 0x7fba940ce830 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba927fc700 1 --2- 172.21.15.183:0/3094670316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba940ce830 0x7fba940cecb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fba94072420 tx=0x7fba88009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e212c093f1fadf2a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba91ffb700 1 -- 172.21.15.183:0/3094670316 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88011830 con 0x7fba940ce830 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba91ffb700 1 -- 172.21.15.183:0/3094670316 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88011990 con 0x7fba940ce830 2023-11-24T06:08:34.350 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba91ffb700 1 -- 172.21.15.183:0/3094670316 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88011d20 con 0x7fba940ce830 2023-11-24T06:08:34.351 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba993a3700 1 -- 172.21.15.183:0/3094670316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba940ce830 msgr2=0x7fba940cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.351 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.346+0000 7fba993a3700 1 --2- 172.21.15.183:0/3094670316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba940ce830 0x7fba940cecb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fba94072420 tx=0x7fba88009b20 comp rx=0 tx=0).stop 2023-11-24T06:08:34.351 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.347+0000 7fba993a3700 1 -- 172.21.15.183:0/3094670316 shutdown_connections 2023-11-24T06:08:34.351 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.347+0000 7fba993a3700 1 --2- 172.21.15.183:0/3094670316 >> v2:172.21.15.97:3300/0 conn(0x7fba9415bf50 0x7fba940ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.351 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.347+0000 7fba993a3700 1 --2- 172.21.15.183:0/3094670316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba940ce830 0x7fba940cecb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.351 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.347+0000 7fba993a3700 1 -- 172.21.15.183:0/3094670316 >> 172.21.15.183:0/3094670316 conn(0x7fba940b3780 msgr2=0x7fba940b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:34.351 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.347+0000 7fba993a3700 1 -- 172.21.15.183:0/3094670316 shutdown_connections 2023-11-24T06:08:34.351 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.347+0000 7fba993a3700 1 -- 172.21.15.183:0/3094670316 wait complete. 2023-11-24T06:08:34.353 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.348+0000 7fba993a3700 1 Processor -- start 2023-11-24T06:08:34.353 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.348+0000 7fba993a3700 1 -- start start 2023-11-24T06:08:34.353 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba993a3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940ce830 0x7fba941550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba993a3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba940cf1f0 0x7fba940b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba993a3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba9415bf50 0x7fba940b7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba993a3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba94155630 con 0x7fba940ce830 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba993a3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba941e59d0 con 0x7fba9415bf50 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba993a3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba941e5c00 con 0x7fba940cf1f0 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba927fc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba940cf1f0 0x7fba940b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba937fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba9415bf50 0x7fba940b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba927fc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba940cf1f0 0x7fba940b6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:46860/0 (socket says 172.21.15.183:46860) 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba927fc700 1 -- 172.21.15.183:0/3639183440 learned_addr learned my addr 172.21.15.183:0/3639183440 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba92ffd700 1 --2- 172.21.15.183:0/3639183440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940ce830 0x7fba941550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba937fe700 1 -- 172.21.15.183:0/3639183440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba940cf1f0 msgr2=0x7fba940b6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba937fe700 1 --2- 172.21.15.183:0/3639183440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba940cf1f0 0x7fba940b6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.354 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba937fe700 1 -- 172.21.15.183:0/3639183440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940ce830 msgr2=0x7fba941550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba937fe700 1 --2- 172.21.15.183:0/3639183440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940ce830 0x7fba941550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba937fe700 1 -- 172.21.15.183:0/3639183440 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880094a0 con 0x7fba9415bf50 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba937fe700 1 --2- 172.21.15.183:0/3639183440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba9415bf50 0x7fba940b7360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fba940cc1b0 tx=0x7fba8c00cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba7b7fe700 1 -- 172.21.15.183:0/3639183440 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00a9e0 con 0x7fba9415bf50 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba7b7fe700 1 -- 172.21.15.183:0/3639183440 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c00ab40 con 0x7fba9415bf50 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba941e5ee0 con 0x7fba9415bf50 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.349+0000 7fba7b7fe700 1 -- 172.21.15.183:0/3639183440 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c012400 con 0x7fba9415bf50 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.350+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba941e64a0 con 0x7fba9415bf50 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.350+0000 7fba7b7fe700 1 -- 172.21.15.183:0/3639183440 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba8c012560 con 0x7fba9415bf50 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.350+0000 7fba7b7fe700 1 -- 172.21.15.183:0/3639183440 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fba8c01c950 con 0x7fba9415bf50 2023-11-24T06:08:34.355 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.350+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba941e6810 con 0x7fba9415bf50 2023-11-24T06:08:34.356 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.352+0000 7fba7b7fe700 1 -- 172.21.15.183:0/3639183440 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fba8c021060 con 0x7fba9415bf50 2023-11-24T06:08:34.361 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:34.358+0000 7fca4b0b1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:08:34.431 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:34.429+0000 7f828fef3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:08:34.439 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.437+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2efd95ef-8292-4722-b41f-210931316c9f", "id": 4} v 0) v1 -- 0x7fba94068fb0 con 0x7fba9415bf50 2023-11-24T06:08:34.444 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.442+0000 7fba7b7fe700 1 -- 172.21.15.183:0/3639183440 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2efd95ef-8292-4722-b41f-210931316c9f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba8c017070 con 0x7fba9415bf50 2023-11-24T06:08:34.444 INFO:teuthology.orchestra.run.smithi183.stdout:4 2023-11-24T06:08:34.445 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba9415bf50 msgr2=0x7fba940b7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.445 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 --2- 172.21.15.183:0/3639183440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba9415bf50 0x7fba940b7360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fba940cc1b0 tx=0x7fba8c00cd30 comp rx=0 tx=0).stop 2023-11-24T06:08:34.445 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 shutdown_connections 2023-11-24T06:08:34.445 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 --2- 172.21.15.183:0/3639183440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba9415bf50 0x7fba940b7360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.446 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 --2- 172.21.15.183:0/3639183440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba940ce830 0x7fba941550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.446 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 --2- 172.21.15.183:0/3639183440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba940cf1f0 0x7fba940b6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.446 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 >> 172.21.15.183:0/3639183440 conn(0x7fba940b3780 msgr2=0x7fba940c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:34.446 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 shutdown_connections 2023-11-24T06:08:34.446 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.443+0000 7fba993a3700 1 -- 172.21.15.183:0/3639183440 wait complete. 2023-11-24T06:08:34.457 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 1ff1124a-2cde-4e5a-9150-cea260ff05bb 5 2023-11-24T06:08:34.580 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb05b09e700 1 Processor -- start 2023-11-24T06:08:34.580 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb05b09e700 1 -- start start 2023-11-24T06:08:34.581 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb05b09e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb05406ef80 0x7fb0540ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.581 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb05b09e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540cf550 con 0x7fb0540b4230 2023-11-24T06:08:34.581 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb05b09e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540cc9b0 con 0x7fb05415e2d0 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb05b09e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05415e6b0 con 0x7fb05406ef80 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb05963b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb05406ef80 0x7fb0540ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb058e3a700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fb05415e2d0 0x7fb0540ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:34668/0 (socket says 172.21.15.183:34668) 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.578+0000 7fb058e3a700 1 -- 172.21.15.183:0/3459937913 learned_addr learned my addr 172.21.15.183:0/3459937913 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.579+0000 7fb05963b700 1 -- 172.21.15.183:0/3459937913 >> v1:172.21.15.183:6789/0 conn(0x7fb05415e2d0 legacy=0x7fb0540ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.579+0000 7fb05963b700 1 -- 172.21.15.183:0/3459937913 >> v1:172.21.15.97:6789/0 conn(0x7fb0540b4230 legacy=0x7fb0540cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.579+0000 7fb05963b700 1 -- 172.21.15.183:0/3459937913 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540cf270 con 0x7fb05406ef80 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.579+0000 7fb05963b700 1 --2- 172.21.15.183:0/3459937913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05406ef80 0x7fb0540ceb70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009d90 tx=0x7fb04c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa300b80e7ca290e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.579+0000 7fb0537fe700 1 -- 172.21.15.183:0/3459937913 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00e070 con 0x7fb05406ef80 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.579+0000 7fb0537fe700 1 -- 172.21.15.183:0/3459937913 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c00bbd0 con 0x7fb05406ef80 2023-11-24T06:08:34.582 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.579+0000 7fb0537fe700 1 -- 172.21.15.183:0/3459937913 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c012430 con 0x7fb05406ef80 2023-11-24T06:08:34.583 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.580+0000 7fb05b09e700 1 -- 172.21.15.183:0/3459937913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05406ef80 msgr2=0x7fb0540ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.583 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.580+0000 7fb05b09e700 1 --2- 172.21.15.183:0/3459937913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05406ef80 0x7fb0540ceb70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009d90 tx=0x7fb04c00b330 comp rx=0 tx=0).stop 2023-11-24T06:08:34.584 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.580+0000 7fb05b09e700 1 -- 172.21.15.183:0/3459937913 shutdown_connections 2023-11-24T06:08:34.584 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.580+0000 7fb05b09e700 1 --2- 172.21.15.183:0/3459937913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05406ef80 0x7fb0540ceb70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.584 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.580+0000 7fb05b09e700 1 -- 172.21.15.183:0/3459937913 >> 172.21.15.183:0/3459937913 conn(0x7fb0540b3710 msgr2=0x7fb0540b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:34.584 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.580+0000 7fb05b09e700 1 -- 172.21.15.183:0/3459937913 shutdown_connections 2023-11-24T06:08:34.584 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.580+0000 7fb05b09e700 1 -- 172.21.15.183:0/3459937913 wait complete. 2023-11-24T06:08:34.584 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.581+0000 7fb05b09e700 1 Processor -- start 2023-11-24T06:08:34.584 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.581+0000 7fb05b09e700 1 -- start start 2023-11-24T06:08:34.586 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb05b09e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb05406ef80 0x7fb0541f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.586 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb05b09e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0540b4230 0x7fb0540cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.586 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb05b09e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05415e2d0 0x7fb0540cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb05b09e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05415b9c0 con 0x7fb05415e2d0 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb05b09e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b51e0 con 0x7fb0540b4230 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb05b09e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b5d00 con 0x7fb05406ef80 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb058e3a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb05406ef80 0x7fb0541f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb053fff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0540b4230 0x7fb0540cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb058e3a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb05406ef80 0x7fb0541f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:46868/0 (socket says 172.21.15.183:46868) 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.582+0000 7fb058e3a700 1 -- 172.21.15.183:0/2658452883 learned_addr learned my addr 172.21.15.183:0/2658452883 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb05963b700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05415e2d0 0x7fb0540cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb058e3a700 1 -- 172.21.15.183:0/2658452883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0540b4230 msgr2=0x7fb0540cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb058e3a700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0540b4230 0x7fb0540cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb058e3a700 1 -- 172.21.15.183:0/2658452883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05415e2d0 msgr2=0x7fb0540cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb058e3a700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05415e2d0 0x7fb0540cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.587 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb058e3a700 1 -- 172.21.15.183:0/2658452883 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c009a70 con 0x7fb05406ef80 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb053fff700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0540b4230 0x7fb0540cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb058e3a700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb05406ef80 0x7fb0541f2ff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb04800c480 tx=0x7fb04800c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb0517fa700 1 -- 172.21.15.183:0/2658452883 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048005d70 con 0x7fb05406ef80 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb0517fa700 1 -- 172.21.15.183:0/2658452883 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04800ce90 con 0x7fb05406ef80 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb0517fa700 1 -- 172.21.15.183:0/2658452883 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0480072d0 con 0x7fb05406ef80 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0540b5460 con 0x7fb05406ef80 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0540b5a20 con 0x7fb05406ef80 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb0517fa700 1 -- 172.21.15.183:0/2658452883 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb04801c440 con 0x7fb05406ef80 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.583+0000 7fb0517fa700 1 -- 172.21.15.183:0/2658452883 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fb048007730 con 0x7fb05406ef80 2023-11-24T06:08:34.588 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.584+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb054068fb0 con 0x7fb05406ef80 2023-11-24T06:08:34.589 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.586+0000 7fb0517fa700 1 -- 172.21.15.183:0/2658452883 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb048021060 con 0x7fb05406ef80 2023-11-24T06:08:34.671 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.669+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1ff1124a-2cde-4e5a-9150-cea260ff05bb", "id": 5} v 0) v1 -- 0x7fb05406f560 con 0x7fb05406ef80 2023-11-24T06:08:34.681 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.679+0000 7fb0517fa700 1 -- 172.21.15.183:0/2658452883 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ff1124a-2cde-4e5a-9150-cea260ff05bb", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb048017070 con 0x7fb05406ef80 2023-11-24T06:08:34.682 INFO:teuthology.orchestra.run.smithi183.stdout:5 2023-11-24T06:08:34.683 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.680+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb05406ef80 msgr2=0x7fb0541f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.683 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.680+0000 7fb05b09e700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb05406ef80 0x7fb0541f2ff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb04800c480 tx=0x7fb04800c790 comp rx=0 tx=0).stop 2023-11-24T06:08:34.683 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.680+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 shutdown_connections 2023-11-24T06:08:34.683 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.680+0000 7fb05b09e700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05415e2d0 0x7fb0540cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.683 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.680+0000 7fb05b09e700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb05406ef80 0x7fb0541f2ff0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.684 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.680+0000 7fb05b09e700 1 --2- 172.21.15.183:0/2658452883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0540b4230 0x7fb0540cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.684 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.680+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 >> 172.21.15.183:0/2658452883 conn(0x7fb0540b3710 msgr2=0x7fb0540b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:34.684 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.681+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 shutdown_connections 2023-11-24T06:08:34.684 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.681+0000 7fb05b09e700 1 -- 172.21.15.183:0/2658452883 wait complete. 2023-11-24T06:08:34.694 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new b97538d0-893b-4eff-9cca-db8fb6c7e962 6 2023-11-24T06:08:34.815 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.813+0000 7fefab935700 1 Processor -- start 2023-11-24T06:08:34.816 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.813+0000 7fefab935700 1 -- start start 2023-11-24T06:08:34.818 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefab935700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415dfb0 0x7fefa406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.818 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefab935700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefa406e580 0x7fefa40ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.818 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefab935700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa40ad870 con 0x7fefa406c540 2023-11-24T06:08:34.818 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefab935700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa406c9c0 con 0x7fefa406e580 2023-11-24T06:08:34.818 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefab935700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa406eba0 con 0x7fefa415dfb0 2023-11-24T06:08:34.818 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefa9ed2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefa406e580 0x7fefa40ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.818 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefa96d1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415dfb0 0x7fefa406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.818 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefa9ed2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fefa406e580 0x7fefa40ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36218/0 (socket says 172.21.15.183:36218) 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefa96d1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415dfb0 0x7fefa406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:46876/0 (socket says 172.21.15.183:46876) 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.814+0000 7fefa9ed2700 1 -- 172.21.15.183:0/3964153731 learned_addr learned my addr 172.21.15.183:0/3964153731 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefa96d1700 1 -- 172.21.15.183:0/3964153731 >> v2:172.21.15.183:3300/0 conn(0x7fefa406e580 msgr2=0x7fefa40ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefa96d1700 1 --2- 172.21.15.183:0/3964153731 >> v2:172.21.15.183:3300/0 conn(0x7fefa406e580 0x7fefa40ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefa96d1700 1 -- 172.21.15.183:0/3964153731 >> v1:172.21.15.97:6789/0 conn(0x7fefa406c540 legacy=0x7fefa406de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefa96d1700 1 -- 172.21.15.183:0/3964153731 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa406ee80 con 0x7fefa415dfb0 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefa9ed2700 1 --2- 172.21.15.183:0/3964153731 >> v2:172.21.15.183:3300/0 conn(0x7fefa406e580 0x7fefa40ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefa96d1700 1 --2- 172.21.15.183:0/3964153731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415dfb0 0x7fefa406c000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009840 tx=0x7fefa0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6e098390afda1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fef9bfff700 1 -- 172.21.15.183:0/3964153731 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011830 con 0x7fefa415dfb0 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fef9bfff700 1 -- 172.21.15.183:0/3964153731 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa0011990 con 0x7fefa415dfb0 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fef9bfff700 1 -- 172.21.15.183:0/3964153731 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0011d20 con 0x7fefa415dfb0 2023-11-24T06:08:34.819 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefab935700 1 -- 172.21.15.183:0/3964153731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415dfb0 msgr2=0x7fefa406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.820 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefab935700 1 --2- 172.21.15.183:0/3964153731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415dfb0 0x7fefa406c000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009840 tx=0x7fefa0009b50 comp rx=0 tx=0).stop 2023-11-24T06:08:34.820 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefab935700 1 -- 172.21.15.183:0/3964153731 shutdown_connections 2023-11-24T06:08:34.820 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefab935700 1 --2- 172.21.15.183:0/3964153731 >> v2:172.21.15.183:3300/0 conn(0x7fefa406e580 0x7fefa40ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.820 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefab935700 1 --2- 172.21.15.183:0/3964153731 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa415dfb0 0x7fefa406c000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.820 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.815+0000 7fefab935700 1 -- 172.21.15.183:0/3964153731 >> 172.21.15.183:0/3964153731 conn(0x7fefa4071a70 msgr2=0x7fefa4071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:34.820 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.816+0000 7fefab935700 1 -- 172.21.15.183:0/3964153731 shutdown_connections 2023-11-24T06:08:34.820 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.816+0000 7fefab935700 1 -- 172.21.15.183:0/3964153731 wait complete. 2023-11-24T06:08:34.822 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.817+0000 7fefab935700 1 Processor -- start 2023-11-24T06:08:34.822 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.817+0000 7fefab935700 1 -- start start 2023-11-24T06:08:34.822 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefab935700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa406c540 0x7fefa41611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.822 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefab935700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa406e580 0x7fefa41466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.822 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefab935700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415dfb0 0x7fefa4160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:34.822 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefab935700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa415e610 con 0x7fefa406c540 2023-11-24T06:08:34.822 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefab935700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa4146c30 con 0x7fefa415dfb0 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefab935700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa40ce100 con 0x7fefa406e580 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa96d1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa406c540 0x7fefa41611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa9ed2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415dfb0 0x7fefa4160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa8ed0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa406e580 0x7fefa41466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa96d1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa406c540 0x7fefa41611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.183:47072/0 (socket says 172.21.15.183:47072) 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa9ed2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415dfb0 0x7fefa4160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36224/0 (socket says 172.21.15.183:36224) 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa96d1700 1 -- 172.21.15.183:0/2406205582 learned_addr learned my addr 172.21.15.183:0/2406205582 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa96d1700 1 -- 172.21.15.183:0/2406205582 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa406e580 msgr2=0x7fefa41466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa96d1700 1 --2- 172.21.15.183:0/2406205582 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa406e580 0x7fefa41466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa96d1700 1 -- 172.21.15.183:0/2406205582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415dfb0 msgr2=0x7fefa4160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa96d1700 1 --2- 172.21.15.183:0/2406205582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415dfb0 0x7fefa4160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.818+0000 7fefa96d1700 1 -- 172.21.15.183:0/2406205582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c009b30 con 0x7fefa406c540 2023-11-24T06:08:34.823 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.819+0000 7fefa96d1700 1 --2- 172.21.15.183:0/2406205582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa406c540 0x7fefa41611d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009fd0 tx=0x7fefa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:34.824 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.819+0000 7fef99ffb700 1 -- 172.21.15.183:0/2406205582 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00036a0 con 0x7fefa406c540 2023-11-24T06:08:34.824 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.819+0000 7fef99ffb700 1 -- 172.21.15.183:0/2406205582 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa001e070 con 0x7fefa406c540 2023-11-24T06:08:34.824 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.819+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa00094a0 con 0x7fefa406c540 2023-11-24T06:08:34.824 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.819+0000 7fef99ffb700 1 -- 172.21.15.183:0/2406205582 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0003a40 con 0x7fefa406c540 2023-11-24T06:08:34.824 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.819+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa40ce370 con 0x7fefa406c540 2023-11-24T06:08:34.824 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.819+0000 7fef99ffb700 1 -- 172.21.15.183:0/2406205582 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fefa002d450 con 0x7fefa406c540 2023-11-24T06:08:34.824 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.819+0000 7fef99ffb700 1 -- 172.21.15.183:0/2406205582 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fefa001aa10 con 0x7fefa406c540 2023-11-24T06:08:34.824 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.820+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa415e750 con 0x7fefa406c540 2023-11-24T06:08:34.825 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.823+0000 7fef99ffb700 1 -- 172.21.15.183:0/2406205582 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fefa415e750 con 0x7fefa406c540 2023-11-24T06:08:34.938 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.936+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b97538d0-893b-4eff-9cca-db8fb6c7e962", "id": 6} v 0) v1 -- 0x7fefa4068fb0 con 0x7fefa406c540 2023-11-24T06:08:34.942 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.939+0000 7fef99ffb700 1 -- 172.21.15.183:0/2406205582 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b97538d0-893b-4eff-9cca-db8fb6c7e962", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fefa0020070 con 0x7fefa406c540 2023-11-24T06:08:34.942 INFO:teuthology.orchestra.run.smithi183.stdout:6 2023-11-24T06:08:34.943 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.940+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa406c540 msgr2=0x7fefa41611d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:34.943 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.940+0000 7fefab935700 1 --2- 172.21.15.183:0/2406205582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa406c540 0x7fefa41611d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009fd0 tx=0x7fefa0004750 comp rx=0 tx=0).stop 2023-11-24T06:08:34.943 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.941+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 shutdown_connections 2023-11-24T06:08:34.943 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.941+0000 7fefab935700 1 --2- 172.21.15.183:0/2406205582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefa415dfb0 0x7fefa4160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.944 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.941+0000 7fefab935700 1 --2- 172.21.15.183:0/2406205582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fefa406c540 0x7fefa41611d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.944 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.941+0000 7fefab935700 1 --2- 172.21.15.183:0/2406205582 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fefa406e580 0x7fefa41466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:34.944 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.941+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 >> 172.21.15.183:0/2406205582 conn(0x7fefa4071a70 msgr2=0x7fefa40b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:34.944 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.941+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 shutdown_connections 2023-11-24T06:08:34.944 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:34.941+0000 7fefab935700 1 -- 172.21.15.183:0/2406205582 wait complete. 2023-11-24T06:08:34.955 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new d9b1704b-8def-4fe7-b80d-910954755e82 7 2023-11-24T06:08:34.989 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:34.986+0000 7fca4b0b1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:08:35.057 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:35.054+0000 7fca4b0b1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:08:35.078 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.076+0000 7fc27a380700 1 Processor -- start 2023-11-24T06:08:35.078 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.076+0000 7fc27a380700 1 -- start start 2023-11-24T06:08:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27a380700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc2740ae210 0x7fc27415f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27a380700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc27406c000 0x7fc27406c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27a380700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27406e8f0 con 0x7fc274161480 2023-11-24T06:08:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27a380700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2740afe80 con 0x7fc2740ae210 2023-11-24T06:08:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27a380700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27406c960 con 0x7fc27406c000 2023-11-24T06:08:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27891d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc27406c000 0x7fc27406c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc273fff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fc274161480 0x7fc27415f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.183:34690/0 (socket says 172.21.15.183:34690) 2023-11-24T06:08:35.081 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc273fff700 1 -- 172.21.15.183:0/4029071427 learned_addr learned my addr 172.21.15.183:0/4029071427 (peer_addr_for_me v1:172.21.15.183:0/0) 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc2737fe700 1 --2- 172.21.15.183:0/4029071427 >> v2:172.21.15.97:3300/0 conn(0x7fc2740ae210 0x7fc27415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27891d700 1 -- 172.21.15.183:0/4029071427 >> v2:172.21.15.97:3300/0 conn(0x7fc2740ae210 msgr2=0x7fc27415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27891d700 1 --2- 172.21.15.183:0/4029071427 >> v2:172.21.15.97:3300/0 conn(0x7fc2740ae210 0x7fc27415f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27891d700 1 -- 172.21.15.183:0/4029071427 >> v1:172.21.15.183:6789/0 conn(0x7fc274161480 legacy=0x7fc27415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc272ffd700 1 -- 172.21.15.183:0/4029071427 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912335701 0 0) 0x7fc27406e8f0 con 0x7fc274161480 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc27891d700 1 -- 172.21.15.183:0/4029071427 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740ae7d0 con 0x7fc27406c000 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.077+0000 7fc2737fe700 1 --2- 172.21.15.183:0/4029071427 >> v2:172.21.15.97:3300/0 conn(0x7fc2740ae210 0x7fc27415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27891d700 1 --2- 172.21.15.183:0/4029071427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc27406c000 0x7fc27406c420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2740d06d0 tx=0x7fc26c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0446105f16de653 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc272ffd700 1 -- 172.21.15.183:0/4029071427 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00e070 con 0x7fc27406c000 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc272ffd700 1 -- 172.21.15.183:0/4029071427 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c00bbd0 con 0x7fc27406c000 2023-11-24T06:08:35.082 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc272ffd700 1 -- 172.21.15.183:0/4029071427 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c012430 con 0x7fc27406c000 2023-11-24T06:08:35.083 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27a380700 1 -- 172.21.15.183:0/4029071427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc27406c000 msgr2=0x7fc27406c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.083 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27a380700 1 --2- 172.21.15.183:0/4029071427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc27406c000 0x7fc27406c420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2740d06d0 tx=0x7fc26c00b330 comp rx=0 tx=0).stop 2023-11-24T06:08:35.083 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27a380700 1 -- 172.21.15.183:0/4029071427 shutdown_connections 2023-11-24T06:08:35.083 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27a380700 1 --2- 172.21.15.183:0/4029071427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc27406c000 0x7fc27406c420 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.083 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27a380700 1 --2- 172.21.15.183:0/4029071427 >> v2:172.21.15.97:3300/0 conn(0x7fc2740ae210 0x7fc27415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.083 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27a380700 1 -- 172.21.15.183:0/4029071427 >> 172.21.15.183:0/4029071427 conn(0x7fc274071a70 msgr2=0x7fc274071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:35.083 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27a380700 1 -- 172.21.15.183:0/4029071427 shutdown_connections 2023-11-24T06:08:35.083 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.078+0000 7fc27a380700 1 -- 172.21.15.183:0/4029071427 wait complete. 2023-11-24T06:08:35.085 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.080+0000 7fc27a380700 1 Processor -- start 2023-11-24T06:08:35.085 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.080+0000 7fc27a380700 1 -- start start 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.080+0000 7fc27a380700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc27406c000 0x7fc27415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.080+0000 7fc27a380700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2740ae210 0x7fc2741f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.080+0000 7fc27a380700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc274161480 0x7fc2741f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.080+0000 7fc27a380700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2741f6b40 con 0x7fc27406c000 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.080+0000 7fc27a380700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27415b9d0 con 0x7fc274161480 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.080+0000 7fc27a380700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27415bcb0 con 0x7fc2740ae210 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc2737fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2740ae210 0x7fc2741f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27891d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc274161480 0x7fc2741f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc2737fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2740ae210 0x7fc2741f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.183:46894/0 (socket says 172.21.15.183:46894) 2023-11-24T06:08:35.086 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27891d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc274161480 0x7fc2741f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36236/0 (socket says 172.21.15.183:36236) 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc2737fe700 1 -- 172.21.15.183:0/3699504467 learned_addr learned my addr 172.21.15.183:0/3699504467 (peer_addr_for_me v2:172.21.15.183:0/0) 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc273fff700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc27406c000 0x7fc27415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27891d700 1 -- 172.21.15.183:0/3699504467 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2740ae210 msgr2=0x7fc2741f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27891d700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2740ae210 0x7fc2741f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27891d700 1 -- 172.21.15.183:0/3699504467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc27406c000 msgr2=0x7fc27415b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27891d700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc27406c000 0x7fc27415b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27891d700 1 -- 172.21.15.183:0/3699504467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c009a70 con 0x7fc274161480 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc273fff700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc27406c000 0x7fc27415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc2737fe700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2740ae210 0x7fc2741f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27891d700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc274161480 0x7fc2741f7310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc26c016000 tx=0x7fc26c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc270ff9700 1 -- 172.21.15.183:0/3699504467 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c012430 con 0x7fc274161480 2023-11-24T06:08:35.087 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc270ff9700 1 -- 172.21.15.183:0/3699504467 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c020070 con 0x7fc274161480 2023-11-24T06:08:35.088 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27415bf90 con 0x7fc274161480 2023-11-24T06:08:35.088 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc270ff9700 1 -- 172.21.15.183:0/3699504467 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c004280 con 0x7fc274161480 2023-11-24T06:08:35.088 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.081+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27415c550 con 0x7fc274161480 2023-11-24T06:08:35.088 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.082+0000 7fc270ff9700 1 -- 172.21.15.183:0/3699504467 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc26c0046e0 con 0x7fc274161480 2023-11-24T06:08:35.088 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.082+0000 7fc270ff9700 1 -- 172.21.15.183:0/3699504467 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fc26c022070 con 0x7fc274161480 2023-11-24T06:08:35.088 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.082+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27415c6e0 con 0x7fc274161480 2023-11-24T06:08:35.088 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.084+0000 7fc270ff9700 1 -- 172.21.15.183:0/3699504467 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc26c012e40 con 0x7fc274161480 2023-11-24T06:08:35.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:35.110+0000 7f828fef3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:08:35.173 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.170+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d9b1704b-8def-4fe7-b80d-910954755e82", "id": 7} v 0) v1 -- 0x7fc274068fb0 con 0x7fc274161480 2023-11-24T06:08:35.181 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.178+0000 7fc270ff9700 1 -- 172.21.15.183:0/3699504467 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9b1704b-8def-4fe7-b80d-910954755e82", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc26c027480 con 0x7fc274161480 2023-11-24T06:08:35.181 INFO:teuthology.orchestra.run.smithi183.stdout:7 2023-11-24T06:08:35.182 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc274161480 msgr2=0x7fc2741f7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.182 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc274161480 0x7fc2741f7310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc26c016000 tx=0x7fc26c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.182 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 shutdown_connections 2023-11-24T06:08:35.182 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc274161480 0x7fc2741f7310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.182 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc27406c000 0x7fc27415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.182 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 --2- 172.21.15.183:0/3699504467 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc2740ae210 0x7fc2741f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.182 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 >> 172.21.15.183:0/3699504467 conn(0x7fc274071a70 msgr2=0x7fc2740cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:35.182 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 shutdown_connections 2023-11-24T06:08:35.183 INFO:teuthology.orchestra.run.smithi183.stderr:2023-11-24T06:08:35.179+0000 7fc27a380700 1 -- 172.21.15.183:0/3699504467 wait complete. 2023-11-24T06:08:35.183 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:35.181+0000 7f828fef3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:08:35.193 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T06:08:35.194 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T06:08:35.196 INFO:tasks.ceph.osd.0:Started 2023-11-24T06:08:35.196 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T06:08:35.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T06:08:35.198 INFO:tasks.ceph.osd.1:Started 2023-11-24T06:08:35.198 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T06:08:35.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T06:08:35.200 INFO:tasks.ceph.osd.2:Started 2023-11-24T06:08:35.200 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T06:08:35.200 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T06:08:35.202 INFO:tasks.ceph.osd.3:Started 2023-11-24T06:08:35.202 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T06:08:35.202 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T06:08:35.204 INFO:tasks.ceph.osd.4:Started 2023-11-24T06:08:35.204 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T06:08:35.205 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T06:08:35.206 INFO:tasks.ceph.osd.5:Started 2023-11-24T06:08:35.206 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T06:08:35.207 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T06:08:35.208 INFO:tasks.ceph.osd.6:Started 2023-11-24T06:08:35.209 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T06:08:35.209 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T06:08:35.210 INFO:tasks.ceph.osd.7:Started 2023-11-24T06:08:35.211 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:08:35.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.359+0000 7f7ffb2e5700 1 Processor -- start 2023-11-24T06:08:35.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.360+0000 7f7ffb2e5700 1 -- start start 2023-11-24T06:08:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.360+0000 7f7ffb2e5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff415e080 0x7f7ff40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.360+0000 7f7ffb2e5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7ff406c710 0x7f7ff40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.360+0000 7f7ffb2e5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff40ae430 con 0x7f7ff40af4d0 2023-11-24T06:08:35.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.360+0000 7f7ffb2e5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff40af0b0 con 0x7f7ff406c710 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.360+0000 7f7ffb2e5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40b05a0 con 0x7f7ff415e080 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff8880700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ff40af4d0 0x7f7ff406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36856/0 (socket says 172.21.15.97:36856) 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff8880700 1 -- 172.21.15.97:0/3463957854 learned_addr learned my addr 172.21.15.97:0/3463957854 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff9081700 1 --2- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff415e080 0x7f7ff40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff9882700 1 --2- 172.21.15.97:0/3463957854 >> v2:172.21.15.183:3300/0 conn(0x7f7ff406c710 0x7f7ff40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7febfff700 1 -- 172.21.15.97:0/3463957854 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981254848 0 0) 0x7f7ff40ae430 con 0x7f7ff40af4d0 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7febfff700 1 -- 172.21.15.97:0/3463957854 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe00035b0 con 0x7f7ff40af4d0 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff9882700 1 -- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff415e080 msgr2=0x7f7ff40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff9882700 1 --2- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff415e080 0x7f7ff40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff9882700 1 -- 172.21.15.97:0/3463957854 >> v1:172.21.15.97:6789/0 conn(0x7f7ff40af4d0 legacy=0x7f7ff406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff9882700 1 -- 172.21.15.97:0/3463957854 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff40ae710 con 0x7f7ff406c710 2023-11-24T06:08:35.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff9081700 1 --2- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff415e080 0x7f7ff40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:35.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7ff9882700 1 --2- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff406c710 0x7f7ff40afb90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009dc0 tx=0x7f7fec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574affaa9c8386ec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:35.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7febfff700 1 -- 172.21.15.97:0/3463957854 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00e070 con 0x7f7ff406c710 2023-11-24T06:08:35.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7febfff700 1 -- 172.21.15.97:0/3463957854 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec00bae0 con 0x7f7ff406c710 2023-11-24T06:08:35.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.361+0000 7f7febfff700 1 -- 172.21.15.97:0/3463957854 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec00be90 con 0x7f7ff406c710 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.362+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff406c710 msgr2=0x7f7ff40afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.362+0000 7f7ffb2e5700 1 --2- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff406c710 0x7f7ff40afb90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009dc0 tx=0x7f7fec00b330 comp rx=0 tx=0).stop 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.362+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/3463957854 shutdown_connections 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.362+0000 7f7ffb2e5700 1 --2- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff406c710 0x7f7ff40afb90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.362+0000 7f7ffb2e5700 1 --2- 172.21.15.97:0/3463957854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff415e080 0x7f7ff40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.362+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/3463957854 >> 172.21.15.97:0/3463957854 conn(0x7f7ff4071b90 msgr2=0x7f7ff4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.362+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/3463957854 shutdown_connections 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.362+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/3463957854 wait complete. 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.364+0000 7f7ffb2e5700 1 Processor -- start 2023-11-24T06:08:35.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.364+0000 7f7ffb2e5700 1 -- start start 2023-11-24T06:08:35.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ffb2e5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff406c710 0x7f7ff4159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ffb2e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40af4d0 0x7f7ff4152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ffb2e5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff415e080 0x7f7ff4153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ffb2e5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4154a10 con 0x7f7ff40af4d0 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ffb2e5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4154cf0 con 0x7f7ff415e080 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ffb2e5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4154fd0 con 0x7f7ff406c710 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40af4d0 0x7f7ff4152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff9081700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff406c710 0x7f7ff4159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40af4d0 0x7f7ff4152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44410/0 (socket says 172.21.15.97:44410) 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 -- 172.21.15.97:0/64483198 learned_addr learned my addr 172.21.15.97:0/64483198 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff9882700 1 --2- 172.21.15.97:0/64483198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff415e080 0x7f7ff4153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:35.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 -- 172.21.15.97:0/64483198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff406c710 msgr2=0x7f7ff4159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 --2- 172.21.15.97:0/64483198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff406c710 0x7f7ff4159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 -- 172.21.15.97:0/64483198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff415e080 msgr2=0x7f7ff4153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 --2- 172.21.15.97:0/64483198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff415e080 0x7f7ff4153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 -- 172.21.15.97:0/64483198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff00094a0 con 0x7f7ff40af4d0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ff8880700 1 --2- 172.21.15.97:0/64483198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40af4d0 0x7f7ff4152ef0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4005910 tx=0x7f7fe400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7fe9ffb700 1 -- 172.21.15.97:0/64483198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4005d70 con 0x7f7ff40af4d0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7fe9ffb700 1 -- 172.21.15.97:0/64483198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe40070d0 con 0x7f7ff40af4d0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec009a70 con 0x7f7ff40af4d0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.365+0000 7f7fe9ffb700 1 -- 172.21.15.97:0/64483198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4007460 con 0x7f7ff40af4d0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.366+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff4159860 con 0x7f7ff40af4d0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.366+0000 7f7fe9ffb700 1 -- 172.21.15.97:0/64483198 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7fe401c440 con 0x7f7ff40af4d0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.366+0000 7f7fe9ffb700 1 -- 172.21.15.97:0/64483198 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7fe4012ad0 con 0x7f7ff40af4d0 2023-11-24T06:08:35.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.366+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff4155620 con 0x7f7ff40af4d0 2023-11-24T06:08:35.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.369+0000 7f7fe9ffb700 1 -- 172.21.15.97:0/64483198 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f7fe4017070 con 0x7f7ff40af4d0 2023-11-24T06:08:35.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.438+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ff40753f0 con 0x7f7ff40af4d0 2023-11-24T06:08:35.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.439+0000 7f7fe9ffb700 1 -- 172.21.15.97:0/64483198 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f7fe4012400 con 0x7f7ff40af4d0 2023-11-24T06:08:35.442 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:08:35.442 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:35.171857+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:08:35.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40af4d0 msgr2=0x7f7ff4152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:35.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 --2- 172.21.15.97:0/64483198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40af4d0 0x7f7ff4152ef0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4005910 tx=0x7f7fe400c9e0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 shutdown_connections 2023-11-24T06:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 --2- 172.21.15.97:0/64483198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ff415e080 0x7f7ff4153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 --2- 172.21.15.97:0/64483198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ff406c710 0x7f7ff4159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 --2- 172.21.15.97:0/64483198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ff40af4d0 0x7f7ff4152ef0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 >> 172.21.15.97:0/64483198 conn(0x7f7ff4071b90 msgr2=0x7f7ff406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 shutdown_connections 2023-11-24T06:08:35.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:35.440+0000 7f7ffb2e5700 1 -- 172.21.15.97:0/64483198 wait complete. 2023-11-24T06:08:35.455 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T06:08:35.455 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T06:08:35.795 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:35.793+0000 7fca4b0b1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:08:35.807 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-11-24T06:08:35.805+0000 7fc9b4933640 -1 Falling back to public interface 2023-11-24T06:08:35.815 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-11-24T06:08:35.813+0000 7f68db029640 -1 Falling back to public interface 2023-11-24T06:08:35.818 INFO:tasks.ceph.osd.0.smithi097.stderr:2023-11-24T06:08:35.815+0000 7fa17185a640 -1 Falling back to public interface 2023-11-24T06:08:35.822 INFO:tasks.ceph.osd.1.smithi097.stderr:2023-11-24T06:08:35.819+0000 7f3cfe85c640 -1 Falling back to public interface 2023-11-24T06:08:35.827 INFO:tasks.ceph.osd.2.smithi097.stderr:2023-11-24T06:08:35.824+0000 7ff59cd9d640 -1 Falling back to public interface 2023-11-24T06:08:35.827 INFO:tasks.ceph.osd.3.smithi097.stderr:2023-11-24T06:08:35.824+0000 7f1f30c53640 -1 Falling back to public interface 2023-11-24T06:08:35.831 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-11-24T06:08:35.829+0000 7f5c3be79640 -1 Falling back to public interface 2023-11-24T06:08:35.832 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-11-24T06:08:35.829+0000 7fb84f955640 -1 Falling back to public interface 2023-11-24T06:08:35.878 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:35.875+0000 7fca4b0b1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:08:35.908 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:35.905+0000 7f828fef3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:08:35.984 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:35.982+0000 7f828fef3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:08:36.016 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:36.013+0000 7fca4b0b1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:08:36.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:36.123+0000 7f828fef3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:08:36.252 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:36.249+0000 7fca4b0b1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:08:36.260 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:08:36.360 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:36.358+0000 7f828fef3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:08:36.377 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.374+0000 7ffa91a31700 1 Processor -- start 2023-11-24T06:08:36.377 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.374+0000 7ffa91a31700 1 -- start start 2023-11-24T06:08:36.379 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa91a31700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ffa8c0aff30 0x7ffa8c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:36.379 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa91a31700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c0b0390 0x7ffa8c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:36.379 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa91a31700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8c15ec90 con 0x7ffa8c158910 2023-11-24T06:08:36.379 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa91a31700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c158cf0 con 0x7ffa8c0aff30 2023-11-24T06:08:36.379 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa91a31700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c15ef70 con 0x7ffa8c0b0390 2023-11-24T06:08:36.379 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffa8c158910 0x7ffa8c0b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:36866/0 (socket says 172.21.15.97:36866) 2023-11-24T06:08:36.379 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8affd700 1 -- 172.21.15.97:0/934875933 learned_addr learned my addr 172.21.15.97:0/934875933 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:36.379 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8a7fc700 1 --2- 172.21.15.97:0/934875933 >> v2:172.21.15.183:3300/0 conn(0x7ffa8c0aff30 0x7ffa8c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8b7fe700 1 --2- 172.21.15.97:0/934875933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c0b0390 0x7ffa8c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa89ffb700 1 -- 172.21.15.97:0/934875933 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951474920 0 0) 0x7ffa8c15ec90 con 0x7ffa8c158910 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa89ffb700 1 -- 172.21.15.97:0/934875933 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa740035b0 con 0x7ffa8c158910 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8b7fe700 1 -- 172.21.15.97:0/934875933 >> v2:172.21.15.183:3300/0 conn(0x7ffa8c0aff30 msgr2=0x7ffa8c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8b7fe700 1 --2- 172.21.15.97:0/934875933 >> v2:172.21.15.183:3300/0 conn(0x7ffa8c0aff30 0x7ffa8c06c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8b7fe700 1 -- 172.21.15.97:0/934875933 >> v1:172.21.15.97:6789/0 conn(0x7ffa8c158910 legacy=0x7ffa8c0b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8b7fe700 1 -- 172.21.15.97:0/934875933 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c15f250 con 0x7ffa8c0b0390 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.375+0000 7ffa8a7fc700 1 --2- 172.21.15.97:0/934875933 >> v2:172.21.15.183:3300/0 conn(0x7ffa8c0aff30 0x7ffa8c06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.376+0000 7ffa8b7fe700 1 --2- 172.21.15.97:0/934875933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c0b0390 0x7ffa8c06c540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009d90 tx=0x7ffa8400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb5e2b09d9530e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.376+0000 7ffa89ffb700 1 -- 172.21.15.97:0/934875933 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400e070 con 0x7ffa8c0b0390 2023-11-24T06:08:36.380 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.376+0000 7ffa89ffb700 1 -- 172.21.15.97:0/934875933 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8400bb20 con 0x7ffa8c0b0390 2023-11-24T06:08:36.381 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.376+0000 7ffa89ffb700 1 -- 172.21.15.97:0/934875933 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84012430 con 0x7ffa8c0b0390 2023-11-24T06:08:36.383 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.377+0000 7ffa91a31700 1 -- 172.21.15.97:0/934875933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c0b0390 msgr2=0x7ffa8c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:36.383 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.377+0000 7ffa91a31700 1 --2- 172.21.15.97:0/934875933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c0b0390 0x7ffa8c06c540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009d90 tx=0x7ffa8400b330 comp rx=0 tx=0).stop 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.377+0000 7ffa91a31700 1 -- 172.21.15.97:0/934875933 shutdown_connections 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.377+0000 7ffa91a31700 1 --2- 172.21.15.97:0/934875933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c0b0390 0x7ffa8c06c540 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.377+0000 7ffa91a31700 1 --2- 172.21.15.97:0/934875933 >> v2:172.21.15.183:3300/0 conn(0x7ffa8c0aff30 0x7ffa8c06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.377+0000 7ffa91a31700 1 -- 172.21.15.97:0/934875933 >> 172.21.15.97:0/934875933 conn(0x7ffa8c071ae0 msgr2=0x7ffa8c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.377+0000 7ffa91a31700 1 -- 172.21.15.97:0/934875933 shutdown_connections 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.377+0000 7ffa91a31700 1 -- 172.21.15.97:0/934875933 wait complete. 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.378+0000 7ffa91a31700 1 Processor -- start 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.378+0000 7ffa91a31700 1 -- start start 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa91a31700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa8c0aff30 0x7ffa8c1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa91a31700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa8c0b0390 0x7ffa8c1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa91a31700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c158910 0x7ffa8c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:36.384 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa91a31700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c1f4ac0 con 0x7ffa8c0aff30 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa91a31700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c159170 con 0x7ffa8c0b0390 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa91a31700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c15de00 con 0x7ffa8c158910 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa8c0aff30 0x7ffa8c1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c158910 0x7ffa8c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa8c0aff30 0x7ffa8c1533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44420/0 (socket says 172.21.15.97:44420) 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8a7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa8c0b0390 0x7ffa8c1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 -- 172.21.15.97:0/2462803693 learned_addr learned my addr 172.21.15.97:0/2462803693 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 -- 172.21.15.97:0/2462803693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c158910 msgr2=0x7ffa8c15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 --2- 172.21.15.97:0/2462803693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c158910 0x7ffa8c15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 -- 172.21.15.97:0/2462803693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa8c0b0390 msgr2=0x7ffa8c1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 --2- 172.21.15.97:0/2462803693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa8c0b0390 0x7ffa8c1f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.385 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 -- 172.21.15.97:0/2462803693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa800094a0 con 0x7ffa8c0aff30 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8b7fe700 1 --2- 172.21.15.97:0/2462803693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c158910 0x7ffa8c15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.379+0000 7ffa8affd700 1 --2- 172.21.15.97:0/2462803693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa8c0aff30 0x7ffa8c1533e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c001eb0 tx=0x7ffa7c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.380+0000 7ffa73fff700 1 -- 172.21.15.97:0/2462803693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c005d70 con 0x7ffa8c0aff30 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.380+0000 7ffa73fff700 1 -- 172.21.15.97:0/2462803693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c0070d0 con 0x7ffa8c0aff30 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.380+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa84009a70 con 0x7ffa8c0aff30 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.380+0000 7ffa73fff700 1 -- 172.21.15.97:0/2462803693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c007460 con 0x7ffa8c0aff30 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.380+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c15bc40 con 0x7ffa8c0aff30 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.380+0000 7ffa73fff700 1 -- 172.21.15.97:0/2462803693 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffa7c01c440 con 0x7ffa8c0aff30 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.380+0000 7ffa73fff700 1 -- 172.21.15.97:0/2462803693 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ffa7c012ad0 con 0x7ffa8c0aff30 2023-11-24T06:08:36.386 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.381+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c15baa0 con 0x7ffa8c0aff30 2023-11-24T06:08:36.387 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.383+0000 7ffa73fff700 1 -- 172.21.15.97:0/2462803693 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ffa8c15baa0 con 0x7ffa8c0aff30 2023-11-24T06:08:36.458 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.455+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa8c068fb0 con 0x7ffa8c0aff30 2023-11-24T06:08:36.458 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.455+0000 7ffa73fff700 1 -- 172.21.15.97:0/2462803693 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ffa7c017070 con 0x7ffa8c0aff30 2023-11-24T06:08:36.458 INFO:teuthology.misc.health.smithi097.stdout: 2023-11-24T06:08:36.459 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":10,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:35.171857+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.456+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa8c0aff30 msgr2=0x7ffa8c1533e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.456+0000 7ffa91a31700 1 --2- 172.21.15.97:0/2462803693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa8c0aff30 0x7ffa8c1533e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c001eb0 tx=0x7ffa7c00c7b0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.457+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 shutdown_connections 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.457+0000 7ffa91a31700 1 --2- 172.21.15.97:0/2462803693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa8c158910 0x7ffa8c15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.457+0000 7ffa91a31700 1 --2- 172.21.15.97:0/2462803693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa8c0aff30 0x7ffa8c1533e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.457+0000 7ffa91a31700 1 --2- 172.21.15.97:0/2462803693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa8c0b0390 0x7ffa8c1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.457+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 >> 172.21.15.97:0/2462803693 conn(0x7ffa8c071ae0 msgr2=0x7ffa8c06e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.457+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 shutdown_connections 2023-11-24T06:08:36.460 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:36.457+0000 7ffa91a31700 1 -- 172.21.15.97:0/2462803693 wait complete. 2023-11-24T06:08:36.470 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T06:08:36.473 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:36.471+0000 7fca4b0b1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:08:36.575 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:36.572+0000 7f828fef3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:08:36.606 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:36.603+0000 7fca4b0b1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:08:36.708 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:36.706+0000 7f828fef3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:08:36.824 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:36.822+0000 7fca4b0b1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:08:36.926 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:36.923+0000 7f828fef3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:08:36.979 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:36.976+0000 7fca4b0b1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:08:37.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:37.079+0000 7f828fef3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:08:37.093 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:37.090+0000 7fca4b0b1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:08:37.196 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:37.193+0000 7f828fef3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:08:37.426 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:37.423+0000 7fca4b0b1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:08:37.521 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:37.518+0000 7f828fef3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:08:37.525 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:37.523+0000 7fca4b0b1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:08:37.620 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:37.617+0000 7f828fef3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:08:37.710 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:37.708+0000 7fca4b0b1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:08:37.807 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:37.804+0000 7f828fef3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:08:38.304 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:38.301+0000 7fca4b0b1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:08:38.416 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-11-24T06:08:38.413+0000 7f828fef3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:08:39.327 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:39.325+0000 7fca0303f700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T06:08:39.328 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:08:39.325+0000 7fca0303f700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T06:08:39.362 INFO:tasks.ceph.osd.1.smithi097.stderr:2023-11-24T06:08:39.359+0000 7f3cfe85c640 -1 osd.1 0 log_to_monitors true 2023-11-24T06:08:39.368 INFO:tasks.ceph.osd.3.smithi097.stderr:2023-11-24T06:08:39.365+0000 7f1f30c53640 -1 osd.3 0 log_to_monitors true 2023-11-24T06:08:39.376 INFO:tasks.ceph.osd.2.smithi097.stderr:2023-11-24T06:08:39.373+0000 7ff59cd9d640 -1 osd.2 0 log_to_monitors true 2023-11-24T06:08:39.378 INFO:tasks.ceph.osd.0.smithi097.stderr:2023-11-24T06:08:39.376+0000 7fa17185a640 -1 osd.0 0 log_to_monitors true 2023-11-24T06:08:39.404 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-11-24T06:08:39.401+0000 7fc9b4933640 -1 osd.4 0 log_to_monitors true 2023-11-24T06:08:39.404 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-11-24T06:08:39.401+0000 7fb84f955640 -1 osd.6 0 log_to_monitors true 2023-11-24T06:08:39.415 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-11-24T06:08:39.411+0000 7f68db029640 -1 osd.7 0 log_to_monitors true 2023-11-24T06:08:39.424 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-11-24T06:08:39.421+0000 7f5c3be79640 -1 osd.5 0 log_to_monitors true 2023-11-24T06:08:43.280 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:08:43.301 INFO:tasks.ceph.osd.2.smithi097.stderr:2023-11-24T06:08:43.299+0000 7ff590954700 -1 osd.2 0 waiting for initial osdmap 2023-11-24T06:08:43.315 INFO:tasks.ceph.osd.2.smithi097.stderr:2023-11-24T06:08:43.313+0000 7ff588d34700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:08:43.412 INFO:tasks.ceph.osd.1.smithi097.stderr:2023-11-24T06:08:43.409+0000 7f3cf2413700 -1 osd.1 0 waiting for initial osdmap 2023-11-24T06:08:43.413 INFO:tasks.ceph.osd.3.smithi097.stderr:2023-11-24T06:08:43.410+0000 7f1f2480a700 -1 osd.3 0 waiting for initial osdmap 2023-11-24T06:08:43.422 INFO:tasks.ceph.osd.0.smithi097.stderr:2023-11-24T06:08:43.419+0000 7fa165411700 -1 osd.0 0 waiting for initial osdmap 2023-11-24T06:08:43.427 INFO:tasks.ceph.osd.1.smithi097.stderr:2023-11-24T06:08:43.425+0000 7f3cea7f3700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:08:43.429 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-11-24T06:08:43.425+0000 7fc9a84ea700 -1 osd.4 0 waiting for initial osdmap 2023-11-24T06:08:43.431 INFO:tasks.ceph.osd.3.smithi097.stderr:2023-11-24T06:08:43.429+0000 7f1f1cbea700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:08:43.434 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.432+0000 7f8fcbe68700 1 Processor -- start 2023-11-24T06:08:43.434 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.432+0000 7f8fcbe68700 1 -- start start 2023-11-24T06:08:43.435 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fcbe68700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8fc40d02c0 0x7f8fc40d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:43.435 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fcbe68700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc40d0c40 con 0x7f8fc415bf60 2023-11-24T06:08:43.436 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fcbe68700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc406f220 con 0x7f8fc40d0740 2023-11-24T06:08:43.436 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fcbe68700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc415c340 con 0x7f8fc40d02c0 2023-11-24T06:08:43.436 INFO:tasks.ceph.osd.0.smithi097.stderr:2023-11-24T06:08:43.433+0000 7fa15d7f1700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:08:43.437 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fca405700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f8fc40d0740 0x7f8fc4070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46300/0 (socket says 172.21.15.97:46300) 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fca405700 1 -- 172.21.15.97:0/4164537950 learned_addr learned my addr 172.21.15.97:0/4164537950 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc9403700 1 --2- 172.21.15.97:0/4164537950 >> v2:172.21.15.97:3300/0 conn(0x7f8fc40d02c0 0x7f8fc40d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc9403700 1 -- 172.21.15.97:0/4164537950 >> v1:172.21.15.183:6789/0 conn(0x7f8fc40d0740 legacy=0x7f8fc4070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc9403700 1 -- 172.21.15.97:0/4164537950 >> v1:172.21.15.97:6789/0 conn(0x7f8fc415bf60 legacy=0x7f8fc40cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc8c02700 1 -- 172.21.15.97:0/4164537950 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357426372 0 0) 0x7f8fc406f220 con 0x7f8fc40d0740 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc9403700 1 -- 172.21.15.97:0/4164537950 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40cce20 con 0x7f8fc40d02c0 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc9403700 1 --2- 172.21.15.97:0/4164537950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d02c0 0x7f8fc40d1b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009840 tx=0x7f8fb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ff8862caaadb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc8c02700 1 -- 172.21.15.97:0/4164537950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8004230 con 0x7f8fc40d02c0 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc8c02700 1 -- 172.21.15.97:0/4164537950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb8004390 con 0x7f8fc40d02c0 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.433+0000 7f8fc8c02700 1 -- 172.21.15.97:0/4164537950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80036a0 con 0x7f8fc40d02c0 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.434+0000 7f8fcbe68700 1 -- 172.21.15.97:0/4164537950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d02c0 msgr2=0x7f8fc40d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:43.438 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.434+0000 7f8fcbe68700 1 --2- 172.21.15.97:0/4164537950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d02c0 0x7f8fc40d1b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009840 tx=0x7f8fb8009b50 comp rx=0 tx=0).stop 2023-11-24T06:08:43.439 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.434+0000 7f8fcbe68700 1 -- 172.21.15.97:0/4164537950 shutdown_connections 2023-11-24T06:08:43.439 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.434+0000 7f8fcbe68700 1 --2- 172.21.15.97:0/4164537950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d02c0 0x7f8fc40d1b50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:43.439 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.434+0000 7f8fcbe68700 1 -- 172.21.15.97:0/4164537950 >> 172.21.15.97:0/4164537950 conn(0x7f8fc409aa00 msgr2=0x7f8fc409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:43.439 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.434+0000 7f8fcbe68700 1 -- 172.21.15.97:0/4164537950 shutdown_connections 2023-11-24T06:08:43.439 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.434+0000 7f8fcbe68700 1 -- 172.21.15.97:0/4164537950 wait complete. 2023-11-24T06:08:43.441 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.435+0000 7f8fcbe68700 1 Processor -- start 2023-11-24T06:08:43.441 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.435+0000 7f8fcbe68700 1 -- start start 2023-11-24T06:08:43.441 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fcbe68700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8fc40d02c0 0x7f8fc4157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:43.441 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fcbe68700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d0740 0x7f8fc40c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:43.441 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fcbe68700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fc415bf60 0x7f8fc4159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fcbe68700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c8db0 con 0x7f8fc40d0740 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fcbe68700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c9000 con 0x7f8fc415bf60 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fcbe68700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c92e0 con 0x7f8fc40d02c0 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fc9403700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d0740 0x7f8fc40c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fc9403700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d0740 0x7f8fc40c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44530/0 (socket says 172.21.15.97:44530) 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fc9403700 1 -- 172.21.15.97:0/888001728 learned_addr learned my addr 172.21.15.97:0/888001728 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fc9c04700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8fc40d02c0 0x7f8fc4157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fca405700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fc415bf60 0x7f8fc4159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fc9403700 1 -- 172.21.15.97:0/888001728 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8fc40d02c0 msgr2=0x7f8fc4157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fc9403700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8fc40d02c0 0x7f8fc4157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.436+0000 7f8fc9403700 1 -- 172.21.15.97:0/888001728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fc415bf60 msgr2=0x7f8fc4159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:43.442 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.437+0000 7f8fc9403700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fc415bf60 0x7f8fc4159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.437+0000 7f8fc9403700 1 -- 172.21.15.97:0/888001728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb80094a0 con 0x7f8fc40d0740 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.437+0000 7f8fc9403700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d0740 0x7f8fc40c7f40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80090b0 tx=0x7f8fb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.437+0000 7f8fc27fc700 1 -- 172.21.15.97:0/888001728 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8011630 con 0x7f8fc40d0740 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.437+0000 7f8fc27fc700 1 -- 172.21.15.97:0/888001728 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb801f070 con 0x7f8fc40d0740 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.437+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc40c95c0 con 0x7f8fc40d0740 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.437+0000 7f8fc27fc700 1 -- 172.21.15.97:0/888001728 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80119d0 con 0x7f8fc40d0740 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.437+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc41576a0 con 0x7f8fc40d0740 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.438+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40d0130 con 0x7f8fc40d0740 2023-11-24T06:08:43.443 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.438+0000 7f8fc27fc700 1 -- 172.21.15.97:0/888001728 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f8fb8011b30 con 0x7f8fc40d0740 2023-11-24T06:08:43.444 INFO:tasks.ceph.osd.4.smithi183.stderr:2023-11-24T06:08:43.439+0000 7fc9a08ca700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:08:43.444 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.439+0000 7f8fc27fc700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8fb006ae40 0x7f8fb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:43.444 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.439+0000 7f8fc9c04700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8fb006ae40 0x7f8fb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:43.444 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.439+0000 7f8fc27fc700 1 -- 172.21.15.97:0/888001728 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f8fb8021070 con 0x7f8fc40d0740 2023-11-24T06:08:43.444 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.439+0000 7f8fc9c04700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8fb006ae40 0x7f8fb006d300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4001f80 tx=0x7f8fb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:43.446 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.443+0000 7f8fc27fc700 1 -- 172.21.15.97:0/888001728 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8fb80bce90 con 0x7f8fc40d0740 2023-11-24T06:08:43.446 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-11-24T06:08:43.443+0000 7f5c2fa30700 -1 osd.5 0 waiting for initial osdmap 2023-11-24T06:08:43.453 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-11-24T06:08:43.450+0000 7fb84350c700 -1 osd.6 0 waiting for initial osdmap 2023-11-24T06:08:43.459 INFO:tasks.ceph.osd.5.smithi183.stderr:2023-11-24T06:08:43.456+0000 7f5c27e10700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:08:43.461 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-11-24T06:08:43.458+0000 7f68cebe0700 -1 osd.7 0 waiting for initial osdmap 2023-11-24T06:08:43.466 INFO:tasks.ceph.osd.6.smithi183.stderr:2023-11-24T06:08:43.463+0000 7fb83b8ec700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:08:43.478 INFO:tasks.ceph.osd.7.smithi183.stderr:2023-11-24T06:08:43.475+0000 7f68c6fc0700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:08:43.645 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.642+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fc4068fb0 con 0x7f8fc40d0740 2023-11-24T06:08:43.646 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.642+0000 7f8fc27fc700 1 -- 172.21.15.97:0/888001728 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8fb802b020 con 0x7f8fc40d0740 2023-11-24T06:08:43.646 INFO:teuthology.misc.health.smithi097.stdout: 2023-11-24T06:08:43.646 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":12,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:41.325020+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:08:43.648 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.644+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8fb006ae40 msgr2=0x7f8fb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:43.648 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.644+0000 7f8fcbe68700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8fb006ae40 0x7f8fb006d300 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4001f80 tx=0x7f8fb4009040 comp rx=0 tx=0).stop 2023-11-24T06:08:43.648 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.644+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d0740 msgr2=0x7f8fc40c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:43.648 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.644+0000 7f8fcbe68700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d0740 0x7f8fc40c7f40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80090b0 tx=0x7f8fb8004750 comp rx=0 tx=0).stop 2023-11-24T06:08:43.648 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.644+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 shutdown_connections 2023-11-24T06:08:43.649 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.644+0000 7f8fcbe68700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fc415bf60 0x7f8fc4159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:43.649 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.644+0000 7f8fcbe68700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8fb006ae40 0x7f8fb006d300 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:43.649 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.645+0000 7f8fcbe68700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8fc40d02c0 0x7f8fc4157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:43.649 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.645+0000 7f8fcbe68700 1 --2- 172.21.15.97:0/888001728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc40d0740 0x7f8fc40c7f40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:43.649 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.645+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 >> 172.21.15.97:0/888001728 conn(0x7f8fc409aa00 msgr2=0x7f8fc40b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:43.649 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.645+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 shutdown_connections 2023-11-24T06:08:43.649 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:43.645+0000 7f8fcbe68700 1 -- 172.21.15.97:0/888001728 wait complete. 2023-11-24T06:08:43.659 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T06:08:50.465 DEBUG:teuthology.orchestra.run.smithi097:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:08:50.588 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.585+0000 7fa97dcdc700 1 Processor -- start 2023-11-24T06:08:50.588 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.585+0000 7fa97dcdc700 1 -- start start 2023-11-24T06:08:50.591 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa97dcdc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa97815e150 0x7fa9780d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.591 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa97dcdc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9780ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.591 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa97dcdc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa9780d3320 0x7fa9780d3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.591 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa97dcdc700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9780d3cd0 con 0x7fa97815e150 2023-11-24T06:08:50.591 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa97dcdc700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97815e530 con 0x7fa9780d3320 2023-11-24T06:08:50.591 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa97dcdc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780cf170 con 0x7fa9780ce6b0 2023-11-24T06:08:50.591 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa9777fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa97815e150 0x7fa9780d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa9777fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa97815e150 0x7fa9780d2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44750/0 (socket says 172.21.15.97:44750) 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.586+0000 7fa9777fe700 1 -- 172.21.15.97:0/611888240 learned_addr learned my addr 172.21.15.97:0/611888240 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa977fff700 1 --2- 172.21.15.97:0/611888240 >> v2:172.21.15.183:3300/0 conn(0x7fa9780d3320 0x7fa9780d3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa976ffd700 1 --2- 172.21.15.97:0/611888240 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9780ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9777fe700 1 -- 172.21.15.97:0/611888240 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 msgr2=0x7fa9780ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9777fe700 1 --2- 172.21.15.97:0/611888240 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9780ceb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9777fe700 1 -- 172.21.15.97:0/611888240 >> v2:172.21.15.183:3300/0 conn(0x7fa9780d3320 msgr2=0x7fa9780d3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9777fe700 1 --2- 172.21.15.97:0/611888240 >> v2:172.21.15.183:3300/0 conn(0x7fa9780d3320 0x7fa9780d3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9777fe700 1 -- 172.21.15.97:0/611888240 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780d2030 con 0x7fa97815e150 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa977fff700 1 --2- 172.21.15.97:0/611888240 >> v2:172.21.15.183:3300/0 conn(0x7fa9780d3320 0x7fa9780d3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa976ffd700 1 --2- 172.21.15.97:0/611888240 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9780ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:50.592 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9777fe700 1 --2- 172.21.15.97:0/611888240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa97815e150 0x7fa9780d2b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa97808cb20 tx=0x7fa968009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a6b2ad71a02c80 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9767fc700 1 -- 172.21.15.97:0/611888240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96800b870 con 0x7fa97815e150 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9767fc700 1 -- 172.21.15.97:0/611888240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96800b9d0 con 0x7fa97815e150 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa9767fc700 1 -- 172.21.15.97:0/611888240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96800bd60 con 0x7fa97815e150 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa97dcdc700 1 -- 172.21.15.97:0/611888240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa97815e150 msgr2=0x7fa9780d2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.587+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/611888240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa97815e150 0x7fa9780d2b10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa97808cb20 tx=0x7fa968009bd0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.588+0000 7fa97dcdc700 1 -- 172.21.15.97:0/611888240 shutdown_connections 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.588+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/611888240 >> v2:172.21.15.183:3300/0 conn(0x7fa9780d3320 0x7fa9780d3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.588+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/611888240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa97815e150 0x7fa9780d2b10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.588+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/611888240 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9780ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.588+0000 7fa97dcdc700 1 -- 172.21.15.97:0/611888240 >> 172.21.15.97:0/611888240 conn(0x7fa9780d08d0 msgr2=0x7fa9780d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.588+0000 7fa97dcdc700 1 -- 172.21.15.97:0/611888240 shutdown_connections 2023-11-24T06:08:50.593 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.588+0000 7fa97dcdc700 1 -- 172.21.15.97:0/611888240 wait complete. 2023-11-24T06:08:50.596 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.589+0000 7fa97dcdc700 1 Processor -- start 2023-11-24T06:08:50.596 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.589+0000 7fa97dcdc700 1 -- start start 2023-11-24T06:08:50.596 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa97dcdc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9781f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.596 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa97dcdc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9780d3320 0x7fa9780c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.596 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa97dcdc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97815e150 0x7fa9780b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa97dcdc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780caf20 con 0x7fa9780d3320 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa97dcdc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780cb1e0 con 0x7fa97815e150 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa97dcdc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780cb4c0 con 0x7fa9780ce6b0 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9780d3320 0x7fa9780c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9780d3320 0x7fa9780c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44758/0 (socket says 172.21.15.97:44758) 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 -- 172.21.15.97:0/2574660861 learned_addr learned my addr 172.21.15.97:0/2574660861 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa977fff700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97815e150 0x7fa9780b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa9777fe700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9781f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 -- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 msgr2=0x7fa9781f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9781f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 -- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97815e150 msgr2=0x7fa9780b82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97815e150 0x7fa9780b82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.597 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 -- 172.21.15.97:0/2574660861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa968009550 con 0x7fa9780d3320 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa977fff700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97815e150 0x7fa9780b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa9777fe700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9781f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa976ffd700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9780d3320 0x7fa9780c8130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa96000c750 tx=0x7fa96000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa96ffff700 1 -- 172.21.15.97:0/2574660861 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960011800 con 0x7fa9780d3320 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.590+0000 7fa96ffff700 1 -- 172.21.15.97:0/2574660861 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa960011960 con 0x7fa9780d3320 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.591+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9780cb7a0 con 0x7fa9780d3320 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.591+0000 7fa96ffff700 1 -- 172.21.15.97:0/2574660861 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960011cd0 con 0x7fa9780d3320 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.591+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9781f5000 con 0x7fa9780d3320 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.591+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9780cbcc0 con 0x7fa9780d3320 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.592+0000 7fa96ffff700 1 -- 172.21.15.97:0/2574660861 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa960021350 con 0x7fa9780d3320 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.592+0000 7fa96ffff700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa95806aed0 0x7fa95806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.598 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.592+0000 7fa96ffff700 1 -- 172.21.15.97:0/2574660861 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa9600892e0 con 0x7fa9780d3320 2023-11-24T06:08:50.599 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.592+0000 7fa9777fe700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa95806aed0 0x7fa95806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.599 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.593+0000 7fa9777fe700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa95806aed0 0x7fa95806d390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa968014040 tx=0x7fa968008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:50.600 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.597+0000 7fa96ffff700 1 -- 172.21.15.97:0/2574660861 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9780cbcc0 con 0x7fa9780d3320 2023-11-24T06:08:50.791 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.788+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9780cbcc0 con 0x7fa9780d3320 2023-11-24T06:08:50.791 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.789+0000 7fa96ffff700 1 -- 172.21.15.97:0/2574660861 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12862 (secure 0 0 0) 0x7fa9780cbcc0 con 0x7fa9780d3320 2023-11-24T06:08:50.792 INFO:teuthology.misc.health.smithi097.stdout: 2023-11-24T06:08:50.792 INFO:teuthology.misc.health.smithi097.stdout:{"epoch":16,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:47.309680+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:08:50.794 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa95806aed0 msgr2=0x7fa95806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.794 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa95806aed0 0x7fa95806d390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa968014040 tx=0x7fa968008030 comp rx=0 tx=0).stop 2023-11-24T06:08:50.794 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9780d3320 msgr2=0x7fa9780c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9780d3320 0x7fa9780c8130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa96000c750 tx=0x7fa96000cb10 comp rx=0 tx=0).stop 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 shutdown_connections 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa97815e150 0x7fa9780b82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa95806aed0 0x7fa95806d390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa9780ce6b0 0x7fa9781f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 --2- 172.21.15.97:0/2574660861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9780d3320 0x7fa9780c8130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 >> 172.21.15.97:0/2574660861 conn(0x7fa9780d08d0 msgr2=0x7fa9780b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 shutdown_connections 2023-11-24T06:08:50.795 INFO:teuthology.misc.health.smithi097.stderr:2023-11-24T06:08:50.791+0000 7fa97dcdc700 1 -- 172.21.15.97:0/2574660861 wait complete. 2023-11-24T06:08:50.805 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T06:08:50.805 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T06:08:50.806 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:08:50.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:08:50.808 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:08:50.808 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:08:50.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:08:50.810 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:08:50.810 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:08:50.810 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:08:50.813 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:08:50.813 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:08:50.813 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:08:50.815 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:08:50.815 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T06:08:50.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:08:50.886 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-11-24T06:08:50.887 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-11-24T06:08:50.903 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-11-24T06:08:50.903 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-11-24T06:08:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.940+0000 7f14f0a6a700 1 Processor -- start 2023-11-24T06:08:50.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.940+0000 7f14f0a6a700 1 -- start start 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.941+0000 7f14f0a6a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f14ec15e080 0x7f14ec0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.941+0000 7f14f0a6a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14ec06c710 0x7f14ec0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.941+0000 7f14f0a6a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec0ae430 con 0x7f14ec0af4d0 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.941+0000 7f14f0a6a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0af0b0 con 0x7f14ec15e080 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.941+0000 7f14f0a6a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0b05a0 con 0x7f14ec06c710 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e9d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f14ec0af4d0 0x7f14ec06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37244/0 (socket says 172.21.15.97:37244) 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e9d9b700 1 -- 172.21.15.97:0/3225555357 learned_addr learned my addr 172.21.15.97:0/3225555357 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138540822 0 0) 0x7f14ec0ae430 con 0x7f14ec0af4d0 2023-11-24T06:08:50.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ead9d700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14ec06c710 0x7f14ec0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d40035b0 con 0x7f14ec0af4d0 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ea59c700 1 --2- 172.21.15.97:0/3225555357 >> v2:172.21.15.183:3300/0 conn(0x7f14ec15e080 0x7f14ec0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2703175536 0 0) 0x7f14d40035b0 con 0x7f14ec0af4d0 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14ec06c710 msgr2=0x7f14ec0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14ec06c710 0x7f14ec0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 >> v2:172.21.15.183:3300/0 conn(0x7f14ec15e080 msgr2=0x7f14ec0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 --2- 172.21.15.97:0/3225555357 >> v2:172.21.15.183:3300/0 conn(0x7f14ec15e080 0x7f14ec0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0ae710 con 0x7f14ec0af4d0 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f14dc001670 con 0x7f14ec0af4d0 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 >> v1:172.21.15.97:6789/0 conn(0x7f14ec0af4d0 legacy=0x7f14ec06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14d4003a30 0x7f14d4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14ec06c710 0x7f14d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4004d90 0x7f14d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0af0b0 con 0x7f14d4003a30 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0b05a0 con 0x7f14d4004d90 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d400bb10 con 0x7f14ec06c710 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ea59c700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14d4003a30 0x7f14d4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14e9d9b700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14ec06c710 0x7f14d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ead9d700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4004d90 0x7f14d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ea59c700 1 -- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14ec06c710 msgr2=0x7f14d4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ea59c700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14ec06c710 0x7f14d4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ea59c700 1 -- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4004d90 msgr2=0x7f14d4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ea59c700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4004d90 0x7f14d4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ea59c700 1 -- 172.21.15.97:0/3225555357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40035b0 con 0x7f14d4003a30 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.942+0000 7f14ead9d700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14d4004d90 0x7f14d4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:50.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14ea59c700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14d4003a30 0x7f14d4003e50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009890 tx=0x7f14d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78855d8a156c6bc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e000c070 con 0x7f14d4003a30 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e0013610 con 0x7f14d4003a30 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14e959a700 1 -- 172.21.15.97:0/3225555357 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e00139a0 con 0x7f14d4003a30 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3225555357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14d4003a30 msgr2=0x7f14d4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14f0a6a700 1 --2- 172.21.15.97:0/3225555357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14d4003a30 0x7f14d4003e50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009890 tx=0x7f14d4005880 comp rx=0 tx=0).stop 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14ead9d700 1 -- 172.21.15.97:0/3225555357 reap_dead start 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3225555357 shutdown_connections 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3225555357 >> 172.21.15.97:0/3225555357 conn(0x7f14ec071b90 msgr2=0x7f14ec071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3225555357 shutdown_connections 2023-11-24T06:08:50.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.943+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3225555357 wait complete. 2023-11-24T06:08:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14f0a6a700 1 Processor -- start 2023-11-24T06:08:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14f0a6a700 1 -- start start 2023-11-24T06:08:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14f0a6a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14d4003a30 0x7f14ec147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14f0a6a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14ec15aca0 0x7f14ec1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14f0a6a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14ec156000 0x7f14ec156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14f0a6a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec1569c0 con 0x7f14ec156000 2023-11-24T06:08:50.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14f0a6a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec147a80 con 0x7f14ec15aca0 2023-11-24T06:08:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14f0a6a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec1583e0 con 0x7f14d4003a30 2023-11-24T06:08:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14ead9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14ec156000 0x7f14ec156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14ead9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14ec156000 0x7f14ec156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44806/0 (socket says 172.21.15.97:44806) 2023-11-24T06:08:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.945+0000 7f14ead9d700 1 -- 172.21.15.97:0/3734598295 learned_addr learned my addr 172.21.15.97:0/3734598295 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14e9d9b700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14ec15aca0 0x7f14ec1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14ead9d700 1 -- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14d4003a30 msgr2=0x7f14ec147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14ea59c700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14d4003a30 0x7f14ec147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14ead9d700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14d4003a30 0x7f14ec147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14ead9d700 1 -- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14ec15aca0 msgr2=0x7f14ec1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14ead9d700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14ec15aca0 0x7f14ec1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14ead9d700 1 -- 172.21.15.97:0/3734598295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec155cd0 con 0x7f14ec156000 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14e9d9b700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14ec15aca0 0x7f14ec1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14ead9d700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14ec156000 0x7f14ec156480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14e40045e0 tx=0x7f14e40049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14daffd700 1 -- 172.21.15.97:0/3734598295 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4007b50 con 0x7f14ec156000 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14daffd700 1 -- 172.21.15.97:0/3734598295 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e4007cd0 con 0x7f14ec156000 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec1559f0 con 0x7f14ec156000 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14daffd700 1 -- 172.21.15.97:0/3734598295 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4014640 con 0x7f14ec156000 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.946+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec158070 con 0x7f14ec156000 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.947+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec0753f0 con 0x7f14ec156000 2023-11-24T06:08:50.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.947+0000 7f14daffd700 1 -- 172.21.15.97:0/3734598295 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14e40133a0 con 0x7f14ec156000 2023-11-24T06:08:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.948+0000 7f14daffd700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f14cc06aed0 0x7f14cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.948+0000 7f14daffd700 1 -- 172.21.15.97:0/3734598295 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f14e4088d00 con 0x7f14ec156000 2023-11-24T06:08:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.948+0000 7f14ea59c700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f14cc06aed0 0x7f14cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.948+0000 7f14ea59c700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f14cc06aed0 0x7f14cc06d390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0b1c60 tx=0x7f14e000c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:50.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:50.952+0000 7f14daffd700 1 -- 172.21.15.97:0/3734598295 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14e4054190 con 0x7f14ec156000 2023-11-24T06:08:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.143+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14ec067f60 con 0x7f14ec156000 2023-11-24T06:08:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.144+0000 7f14daffd700 1 -- 172.21.15.97:0/3734598295 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12862 (secure 0 0 0) 0x7f14e40577b0 con 0x7f14ec156000 2023-11-24T06:08:51.147 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:08:51.147 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:47.309680+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.145+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f14cc06aed0 msgr2=0x7f14cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f14cc06aed0 0x7f14cc06d390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0b1c60 tx=0x7f14e000c000 comp rx=0 tx=0).stop 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14ec156000 msgr2=0x7f14ec156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14ec156000 0x7f14ec156480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f14e40045e0 tx=0x7f14e40049a0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 shutdown_connections 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14ec156000 0x7f14ec156480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f14cc06aed0 0x7f14cc06d390 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14d4003a30 0x7f14ec147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 --2- 172.21.15.97:0/3734598295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14ec15aca0 0x7f14ec1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 >> 172.21.15.97:0/3734598295 conn(0x7f14ec071b90 msgr2=0x7f14ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:51.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 shutdown_connections 2023-11-24T06:08:51.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.146+0000 7f14f0a6a700 1 -- 172.21.15.97:0/3734598295 wait complete. 2023-11-24T06:08:51.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:08:51.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:08:51.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.281+0000 7f6f02527700 1 Processor -- start 2023-11-24T06:08:51.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.281+0000 7f6f02527700 1 -- start start 2023-11-24T06:08:51.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6f02527700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 0x7f6efc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6f02527700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6efc0746a0 0x7f6efc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6f02527700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc0b3c60 con 0x7f6efc0afc90 2023-11-24T06:08:51.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6f02527700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc0b4500 con 0x7f6efc0746a0 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6f02527700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0b58c0 con 0x7f6efc15e160 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6f00ac4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6efc0746a0 0x7f6efc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6efb7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6efc0afc90 0x7f6efc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37262/0 (socket says 172.21.15.97:37262) 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6f00ac4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6efc0746a0 0x7f6efc06d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44812/0 (socket says 172.21.15.97:44812) 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.282+0000 7f6efb7fe700 1 -- 172.21.15.97:0/3524602144 learned_addr learned my addr 172.21.15.97:0/3524602144 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6efbfff700 1 --2- 172.21.15.97:0/3524602144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 0x7f6efc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f00ac4700 1 -- 172.21.15.97:0/3524602144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 msgr2=0x7f6efc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f00ac4700 1 --2- 172.21.15.97:0/3524602144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 0x7f6efc0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f00ac4700 1 -- 172.21.15.97:0/3524602144 >> v1:172.21.15.97:6789/0 conn(0x7f6efc0afc90 legacy=0x7f6efc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6efaffd700 1 -- 172.21.15.97:0/3524602144 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080744183 0 0) 0x7f6efc0b3c60 con 0x7f6efc0afc90 2023-11-24T06:08:51.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f00ac4700 1 -- 172.21.15.97:0/3524602144 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc06bdc0 con 0x7f6efc0746a0 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6efbfff700 1 --2- 172.21.15.97:0/3524602144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 0x7f6efc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f00ac4700 1 --2- 172.21.15.97:0/3524602144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc06d930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d90 tx=0x7f6ef400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdc5fc81ab9c964 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6efaffd700 1 -- 172.21.15.97:0/3524602144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400e070 con 0x7f6efc0746a0 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6efaffd700 1 -- 172.21.15.97:0/3524602144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400bbd0 con 0x7f6efc0746a0 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6efaffd700 1 -- 172.21.15.97:0/3524602144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4012430 con 0x7f6efc0746a0 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f02527700 1 -- 172.21.15.97:0/3524602144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 msgr2=0x7f6efc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f02527700 1 --2- 172.21.15.97:0/3524602144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc06d930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d90 tx=0x7f6ef400b330 comp rx=0 tx=0).stop 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f02527700 1 -- 172.21.15.97:0/3524602144 shutdown_connections 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.283+0000 7f6f02527700 1 --2- 172.21.15.97:0/3524602144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc06d930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.284+0000 7f6f02527700 1 --2- 172.21.15.97:0/3524602144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 0x7f6efc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.284+0000 7f6f02527700 1 -- 172.21.15.97:0/3524602144 >> 172.21.15.97:0/3524602144 conn(0x7f6efc0d0bf0 msgr2=0x7f6efc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:51.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.284+0000 7f6f02527700 1 -- 172.21.15.97:0/3524602144 shutdown_connections 2023-11-24T06:08:51.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.284+0000 7f6f02527700 1 -- 172.21.15.97:0/3524602144 wait complete. 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.285+0000 7f6f02527700 1 Processor -- start 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.285+0000 7f6f02527700 1 -- start start 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6f02527700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6f02527700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6efc0afc90 0x7f6efc1f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6f02527700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 0x7f6efc1f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6f02527700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c69a0 con 0x7f6efc0746a0 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6f02527700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c6c80 con 0x7f6efc0afc90 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6f02527700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c6f60 con 0x7f6efc15e160 2023-11-24T06:08:51.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc1550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44814/0 (socket says 172.21.15.97:44814) 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 -- 172.21.15.97:0/772254303 learned_addr learned my addr 172.21.15.97:0/772254303 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 -- 172.21.15.97:0/772254303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 msgr2=0x7f6efc1f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 0x7f6efc1f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 -- 172.21.15.97:0/772254303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6efc0afc90 msgr2=0x7f6efc1f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6efc0afc90 0x7f6efc1f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 -- 172.21.15.97:0/772254303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4009a70 con 0x7f6efc0746a0 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6efbfff700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc1550d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400c7f0 tx=0x7f6ee400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6ef8ff9700 1 -- 172.21.15.97:0/772254303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4011800 con 0x7f6efc0746a0 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6ef8ff9700 1 -- 172.21.15.97:0/772254303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee4011960 con 0x7f6efc0746a0 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6ef8ff9700 1 -- 172.21.15.97:0/772254303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4011d10 con 0x7f6efc0746a0 2023-11-24T06:08:51.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc0c7240 con 0x7f6efc0746a0 2023-11-24T06:08:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.286+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc158c90 con 0x7f6efc0746a0 2023-11-24T06:08:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.287+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc0c7940 con 0x7f6efc0746a0 2023-11-24T06:08:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.288+0000 7f6ef8ff9700 1 -- 172.21.15.97:0/772254303 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ee4018360 con 0x7f6efc0746a0 2023-11-24T06:08:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.288+0000 7f6ef8ff9700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6edc06aed0 0x7f6edc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.288+0000 7f6ef8ff9700 1 -- 172.21.15.97:0/772254303 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6ee40892a0 con 0x7f6efc0746a0 2023-11-24T06:08:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.288+0000 7f6efb7fe700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6edc06aed0 0x7f6edc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.288+0000 7f6efb7fe700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6edc06aed0 0x7f6edc06d390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0011e0 tx=0x7f6eec0079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:51.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.292+0000 7f6ef8ff9700 1 -- 172.21.15.97:0/772254303 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ee40546a0 con 0x7f6efc0746a0 2023-11-24T06:08:51.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.483+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6efc0c73d0 con 0x7f6efc0746a0 2023-11-24T06:08:51.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.483+0000 7f6ef8ff9700 1 -- 172.21.15.97:0/772254303 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ee4057cc0 con 0x7f6efc0746a0 2023-11-24T06:08:51.487 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:08:51.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6edc06aed0 msgr2=0x7f6edc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6edc06aed0 0x7f6edc06d390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0011e0 tx=0x7f6eec0079d0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 msgr2=0x7f6efc1550d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc1550d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400c7f0 tx=0x7f6ee400cbb0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 shutdown_connections 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6edc06aed0 0x7f6edc06d390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6efc15e160 0x7f6efc1f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6efc0746a0 0x7f6efc1550d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 --2- 172.21.15.97:0/772254303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6efc0afc90 0x7f6efc1f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 >> 172.21.15.97:0/772254303 conn(0x7f6efc0d0bf0 msgr2=0x7f6efc15d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 shutdown_connections 2023-11-24T06:08:51.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.485+0000 7f6f02527700 1 -- 172.21.15.97:0/772254303 wait complete. 2023-11-24T06:08:51.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:08:51.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.617+0000 7f770acac700 1 Processor -- start 2023-11-24T06:08:51.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.617+0000 7f770acac700 1 -- start start 2023-11-24T06:08:51.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f770acac700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f770415e080 0x7f77040b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f770acac700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77040af4d0 0x7f77040afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f770acac700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77040ae430 con 0x7f770406c000 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f770acac700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77040af0b0 con 0x7f77040af4d0 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f770acac700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77040b05a0 con 0x7f770415e080 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f7703fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77040af4d0 0x7f77040afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f7709249700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f770406c000 0x7f7704073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37278/0 (socket says 172.21.15.97:37278) 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f7703fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77040af4d0 0x7f77040afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44824/0 (socket says 172.21.15.97:44824) 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f7709249700 1 -- 172.21.15.97:0/2672925848 learned_addr learned my addr 172.21.15.97:0/2672925848 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f7708a48700 1 --2- 172.21.15.97:0/2672925848 >> v2:172.21.15.183:3300/0 conn(0x7f770415e080 0x7f77040b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.618+0000 7f7703fff700 1 -- 172.21.15.97:0/2672925848 >> v2:172.21.15.183:3300/0 conn(0x7f770415e080 msgr2=0x7f77040b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f7703fff700 1 --2- 172.21.15.97:0/2672925848 >> v2:172.21.15.183:3300/0 conn(0x7f770415e080 0x7f77040b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f7703fff700 1 -- 172.21.15.97:0/2672925848 >> v1:172.21.15.97:6789/0 conn(0x7f770406c000 legacy=0x7f7704073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f7703fff700 1 -- 172.21.15.97:0/2672925848 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040ae710 con 0x7f77040af4d0 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f77037fe700 1 -- 172.21.15.97:0/2672925848 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130886207 0 0) 0x7f77040ae430 con 0x7f770406c000 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f7703fff700 1 --2- 172.21.15.97:0/2672925848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77040af4d0 0x7f77040afb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7704072a90 tx=0x7f76f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c3e91656e8ee77 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f77037fe700 1 -- 172.21.15.97:0/2672925848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011830 con 0x7f77040af4d0 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f77037fe700 1 -- 172.21.15.97:0/2672925848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f0011990 con 0x7f77040af4d0 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f77037fe700 1 -- 172.21.15.97:0/2672925848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0011d20 con 0x7f77040af4d0 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f770acac700 1 -- 172.21.15.97:0/2672925848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77040af4d0 msgr2=0x7f77040afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f770acac700 1 --2- 172.21.15.97:0/2672925848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77040af4d0 0x7f77040afb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7704072a90 tx=0x7f76f0009b50 comp rx=0 tx=0).stop 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f770acac700 1 -- 172.21.15.97:0/2672925848 shutdown_connections 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f770acac700 1 --2- 172.21.15.97:0/2672925848 >> v2:172.21.15.183:3300/0 conn(0x7f770415e080 0x7f77040b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f770acac700 1 --2- 172.21.15.97:0/2672925848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77040af4d0 0x7f77040afb90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f770acac700 1 -- 172.21.15.97:0/2672925848 >> 172.21.15.97:0/2672925848 conn(0x7f7704071b90 msgr2=0x7f7704071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f770acac700 1 -- 172.21.15.97:0/2672925848 shutdown_connections 2023-11-24T06:08:51.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.619+0000 7f770acac700 1 -- 172.21.15.97:0/2672925848 wait complete. 2023-11-24T06:08:51.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.621+0000 7f770acac700 1 Processor -- start 2023-11-24T06:08:51.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.621+0000 7f770acac700 1 -- start start 2023-11-24T06:08:51.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f770acac700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f770406c000 0x7f77041426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f770acac700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040af4d0 0x7f77041520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f770acac700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770415e080 0x7f7704141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f770acac700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77041538f0 con 0x7f770415e080 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f770acac700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704161350 con 0x7f770406c000 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f770acac700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7704154170 con 0x7f77040af4d0 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7709249700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770415e080 0x7f7704141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7709249700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770415e080 0x7f7704141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44832/0 (socket says 172.21.15.97:44832) 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7709249700 1 -- 172.21.15.97:0/6437946 learned_addr learned my addr 172.21.15.97:0/6437946 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7708a48700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f770406c000 0x7f77041426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7703fff700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040af4d0 0x7f77041520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7709249700 1 -- 172.21.15.97:0/6437946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040af4d0 msgr2=0x7f77041520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7709249700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040af4d0 0x7f77041520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7709249700 1 -- 172.21.15.97:0/6437946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f770406c000 msgr2=0x7f77041426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7709249700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f770406c000 0x7f77041426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.622+0000 7f7709249700 1 -- 172.21.15.97:0/6437946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00094a0 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.623+0000 7f7709249700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770415e080 0x7f7704141e20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f76fc002230 tx=0x7f76fc00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.623+0000 7f77017fa700 1 -- 172.21.15.97:0/6437946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc00aac0 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.623+0000 7f77017fa700 1 -- 172.21.15.97:0/6437946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc00ac40 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.623+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7704161070 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.623+0000 7f77017fa700 1 -- 172.21.15.97:0/6437946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0070d0 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.623+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7704153bd0 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.624+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77040753f0 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.624+0000 7f77017fa700 1 -- 172.21.15.97:0/6437946 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76fc007250 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.624+0000 7f77017fa700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f76e806aed0 0x7f76e806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.625+0000 7f77017fa700 1 -- 172.21.15.97:0/6437946 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76fc088c80 con 0x7f770415e080 2023-11-24T06:08:51.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.625+0000 7f7708a48700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f76e806aed0 0x7f76e806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:51.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.625+0000 7f7708a48700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f76e806aed0 0x7f76e806d390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f76f8006b20 tx=0x7f76f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:51.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.629+0000 7f77017fa700 1 -- 172.21.15.97:0/6437946 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76fc054080 con 0x7f770415e080 2023-11-24T06:08:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.860+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77040d09b0 con 0x7f770415e080 2023-11-24T06:08:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.860+0000 7f77017fa700 1 -- 172.21.15.97:0/6437946 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f76fc018090 con 0x7f770415e080 2023-11-24T06:08:51.864 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:08:51.864 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f76e806aed0 msgr2=0x7f76e806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f76e806aed0 0x7f76e806d390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f76f8006b20 tx=0x7f76f8008040 comp rx=0 tx=0).stop 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770415e080 msgr2=0x7f7704141e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770415e080 0x7f7704141e20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f76fc002230 tx=0x7f76fc00cf50 comp rx=0 tx=0).stop 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 shutdown_connections 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f770415e080 0x7f7704141e20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f770406c000 0x7f77041426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f76e806aed0 0x7f76e806d390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 --2- 172.21.15.97:0/6437946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77040af4d0 0x7f77041520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.862+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 >> 172.21.15.97:0/6437946 conn(0x7f7704071b90 msgr2=0x7f7704073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.863+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 shutdown_connections 2023-11-24T06:08:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:51.863+0000 7f770acac700 1 -- 172.21.15.97:0/6437946 wait complete. 2023-11-24T06:08:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2023-11-24T06:08:53.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:08:53.923 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:08:53.925 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:08:54.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.010+0000 7f4bbfa36700 1 Processor -- start 2023-11-24T06:08:54.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.011+0000 7f4bbfa36700 1 -- start start 2023-11-24T06:08:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.011+0000 7f4bbfa36700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4bb815bdd0 0x7f4bb81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.011+0000 7f4bbfa36700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb80b22d0 con 0x7f4bb80d1c70 2023-11-24T06:08:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.011+0000 7f4bbfa36700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb80b25b0 con 0x7f4bb80b3370 2023-11-24T06:08:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.011+0000 7f4bbfa36700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb80b2f50 con 0x7f4bb815bdd0 2023-11-24T06:08:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4bbdfd3700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4bb80d1c70 0x7f4bb8165ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37294/0 (socket says 172.21.15.97:37294) 2023-11-24T06:08:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4bbdfd3700 1 -- 172.21.15.97:0/1127228794 learned_addr learned my addr 172.21.15.97:0/1127228794 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397509155 0 0) 0x7f4bb80b22d0 con 0x7f4bb80d1c70 2023-11-24T06:08:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba40035b0 con 0x7f4bb80d1c70 2023-11-24T06:08:54.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/1127228794 >> v2:172.21.15.183:3300/0 conn(0x7f4bb815bdd0 0x7f4bb81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3597630697 0 0) 0x7f4ba40035b0 con 0x7f4bb80d1c70 2023-11-24T06:08:54.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 >> v2:172.21.15.183:3300/0 conn(0x7f4bb815bdd0 msgr2=0x7f4bb81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:54.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 --2- 172.21.15.97:0/1127228794 >> v2:172.21.15.183:3300/0 conn(0x7f4bb815bdd0 0x7f4bb81604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 >> v1:172.21.15.183:6789/0 conn(0x7f4bb80b3370 legacy=0x7f4bb80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80b2890 con 0x7f4bb80d1c70 2023-11-24T06:08:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f4bb00019f0 con 0x7f4bb80d1c70 2023-11-24T06:08:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 >> v1:172.21.15.97:6789/0 conn(0x7f4bb80d1c70 legacy=0x7f4bb8165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ba4003a00 0x7f4ba4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ba4004850 0x7f4ba4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4005210 0x7f4ba400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80b2f50 con 0x7f4ba4003a00 2023-11-24T06:08:54.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba400c0b0 con 0x7f4ba4004850 2023-11-24T06:08:54.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba400c270 con 0x7f4ba4005210 2023-11-24T06:08:54.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ba4003a00 0x7f4ba4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4bbcfd1700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ba4004850 0x7f4ba4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.012+0000 7f4bbdfd3700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4005210 0x7f4ba400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbd7d2700 1 -- 172.21.15.97:0/1127228794 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4005210 msgr2=0x7f4ba400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:54.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4005210 0x7f4ba400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbd7d2700 1 -- 172.21.15.97:0/1127228794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ba4004850 msgr2=0x7f4ba4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:54.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ba4004850 0x7f4ba4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbdfd3700 1 -- 172.21.15.97:0/1127228794 reap_dead start 2023-11-24T06:08:54.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbd7d2700 1 -- 172.21.15.97:0/1127228794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba40035b0 con 0x7f4ba4003a00 2023-11-24T06:08:54.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ba4003a00 0x7f4ba4003e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009b60 tx=0x7f4ba4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b31a793a56200 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800e6c0 con 0x7f4ba4003a00 2023-11-24T06:08:54.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba800e820 con 0x7f4ba4003a00 2023-11-24T06:08:54.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4baffff700 1 -- 172.21.15.97:0/1127228794 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba800ebb0 con 0x7f4ba4003a00 2023-11-24T06:08:54.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbfa36700 1 -- 172.21.15.97:0/1127228794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ba4003a00 msgr2=0x7f4ba4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbfa36700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ba4003a00 0x7f4ba4003e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009b60 tx=0x7f4ba4005880 comp rx=0 tx=0).stop 2023-11-24T06:08:54.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbfa36700 1 -- 172.21.15.97:0/1127228794 shutdown_connections 2023-11-24T06:08:54.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbfa36700 1 --2- 172.21.15.97:0/1127228794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ba4003a00 0x7f4ba4003e20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.013+0000 7f4bbfa36700 1 -- 172.21.15.97:0/1127228794 >> 172.21.15.97:0/1127228794 conn(0x7f4bb80731a0 msgr2=0x7f4bb809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:54.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.014+0000 7f4bbfa36700 1 -- 172.21.15.97:0/1127228794 shutdown_connections 2023-11-24T06:08:54.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.014+0000 7f4bbfa36700 1 -- 172.21.15.97:0/1127228794 wait complete. 2023-11-24T06:08:54.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.015+0000 7f4bbfa36700 1 Processor -- start 2023-11-24T06:08:54.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.015+0000 7f4bbfa36700 1 -- start start 2023-11-24T06:08:54.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.015+0000 7f4bbfa36700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4003a00 0x7f4bb80ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.015+0000 7f4bbfa36700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb80c7400 0x7f4bb8148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.015+0000 7f4bbfa36700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb80b48d0 0x7f4bb80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.015+0000 7f4bbfa36700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80b52a0 con 0x7f4bb80c7400 2023-11-24T06:08:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.015+0000 7f4bbfa36700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80cd150 con 0x7f4bb80b48d0 2023-11-24T06:08:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.015+0000 7f4bbfa36700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80cd430 con 0x7f4ba4003a00 2023-11-24T06:08:54.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb80c7400 0x7f4bb8148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb80c7400 0x7f4bb8148fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44854/0 (socket says 172.21.15.97:44854) 2023-11-24T06:08:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 -- 172.21.15.97:0/3268116886 learned_addr learned my addr 172.21.15.97:0/3268116886 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbdfd3700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb80b48d0 0x7f4bb80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4003a00 0x7f4bb80ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 -- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4003a00 msgr2=0x7f4bb80ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4003a00 0x7f4bb80ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 -- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb80b48d0 msgr2=0x7f4bb80b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb80b48d0 0x7f4bb80b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 -- 172.21.15.97:0/3268116886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80cd710 con 0x7f4bb80c7400 2023-11-24T06:08:54.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbdfd3700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb80b48d0 0x7f4bb80b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:54.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4003a00 0x7f4bb80ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:54.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbcfd1700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb80c7400 0x7f4bb8148fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400f7a0 tx=0x7f4bb400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4badffb700 1 -- 172.21.15.97:0/3268116886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40052e0 con 0x7f4bb80c7400 2023-11-24T06:08:54.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4badffb700 1 -- 172.21.15.97:0/3268116886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb4005440 con 0x7f4bb80c7400 2023-11-24T06:08:54.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb80cd9f0 con 0x7f4bb80c7400 2023-11-24T06:08:54.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4badffb700 1 -- 172.21.15.97:0/3268116886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40038c0 con 0x7f4bb80c7400 2023-11-24T06:08:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.016+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb80c7880 con 0x7f4bb80c7400 2023-11-24T06:08:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.017+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb80b22d0 con 0x7f4bb80c7400 2023-11-24T06:08:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.018+0000 7f4badffb700 1 -- 172.21.15.97:0/3268116886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4bb4003a20 con 0x7f4bb80c7400 2023-11-24T06:08:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.018+0000 7f4badffb700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b9c06aed0 0x7f4b9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.018+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b9c06aed0 0x7f4b9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.018+0000 7f4badffb700 1 -- 172.21.15.97:0/3268116886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4bb4088f40 con 0x7f4bb80c7400 2023-11-24T06:08:54.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.019+0000 7f4bbd7d2700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b9c06aed0 0x7f4b9c06d390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80090b0 tx=0x7f4ba8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.023+0000 7f4badffb700 1 -- 172.21.15.97:0/3268116886 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4bb4054340 con 0x7f4bb80c7400 2023-11-24T06:08:54.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.253+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bb80c7a10 con 0x7f4bb80c7400 2023-11-24T06:08:54.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.254+0000 7f4badffb700 1 -- 172.21.15.97:0/3268116886 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f4bb80c7a10 con 0x7f4bb80c7400 2023-11-24T06:08:54.257 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:08:54.257 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T06:08:54.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b9c06aed0 msgr2=0x7f4b9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b9c06aed0 0x7f4b9c06d390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80090b0 tx=0x7f4ba8008030 comp rx=0 tx=0).stop 2023-11-24T06:08:54.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb80c7400 msgr2=0x7f4bb8148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb80c7400 0x7f4bb8148fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400f7a0 tx=0x7f4bb400fb60 comp rx=0 tx=0).stop 2023-11-24T06:08:54.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 shutdown_connections 2023-11-24T06:08:54.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bb80b48d0 0x7f4bb80b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b9c06aed0 0x7f4b9c06d390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ba4003a00 0x7f4bb80ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 --2- 172.21.15.97:0/3268116886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bb80c7400 0x7f4bb8148fc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 >> 172.21.15.97:0/3268116886 conn(0x7f4bb80731a0 msgr2=0x7f4bb80afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:54.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 shutdown_connections 2023-11-24T06:08:54.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.256+0000 7f4bbfa36700 1 -- 172.21.15.97:0/3268116886 wait complete. 2023-11-24T06:08:54.261 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2023-11-24T06:08:54.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:08:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.429+0000 7fd3e0aeb700 1 Processor -- start 2023-11-24T06:08:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.429+0000 7fd3e0aeb700 1 -- start start 2023-11-24T06:08:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3e0aeb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc06c710 0x7fd3dc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3e0aeb700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3dc0af4d0 0x7fd3dc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3e0aeb700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3dc0ae430 con 0x7fd3dc15e080 2023-11-24T06:08:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3e0aeb700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3dc0af0b0 con 0x7fd3dc0af4d0 2023-11-24T06:08:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3e0aeb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc0b05a0 con 0x7fd3dc06c710 2023-11-24T06:08:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3dad9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3dc0af4d0 0x7fd3dc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3da59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd3dc15e080 0x7fd3dc06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37310/0 (socket says 172.21.15.97:37310) 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3da59c700 1 -- 172.21.15.97:0/1008759313 learned_addr learned my addr 172.21.15.97:0/1008759313 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3d9d9b700 1 --2- 172.21.15.97:0/1008759313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc06c710 0x7fd3dc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3dad9d700 1 -- 172.21.15.97:0/1008759313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc06c710 msgr2=0x7fd3dc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3dad9d700 1 --2- 172.21.15.97:0/1008759313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc06c710 0x7fd3dc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3dad9d700 1 -- 172.21.15.97:0/1008759313 >> v1:172.21.15.97:6789/0 conn(0x7fd3dc15e080 legacy=0x7fd3dc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3dad9d700 1 -- 172.21.15.97:0/1008759313 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0ae710 con 0x7fd3dc0af4d0 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3dad9d700 1 --2- 172.21.15.97:0/1008759313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc0af4d0 0x7fd3dc0afb90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009dc0 tx=0x7fd3d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba3c89acd02bf97 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.430+0000 7fd3d959a700 1 -- 172.21.15.97:0/1008759313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400e070 con 0x7fd3dc0af4d0 2023-11-24T06:08:54.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3d959a700 1 -- 172.21.15.97:0/1008759313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d400bb90 con 0x7fd3dc0af4d0 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3d959a700 1 -- 172.21.15.97:0/1008759313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4012430 con 0x7fd3dc0af4d0 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/1008759313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc0af4d0 msgr2=0x7fd3dc0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/1008759313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc0af4d0 0x7fd3dc0afb90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009dc0 tx=0x7fd3d400b330 comp rx=0 tx=0).stop 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/1008759313 shutdown_connections 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/1008759313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc0af4d0 0x7fd3dc0afb90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/1008759313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc06c710 0x7fd3dc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/1008759313 >> 172.21.15.97:0/1008759313 conn(0x7fd3dc071b90 msgr2=0x7fd3dc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/1008759313 shutdown_connections 2023-11-24T06:08:54.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.431+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/1008759313 wait complete. 2023-11-24T06:08:54.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.432+0000 7fd3e0aeb700 1 Processor -- start 2023-11-24T06:08:54.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.432+0000 7fd3e0aeb700 1 -- start start 2023-11-24T06:08:54.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3e0aeb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc06c710 0x7fd3dc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3e0aeb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc0af4d0 0x7fd3dc1f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3e0aeb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc15e080 0x7fd3dc0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3e0aeb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc0d02e0 con 0x7fd3dc06c710 2023-11-24T06:08:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3e0aeb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc0cfb00 con 0x7fd3dc0af4d0 2023-11-24T06:08:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3e0aeb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc157ed0 con 0x7fd3dc15e080 2023-11-24T06:08:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3da59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc06c710 0x7fd3dc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3da59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc06c710 0x7fd3dc14da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44864/0 (socket says 172.21.15.97:44864) 2023-11-24T06:08:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3da59c700 1 -- 172.21.15.97:0/3017100344 learned_addr learned my addr 172.21.15.97:0/3017100344 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:54.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3d9d9b700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc0af4d0 0x7fd3dc1f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.433+0000 7fd3dad9d700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc15e080 0x7fd3dc0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3da59c700 1 -- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc15e080 msgr2=0x7fd3dc0b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3da59c700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc15e080 0x7fd3dc0b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3da59c700 1 -- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc0af4d0 msgr2=0x7fd3dc1f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3da59c700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc0af4d0 0x7fd3dc1f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3da59c700 1 -- 172.21.15.97:0/3017100344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009a70 con 0x7fd3dc06c710 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3dad9d700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc15e080 0x7fd3dc0b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3d9d9b700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc0af4d0 0x7fd3dc1f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3da59c700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc06c710 0x7fd3dc14da10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00c4b0 tx=0x7fd3cc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3caffd700 1 -- 172.21.15.97:0/3017100344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc005d70 con 0x7fd3dc06c710 2023-11-24T06:08:54.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3caffd700 1 -- 172.21.15.97:0/3017100344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc0070d0 con 0x7fd3dc06c710 2023-11-24T06:08:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc14df50 con 0x7fd3dc06c710 2023-11-24T06:08:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3caffd700 1 -- 172.21.15.97:0/3017100344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc007440 con 0x7fd3dc06c710 2023-11-24T06:08:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.434+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc1f7e10 con 0x7fd3dc06c710 2023-11-24T06:08:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.435+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3dc0753f0 con 0x7fd3dc06c710 2023-11-24T06:08:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.435+0000 7fd3caffd700 1 -- 172.21.15.97:0/3017100344 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3cc0075a0 con 0x7fd3dc06c710 2023-11-24T06:08:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.436+0000 7fd3caffd700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3bc06aed0 0x7fd3bc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.436+0000 7fd3caffd700 1 -- 172.21.15.97:0/3017100344 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd3cc088530 con 0x7fd3dc06c710 2023-11-24T06:08:54.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.436+0000 7fd3d9d9b700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3bc06aed0 0x7fd3bc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.436+0000 7fd3d9d9b700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3bc06aed0 0x7fd3bc06d390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc15a9a0 tx=0x7fd3d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.440+0000 7fd3caffd700 1 -- 172.21.15.97:0/3017100344 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3cc053a70 con 0x7fd3dc06c710 2023-11-24T06:08:54.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.630+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3dc067f60 con 0x7fd3dc06c710 2023-11-24T06:08:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.631+0000 7fd3caffd700 1 -- 172.21.15.97:0/3017100344 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12862 (secure 0 0 0) 0x7fd3cc057090 con 0x7fd3dc06c710 2023-11-24T06:08:54.634 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:08:54.634 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":16,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:47.309680+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:08:54.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3bc06aed0 msgr2=0x7fd3bc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3bc06aed0 0x7fd3bc06d390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc15a9a0 tx=0x7fd3d0008040 comp rx=0 tx=0).stop 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc06c710 msgr2=0x7fd3dc14da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc06c710 0x7fd3dc14da10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc00c4b0 tx=0x7fd3cc00c870 comp rx=0 tx=0).stop 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 shutdown_connections 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3bc06aed0 0x7fd3bc06d390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3dc15e080 0x7fd3dc0b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3dc06c710 0x7fd3dc14da10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 --2- 172.21.15.97:0/3017100344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3dc0af4d0 0x7fd3dc1f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 >> 172.21.15.97:0/3017100344 conn(0x7fd3dc071b90 msgr2=0x7fd3dc06e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 shutdown_connections 2023-11-24T06:08:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.633+0000 7fd3e0aeb700 1 -- 172.21.15.97:0/3017100344 wait complete. 2023-11-24T06:08:54.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:08:54.648 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:08:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.768+0000 7f40d0c4f700 1 Processor -- start 2023-11-24T06:08:54.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.768+0000 7f40d0c4f700 1 -- start start 2023-11-24T06:08:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.769+0000 7f40d0c4f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0d45d0 0x7f40cc0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.769+0000 7f40d0c4f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40cc0d2930 con 0x7f40cc15e140 2023-11-24T06:08:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.769+0000 7f40d0c4f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40cc0d2c10 con 0x7f40cc0d39d0 2023-11-24T06:08:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.769+0000 7f40d0c4f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc15e520 con 0x7f40cc0d45d0 2023-11-24T06:08:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.769+0000 7f40ca59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f40cc15e140 0x7f40cc0d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37324/0 (socket says 172.21.15.97:37324) 2023-11-24T06:08:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.769+0000 7f40ca59c700 1 -- 172.21.15.97:0/2057882428 learned_addr learned my addr 172.21.15.97:0/2057882428 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811619922 0 0) 0x7f40cc0d2930 con 0x7f40cc15e140 2023-11-24T06:08:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40b40035b0 con 0x7f40cc15e140 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c9d9b700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0d45d0 0x7f40cc0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3081539181 0 0) 0x7f40b40035b0 con 0x7f40cc15e140 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0d45d0 msgr2=0x7f40cc0b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0d45d0 0x7f40cc0b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 >> v1:172.21.15.183:6789/0 conn(0x7f40cc0d39d0 legacy=0x7f40cc0b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0d2ef0 con 0x7f40cc15e140 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f40c0001670 con 0x7f40cc15e140 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 >> v1:172.21.15.97:6789/0 conn(0x7f40cc15e140 legacy=0x7f40cc0d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b4003a30 0x7f40b4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0d45d0 0x7f40b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4004d90 0x7f40b4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc15e520 con 0x7f40b4003a30 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b400baf0 con 0x7f40b4004d90 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b400bcb0 con 0x7f40cc0d45d0 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40ca59c700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b4003a30 0x7f40b4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40c9d9b700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0d45d0 0x7f40b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40cad9d700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4004d90 0x7f40b4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40ca59c700 1 -- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0d45d0 msgr2=0x7f40b4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40ca59c700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0d45d0 0x7f40b4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40ca59c700 1 -- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4004d90 msgr2=0x7f40b4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40ca59c700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4004d90 0x7f40b4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40ca59c700 1 -- 172.21.15.97:0/2057882428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b40035b0 con 0x7f40b4003a30 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.770+0000 7f40cad9d700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4004d90 0x7f40b4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40ca59c700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b4003a30 0x7f40b4003e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40c0001eb0 tx=0x7f40b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2a8390b19f5289 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0007b70 con 0x7f40b4003a30 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c000ccd0 con 0x7f40b4003a30 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40c959a700 1 -- 172.21.15.97:0/2057882428 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0005ba0 con 0x7f40b4003a30 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40d0c4f700 1 -- 172.21.15.97:0/2057882428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b4003a30 msgr2=0x7f40b4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40d0c4f700 1 --2- 172.21.15.97:0/2057882428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b4003a30 0x7f40b4003e50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40c0001eb0 tx=0x7f40b4005880 comp rx=0 tx=0).stop 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40cad9d700 1 -- 172.21.15.97:0/2057882428 reap_dead start 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40d0c4f700 1 -- 172.21.15.97:0/2057882428 shutdown_connections 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40d0c4f700 1 -- 172.21.15.97:0/2057882428 >> 172.21.15.97:0/2057882428 conn(0x7f40cc06b410 msgr2=0x7f40cc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40d0c4f700 1 -- 172.21.15.97:0/2057882428 shutdown_connections 2023-11-24T06:08:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.771+0000 7f40d0c4f700 1 -- 172.21.15.97:0/2057882428 wait complete. 2023-11-24T06:08:54.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40d0c4f700 1 Processor -- start 2023-11-24T06:08:54.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40d0c4f700 1 -- start start 2023-11-24T06:08:54.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40d0c4f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4003a30 0x7f40cc14d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40d0c4f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0b3990 0x7f40cc0b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40d0c4f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40cc14df10 0x7f40cc158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40d0c4f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc0b68f0 con 0x7f40cc14df10 2023-11-24T06:08:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40d0c4f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc1f6e20 con 0x7f40b4003a30 2023-11-24T06:08:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40d0c4f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc1f7100 con 0x7f40cc0b3990 2023-11-24T06:08:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40cad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40cc14df10 0x7f40cc158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40cad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40cc14df10 0x7f40cc158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44888/0 (socket says 172.21.15.97:44888) 2023-11-24T06:08:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40cad9d700 1 -- 172.21.15.97:0/3491520465 learned_addr learned my addr 172.21.15.97:0/3491520465 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40c9d9b700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0b3990 0x7f40cc0b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.773+0000 7f40ca59c700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4003a30 0x7f40cc14d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40cad9d700 1 -- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0b3990 msgr2=0x7f40cc0b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40cad9d700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0b3990 0x7f40cc0b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40cad9d700 1 -- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4003a30 msgr2=0x7f40cc14d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40cad9d700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4003a30 0x7f40cc14d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40cad9d700 1 -- 172.21.15.97:0/3491520465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc1f73e0 con 0x7f40cc14df10 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40ca59c700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4003a30 0x7f40cc14d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40cad9d700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40cc14df10 0x7f40cc158e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0d1870 tx=0x7f40c400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40baffd700 1 -- 172.21.15.97:0/3491520465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4003c10 con 0x7f40cc14df10 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40baffd700 1 -- 172.21.15.97:0/3491520465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4003d90 con 0x7f40cc14df10 2023-11-24T06:08:54.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc1f76c0 con 0x7f40cc14df10 2023-11-24T06:08:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40baffd700 1 -- 172.21.15.97:0/3491520465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4005310 con 0x7f40cc14df10 2023-11-24T06:08:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.774+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc1593c0 con 0x7f40cc14df10 2023-11-24T06:08:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.775+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc0d1b90 con 0x7f40cc14df10 2023-11-24T06:08:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.775+0000 7f40baffd700 1 -- 172.21.15.97:0/3491520465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40c4005470 con 0x7f40cc14df10 2023-11-24T06:08:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.776+0000 7f40baffd700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40ac06aed0 0x7f40ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.776+0000 7f40ca59c700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40ac06aed0 0x7f40ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.776+0000 7f40baffd700 1 -- 172.21.15.97:0/3491520465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40c4089230 con 0x7f40cc14df10 2023-11-24T06:08:54.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.776+0000 7f40ca59c700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40ac06aed0 0x7f40ac06d390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40c0000c00 tx=0x7f40c0007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:54.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.780+0000 7f40baffd700 1 -- 172.21.15.97:0/3491520465 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40c40546c0 con 0x7f40cc14df10 2023-11-24T06:08:54.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.968+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40cc068fb0 con 0x7f40cc14df10 2023-11-24T06:08:54.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.969+0000 7f40baffd700 1 -- 172.21.15.97:0/3491520465 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40c4057ce0 con 0x7f40cc14df10 2023-11-24T06:08:54.972 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:08:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.970+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40ac06aed0 msgr2=0x7f40ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.970+0000 7f40d0c4f700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40ac06aed0 0x7f40ac06d390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40c0000c00 tx=0x7f40c0007000 comp rx=0 tx=0).stop 2023-11-24T06:08:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.970+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40cc14df10 msgr2=0x7f40cc158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.970+0000 7f40d0c4f700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40cc14df10 0x7f40cc158e80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0d1870 tx=0x7f40c400ee50 comp rx=0 tx=0).stop 2023-11-24T06:08:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.971+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 shutdown_connections 2023-11-24T06:08:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.971+0000 7f40d0c4f700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40cc14df10 0x7f40cc158e80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.971+0000 7f40d0c4f700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b4003a30 0x7f40cc14d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.971+0000 7f40d0c4f700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40ac06aed0 0x7f40ac06d390 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.971+0000 7f40d0c4f700 1 --2- 172.21.15.97:0/3491520465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40cc0b3990 0x7f40cc0b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:54.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.971+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 >> 172.21.15.97:0/3491520465 conn(0x7f40cc06b410 msgr2=0x7f40cc0b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:54.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.971+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 shutdown_connections 2023-11-24T06:08:54.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:54.971+0000 7f40d0c4f700 1 -- 172.21.15.97:0/3491520465 wait complete. 2023-11-24T06:08:54.985 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:08:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.102+0000 7fb3416f6700 1 Processor -- start 2023-11-24T06:08:55.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.102+0000 7fb3416f6700 1 -- start start 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb3416f6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb33c15e080 0x7fb33c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb3416f6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb33c0af4d0 0x7fb33c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb3416f6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 0x7fb33c06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb3416f6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c0af0b0 con 0x7fb33c15e080 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb3416f6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c06c9f0 con 0x7fb33c0af4d0 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb3416f6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0b05a0 con 0x7fb33c06c000 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb33affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb33c15e080 0x7fb33c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb33affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb33c15e080 0x7fb33c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44892/0 (socket says 172.21.15.97:44892) 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.103+0000 7fb33affd700 1 -- 172.21.15.97:0/381242306 learned_addr learned my addr 172.21.15.97:0/381242306 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb33b7fe700 1 --2- 172.21.15.97:0/381242306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 0x7fb33c06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb33a7fc700 1 --2- 172.21.15.97:0/381242306 >> v2:172.21.15.183:3300/0 conn(0x7fb33c0af4d0 0x7fb33c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb33affd700 1 -- 172.21.15.97:0/381242306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 msgr2=0x7fb33c06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb33affd700 1 --2- 172.21.15.97:0/381242306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 0x7fb33c06c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb33affd700 1 -- 172.21.15.97:0/381242306 >> v2:172.21.15.183:3300/0 conn(0x7fb33c0af4d0 msgr2=0x7fb33c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb33affd700 1 --2- 172.21.15.97:0/381242306 >> v2:172.21.15.183:3300/0 conn(0x7fb33c0af4d0 0x7fb33c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb33affd700 1 -- 172.21.15.97:0/381242306 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0ae430 con 0x7fb33c15e080 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb33affd700 1 --2- 172.21.15.97:0/381242306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c15e080 0x7fb33c0b5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb33c06e4e0 tx=0x7fb330009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444e4bb5cf610fa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb339ffb700 1 -- 172.21.15.97:0/381242306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000b870 con 0x7fb33c15e080 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb339ffb700 1 -- 172.21.15.97:0/381242306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33000b9d0 con 0x7fb33c15e080 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb339ffb700 1 -- 172.21.15.97:0/381242306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33000bd60 con 0x7fb33c15e080 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb3416f6700 1 -- 172.21.15.97:0/381242306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c15e080 msgr2=0x7fb33c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.104+0000 7fb3416f6700 1 --2- 172.21.15.97:0/381242306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c15e080 0x7fb33c0b5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb33c06e4e0 tx=0x7fb330009bd0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.105+0000 7fb3416f6700 1 -- 172.21.15.97:0/381242306 shutdown_connections 2023-11-24T06:08:55.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.105+0000 7fb3416f6700 1 --2- 172.21.15.97:0/381242306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 0x7fb33c06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.105+0000 7fb3416f6700 1 --2- 172.21.15.97:0/381242306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c15e080 0x7fb33c0b5050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.105+0000 7fb3416f6700 1 --2- 172.21.15.97:0/381242306 >> v2:172.21.15.183:3300/0 conn(0x7fb33c0af4d0 0x7fb33c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.105+0000 7fb3416f6700 1 -- 172.21.15.97:0/381242306 >> 172.21.15.97:0/381242306 conn(0x7fb33c071b90 msgr2=0x7fb33c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.105+0000 7fb3416f6700 1 -- 172.21.15.97:0/381242306 shutdown_connections 2023-11-24T06:08:55.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.105+0000 7fb3416f6700 1 -- 172.21.15.97:0/381242306 wait complete. 2023-11-24T06:08:55.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.106+0000 7fb3416f6700 1 Processor -- start 2023-11-24T06:08:55.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.106+0000 7fb3416f6700 1 -- start start 2023-11-24T06:08:55.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3416f6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 0x7fb33c151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3416f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c0af4d0 0x7fb33c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3416f6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb33c15e080 0x7fb33c154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3416f6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c156840 con 0x7fb33c0af4d0 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3416f6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c151e80 con 0x7fb33c15e080 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3416f6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c157340 con 0x7fb33c06c000 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c0af4d0 0x7fb33c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c0af4d0 0x7fb33c15f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44900/0 (socket says 172.21.15.97:44900) 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 -- 172.21.15.97:0/1940153194 learned_addr learned my addr 172.21.15.97:0/1940153194 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33b7fe700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb33c15e080 0x7fb33c154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33affd700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 0x7fb33c151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 -- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 msgr2=0x7fb33c151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 0x7fb33c151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 -- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb33c15e080 msgr2=0x7fb33c154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb33c15e080 0x7fb33c154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 -- 172.21.15.97:0/1940153194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330009550 con 0x7fb33c0af4d0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33b7fe700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb33c15e080 0x7fb33c154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb33a7fc700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c0af4d0 0x7fb33c15f4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00e7e0 tx=0x7fb32c00eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3237fe700 1 -- 172.21.15.97:0/1940153194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c013610 con 0x7fb33c0af4d0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3237fe700 1 -- 172.21.15.97:0/1940153194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c013770 con 0x7fb33c0af4d0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c157620 con 0x7fb33c0af4d0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3237fe700 1 -- 172.21.15.97:0/1940153194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c013ae0 con 0x7fb33c0af4d0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.107+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c157be0 con 0x7fb33c0af4d0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.108+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c0753f0 con 0x7fb33c0af4d0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.109+0000 7fb3237fe700 1 -- 172.21.15.97:0/1940153194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb32c013c40 con 0x7fb33c0af4d0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.109+0000 7fb3237fe700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb32806aed0 0x7fb32806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.109+0000 7fb33affd700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb32806aed0 0x7fb32806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.109+0000 7fb3237fe700 1 -- 172.21.15.97:0/1940153194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb32c088f40 con 0x7fb33c0af4d0 2023-11-24T06:08:55.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.109+0000 7fb33affd700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb32806aed0 0x7fb32806d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb330014040 tx=0x7fb330008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:55.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.113+0000 7fb3237fe700 1 -- 172.21.15.97:0/1940153194 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb32c054340 con 0x7fb33c0af4d0 2023-11-24T06:08:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.309+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb33c0b00d0 con 0x7fb33c0af4d0 2023-11-24T06:08:55.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.310+0000 7fb3237fe700 1 -- 172.21.15.97:0/1940153194 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb32c057960 con 0x7fb33c0af4d0 2023-11-24T06:08:55.313 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:08:55.313 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:08:55.313 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:08:55.313 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-11-24T06:08:55.313 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-11-24T06:08:55.313 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:08:55.313 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.311+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb32806aed0 msgr2=0x7fb32806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.311+0000 7fb3416f6700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb32806aed0 0x7fb32806d390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb330014040 tx=0x7fb330008030 comp rx=0 tx=0).stop 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.311+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c0af4d0 msgr2=0x7fb33c15f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.311+0000 7fb3416f6700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c0af4d0 0x7fb33c15f4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00e7e0 tx=0x7fb32c00eba0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.312+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 shutdown_connections 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.312+0000 7fb3416f6700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb33c15e080 0x7fb33c154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.312+0000 7fb3416f6700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb32806aed0 0x7fb32806d390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.312+0000 7fb3416f6700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb33c06c000 0x7fb33c151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.312+0000 7fb3416f6700 1 --2- 172.21.15.97:0/1940153194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb33c0af4d0 0x7fb33c15f4a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.312+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 >> 172.21.15.97:0/1940153194 conn(0x7fb33c071b90 msgr2=0x7fb33c073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:55.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.312+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 shutdown_connections 2023-11-24T06:08:55.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.312+0000 7fb3416f6700 1 -- 172.21.15.97:0/1940153194 wait complete. 2023-11-24T06:08:55.326 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:08:55.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:08:55.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.447+0000 7f5d46f4d700 1 Processor -- start 2023-11-24T06:08:55.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.448+0000 7f5d46f4d700 1 -- start start 2023-11-24T06:08:55.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.448+0000 7f5d46f4d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5d400b01e0 0x7f5d4006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.448+0000 7f5d46f4d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400746a0 0x7f5d400b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.448+0000 7f5d46f4d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d400b05c0 con 0x7f5d400b59c0 2023-11-24T06:08:55.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.448+0000 7f5d46f4d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d401470a0 con 0x7f5d400b01e0 2023-11-24T06:08:55.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.448+0000 7f5d46f4d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400b5e70 con 0x7f5d400746a0 2023-11-24T06:08:55.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.448+0000 7f5d3ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5d400b59c0 0x7f5d40073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37358/0 (socket says 172.21.15.97:37358) 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d3ffff700 1 -- 172.21.15.97:0/4141668984 learned_addr learned my addr 172.21.15.97:0/4141668984 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d454ea700 1 --2- 172.21.15.97:0/4141668984 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400746a0 0x7f5d400b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d44ce9700 1 --2- 172.21.15.97:0/4141668984 >> v2:172.21.15.183:3300/0 conn(0x7f5d400b01e0 0x7f5d4006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d3f7fe700 1 -- 172.21.15.97:0/4141668984 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766071205 0 0) 0x7f5d400b05c0 con 0x7f5d400b59c0 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d3f7fe700 1 -- 172.21.15.97:0/4141668984 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d300035b0 con 0x7f5d400b59c0 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d3f7fe700 1 -- 172.21.15.97:0/4141668984 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576590682 0 0) 0x7f5d300035b0 con 0x7f5d400b59c0 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d454ea700 1 -- 172.21.15.97:0/4141668984 >> v2:172.21.15.183:3300/0 conn(0x7f5d400b01e0 msgr2=0x7f5d4006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d454ea700 1 --2- 172.21.15.97:0/4141668984 >> v2:172.21.15.183:3300/0 conn(0x7f5d400b01e0 0x7f5d4006e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d454ea700 1 -- 172.21.15.97:0/4141668984 >> v1:172.21.15.97:6789/0 conn(0x7f5d400b59c0 legacy=0x7f5d40073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d454ea700 1 -- 172.21.15.97:0/4141668984 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d400b4250 con 0x7f5d400746a0 2023-11-24T06:08:55.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d454ea700 1 --2- 172.21.15.97:0/4141668984 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400746a0 0x7f5d400b3b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009d90 tx=0x7f5d3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb46e64b6a5a435 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d3f7fe700 1 -- 172.21.15.97:0/4141668984 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800e070 con 0x7f5d400746a0 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d3f7fe700 1 -- 172.21.15.97:0/4141668984 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38003a20 con 0x7f5d400746a0 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.449+0000 7f5d3f7fe700 1 -- 172.21.15.97:0/4141668984 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c001670 con 0x7f5d400746a0 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.450+0000 7f5d46f4d700 1 -- 172.21.15.97:0/4141668984 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400746a0 msgr2=0x7f5d400b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.450+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/4141668984 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400746a0 0x7f5d400b3b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009d90 tx=0x7f5d3800b330 comp rx=0 tx=0).stop 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.450+0000 7f5d46f4d700 1 -- 172.21.15.97:0/4141668984 shutdown_connections 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.450+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/4141668984 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400746a0 0x7f5d400b3b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.450+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/4141668984 >> v2:172.21.15.183:3300/0 conn(0x7f5d400b01e0 0x7f5d4006e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.450+0000 7f5d46f4d700 1 -- 172.21.15.97:0/4141668984 >> 172.21.15.97:0/4141668984 conn(0x7f5d400d0b40 msgr2=0x7f5d400d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.450+0000 7f5d46f4d700 1 -- 172.21.15.97:0/4141668984 shutdown_connections 2023-11-24T06:08:55.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.450+0000 7f5d46f4d700 1 -- 172.21.15.97:0/4141668984 wait complete. 2023-11-24T06:08:55.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.451+0000 7f5d46f4d700 1 Processor -- start 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.451+0000 7f5d46f4d700 1 -- start start 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d46f4d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400746a0 0x7f5d401615d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d46f4d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400b01e0 0x7f5d400ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d46f4d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d400b59c0 0x7f5d401f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d46f4d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400c8860 con 0x7f5d400746a0 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d46f4d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400c7c90 con 0x7f5d400b59c0 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d46f4d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40161b10 con 0x7f5d400b01e0 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d44ce9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400746a0 0x7f5d401615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d44ce9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400746a0 0x7f5d401615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44910/0 (socket says 172.21.15.97:44910) 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d44ce9700 1 -- 172.21.15.97:0/3894848044 learned_addr learned my addr 172.21.15.97:0/3894848044 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:55.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d3ffff700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400b01e0 0x7f5d400ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d454ea700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d400b59c0 0x7f5d401f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d44ce9700 1 -- 172.21.15.97:0/3894848044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400b01e0 msgr2=0x7f5d400ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d44ce9700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400b01e0 0x7f5d400ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d44ce9700 1 -- 172.21.15.97:0/3894848044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d400b59c0 msgr2=0x7f5d401f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d44ce9700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d400b59c0 0x7f5d401f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.452+0000 7f5d44ce9700 1 -- 172.21.15.97:0/3894848044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d38009a70 con 0x7f5d400746a0 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.453+0000 7f5d44ce9700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400746a0 0x7f5d401615d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400e9b0 tx=0x7f5d3400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.453+0000 7f5d3d7fa700 1 -- 172.21.15.97:0/3894848044 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34009710 con 0x7f5d400746a0 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.453+0000 7f5d3d7fa700 1 -- 172.21.15.97:0/3894848044 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400ee50 con 0x7f5d400746a0 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.453+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d400c7f30 con 0x7f5d400746a0 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.453+0000 7f5d3d7fa700 1 -- 172.21.15.97:0/3894848044 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400ca50 con 0x7f5d400746a0 2023-11-24T06:08:55.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.453+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d400c84f0 con 0x7f5d400746a0 2023-11-24T06:08:55.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.454+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d400b5430 con 0x7f5d400746a0 2023-11-24T06:08:55.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.454+0000 7f5d3d7fa700 1 -- 172.21.15.97:0/3894848044 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d3400cbb0 con 0x7f5d400746a0 2023-11-24T06:08:55.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.455+0000 7f5d3d7fa700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d2406aed0 0x7f5d2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:55.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.455+0000 7f5d3d7fa700 1 -- 172.21.15.97:0/3894848044 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d340898f0 con 0x7f5d400746a0 2023-11-24T06:08:55.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.455+0000 7f5d3ffff700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d2406aed0 0x7f5d2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:55.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.455+0000 7f5d3ffff700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d2406aed0 0x7f5d2406d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d400c9d30 tx=0x7f5d2c001e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:55.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.459+0000 7f5d3d7fa700 1 -- 172.21.15.97:0/3894848044 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d34054e60 con 0x7f5d400746a0 2023-11-24T06:08:55.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:55.648+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5d40067f60 con 0x7f5d400746a0 2023-11-24T06:08:56.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.498+0000 7f5d3d7fa700 1 -- 172.21.15.97:0/3894848044 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5d340170b0 con 0x7f5d400746a0 2023-11-24T06:08:56.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d2406aed0 msgr2=0x7f5d2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:56.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d2406aed0 0x7f5d2406d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5d400c9d30 tx=0x7f5d2c001e40 comp rx=0 tx=0).stop 2023-11-24T06:08:56.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400746a0 msgr2=0x7f5d401615d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:56.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400746a0 0x7f5d401615d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400e9b0 tx=0x7f5d3400ecc0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 shutdown_connections 2023-11-24T06:08:56.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d400b59c0 0x7f5d401f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400746a0 0x7f5d401615d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d2406aed0 0x7f5d2406d390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 --2- 172.21.15.97:0/3894848044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5d400b01e0 0x7f5d400ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 >> 172.21.15.97:0/3894848044 conn(0x7f5d400d0b40 msgr2=0x7f5d40072450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:56.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 shutdown_connections 2023-11-24T06:08:56.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.501+0000 7f5d46f4d700 1 -- 172.21.15.97:0/3894848044 wait complete. 2023-11-24T06:08:56.505 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-11-24T06:08:56.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:08:56.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.652+0000 7f4aeb5f3700 1 Processor -- start 2023-11-24T06:08:56.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.652+0000 7f4aeb5f3700 1 -- start start 2023-11-24T06:08:56.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4aeb5f3700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ae415e160 0x7f4ae40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:56.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4aeb5f3700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4ae40afc90 0x7f4ae4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:56.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4aeb5f3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 0x7f4ae406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:56.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4aeb5f3700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae40b4500 con 0x7f4ae415e160 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4aeb5f3700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae4074960 con 0x7f4ae40afc90 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4aeb5f3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae40b3c60 con 0x7f4ae40744d0 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ae415e160 0x7f4ae40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4ae415e160 0x7f4ae40b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44918/0 (socket says 172.21.15.97:44918) 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 -- 172.21.15.97:0/3952290821 learned_addr learned my addr 172.21.15.97:0/3952290821 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 -- 172.21.15.97:0/3952290821 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 msgr2=0x7f4ae406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae9b90700 1 --2- 172.21.15.97:0/3952290821 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 0x7f4ae406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae8b8e700 1 --2- 172.21.15.97:0/3952290821 >> v2:172.21.15.183:3300/0 conn(0x7f4ae40afc90 0x7f4ae4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 --2- 172.21.15.97:0/3952290821 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 0x7f4ae406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 -- 172.21.15.97:0/3952290821 >> v2:172.21.15.183:3300/0 conn(0x7f4ae40afc90 msgr2=0x7f4ae4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:56.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 --2- 172.21.15.97:0/3952290821 >> v2:172.21.15.183:3300/0 conn(0x7f4ae40afc90 0x7f4ae4073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 -- 172.21.15.97:0/3952290821 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae40b58c0 con 0x7f4ae415e160 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4ae938f700 1 --2- 172.21.15.97:0/3952290821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae40b5da0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40d0880 tx=0x7f4ae0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc9ed4510ccaafe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.653+0000 7f4adbfff700 1 -- 172.21.15.97:0/3952290821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000b870 con 0x7f4ae415e160 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4adbfff700 1 -- 172.21.15.97:0/3952290821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae000b9d0 con 0x7f4ae415e160 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4adbfff700 1 -- 172.21.15.97:0/3952290821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000bd60 con 0x7f4ae415e160 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/3952290821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 msgr2=0x7f4ae40b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/3952290821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae40b5da0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40d0880 tx=0x7f4ae0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/3952290821 shutdown_connections 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/3952290821 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 0x7f4ae406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/3952290821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae40b5da0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/3952290821 >> v2:172.21.15.183:3300/0 conn(0x7f4ae40afc90 0x7f4ae4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/3952290821 >> 172.21.15.97:0/3952290821 conn(0x7f4ae40d0bf0 msgr2=0x7f4ae40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:56.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/3952290821 shutdown_connections 2023-11-24T06:08:56.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.654+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/3952290821 wait complete. 2023-11-24T06:08:56.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.655+0000 7f4aeb5f3700 1 Processor -- start 2023-11-24T06:08:56.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.655+0000 7f4aeb5f3700 1 -- start start 2023-11-24T06:08:56.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4aeb5f3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 0x7f4ae414c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:56.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4aeb5f3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae40afc90 0x7f4ae415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:56.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4aeb5f3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae415eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:56.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4aeb5f3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4152240 con 0x7f4ae415e160 2023-11-24T06:08:56.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4aeb5f3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae414ddd0 con 0x7f4ae40afc90 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4aeb5f3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae415f170 con 0x7f4ae40744d0 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae9b90700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae415eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae9b90700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae415eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44928/0 (socket says 172.21.15.97:44928) 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae9b90700 1 -- 172.21.15.97:0/4077038203 learned_addr learned my addr 172.21.15.97:0/4077038203 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae938f700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 0x7f4ae414c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae8b8e700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae40afc90 0x7f4ae415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae9b90700 1 -- 172.21.15.97:0/4077038203 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 msgr2=0x7f4ae414c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae9b90700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 0x7f4ae414c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae9b90700 1 -- 172.21.15.97:0/4077038203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae40afc90 msgr2=0x7f4ae415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae9b90700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae40afc90 0x7f4ae415e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.656+0000 7f4ae9b90700 1 -- 172.21.15.97:0/4077038203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae0009550 con 0x7f4ae415e160 2023-11-24T06:08:56.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.657+0000 7f4ae9b90700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae415eaf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00cd70 tx=0x7f4adc00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.657+0000 7f4ad9ffb700 1 -- 172.21.15.97:0/4077038203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc003a20 con 0x7f4ae415e160 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.657+0000 7f4ad9ffb700 1 -- 172.21.15.97:0/4077038203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4adc003b80 con 0x7f4ae415e160 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.657+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae415f450 con 0x7f4ae415e160 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.657+0000 7f4ad9ffb700 1 -- 172.21.15.97:0/4077038203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc004020 con 0x7f4ae415e160 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.657+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae415fa10 con 0x7f4ae415e160 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.658+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae414dfa0 con 0x7f4ae415e160 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.658+0000 7f4ad9ffb700 1 -- 172.21.15.97:0/4077038203 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4adc004180 con 0x7f4ae415e160 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.659+0000 7f4ad9ffb700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ac806aed0 0x7f4ac806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.659+0000 7f4ae938f700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ac806aed0 0x7f4ac806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.659+0000 7f4ad9ffb700 1 -- 172.21.15.97:0/4077038203 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4adc012070 con 0x7f4ae415e160 2023-11-24T06:08:56.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.659+0000 7f4ae938f700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ac806aed0 0x7f4ac806d390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0014040 tx=0x7f4ae0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:56.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.663+0000 7f4ad9ffb700 1 -- 172.21.15.97:0/4077038203 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ae414dfa0 con 0x7f4ae415e160 2023-11-24T06:08:56.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:56.856+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4ae414dfa0 con 0x7f4ae415e160 2023-11-24T06:08:57.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.502+0000 7f4ad9ffb700 1 -- 172.21.15.97:0/4077038203 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4ae414dfa0 con 0x7f4ae415e160 2023-11-24T06:08:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ac806aed0 msgr2=0x7f4ac806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ac806aed0 0x7f4ac806d390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0014040 tx=0x7f4ae0008030 comp rx=0 tx=0).stop 2023-11-24T06:08:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 msgr2=0x7f4ae415eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae415eaf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00cd70 tx=0x7f4adc00a570 comp rx=0 tx=0).stop 2023-11-24T06:08:57.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 shutdown_connections 2023-11-24T06:08:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ae415e160 0x7f4ae415eaf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ac806aed0 0x7f4ac806d390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ae40744d0 0x7f4ae414c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 --2- 172.21.15.97:0/4077038203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ae40afc90 0x7f4ae415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 >> 172.21.15.97:0/4077038203 conn(0x7f4ae40d0bf0 msgr2=0x7f4ae406eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.505+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 shutdown_connections 2023-11-24T06:08:57.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.506+0000 7f4aeb5f3700 1 -- 172.21.15.97:0/4077038203 wait complete. 2023-11-24T06:08:57.513 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-11-24T06:08:57.534 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:08:57.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.656+0000 7fc6360f2700 1 Processor -- start 2023-11-24T06:08:57.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.656+0000 7fc6360f2700 1 -- start start 2023-11-24T06:08:57.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc6360f2700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc63015a620 0x7fc6300622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc6360f2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63006e5e0 0x7fc63006ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc6360f2700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63015aa80 con 0x7fc6300cc540 2023-11-24T06:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc6360f2700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63006ef90 con 0x7fc63015a620 2023-11-24T06:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc6360f2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6300b2f20 con 0x7fc63006e5e0 2023-11-24T06:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc6300cc540 0x7fc6300ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37384/0 (socket says 172.21.15.97:37384) 2023-11-24T06:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc63015a620 0x7fc6300622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62f7fe700 1 -- 172.21.15.97:0/2455119097 learned_addr learned my addr 172.21.15.97:0/2455119097 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:57.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62ffff700 1 --2- 172.21.15.97:0/2455119097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63006e5e0 0x7fc63006ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62effd700 1 -- 172.21.15.97:0/2455119097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63006e5e0 msgr2=0x7fc63006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62effd700 1 --2- 172.21.15.97:0/2455119097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63006e5e0 0x7fc63006ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62effd700 1 -- 172.21.15.97:0/2455119097 >> v1:172.21.15.97:6789/0 conn(0x7fc6300cc540 legacy=0x7fc6300ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62e7fc700 1 -- 172.21.15.97:0/2455119097 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218381339 0 0) 0x7fc63015aa80 con 0x7fc6300cc540 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62effd700 1 -- 172.21.15.97:0/2455119097 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6300ce960 con 0x7fc63015a620 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.657+0000 7fc62ffff700 1 --2- 172.21.15.97:0/2455119097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63006e5e0 0x7fc63006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc62effd700 1 --2- 172.21.15.97:0/2455119097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015a620 0x7fc6300622a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc620009840 tx=0x7fc620009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ddbb38a8e85cb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc62e7fc700 1 -- 172.21.15.97:0/2455119097 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620011830 con 0x7fc63015a620 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc62e7fc700 1 -- 172.21.15.97:0/2455119097 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc620011990 con 0x7fc63015a620 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc62e7fc700 1 -- 172.21.15.97:0/2455119097 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc620011d20 con 0x7fc63015a620 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc6360f2700 1 -- 172.21.15.97:0/2455119097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015a620 msgr2=0x7fc6300622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:57.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc6360f2700 1 --2- 172.21.15.97:0/2455119097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015a620 0x7fc6300622a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc620009840 tx=0x7fc620009b50 comp rx=0 tx=0).stop 2023-11-24T06:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc6360f2700 1 -- 172.21.15.97:0/2455119097 shutdown_connections 2023-11-24T06:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc6360f2700 1 --2- 172.21.15.97:0/2455119097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63006e5e0 0x7fc63006ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc6360f2700 1 --2- 172.21.15.97:0/2455119097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63015a620 0x7fc6300622a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc6360f2700 1 -- 172.21.15.97:0/2455119097 >> 172.21.15.97:0/2455119097 conn(0x7fc6300b2050 msgr2=0x7fc6300b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc6360f2700 1 -- 172.21.15.97:0/2455119097 shutdown_connections 2023-11-24T06:08:57.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.658+0000 7fc6360f2700 1 -- 172.21.15.97:0/2455119097 wait complete. 2023-11-24T06:08:57.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.659+0000 7fc6360f2700 1 Processor -- start 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc6360f2700 1 -- start start 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc6360f2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63006e5e0 0x7fc6301e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc6360f2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6300cc540 0x7fc63014e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc6360f2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63015a620 0x7fc63015c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc6360f2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6301503f0 con 0x7fc63006e5e0 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc6360f2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6301e3ce0 con 0x7fc6300cc540 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc6360f2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6301e3fc0 con 0x7fc63015a620 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc62f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63006e5e0 0x7fc6301e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc62f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63006e5e0 0x7fc6301e4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44938/0 (socket says 172.21.15.97:44938) 2023-11-24T06:08:57.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.660+0000 7fc62f7fe700 1 -- 172.21.15.97:0/3731015598 learned_addr learned my addr 172.21.15.97:0/3731015598 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62effd700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6300cc540 0x7fc63014e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62ffff700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63015a620 0x7fc63015c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62f7fe700 1 -- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63015a620 msgr2=0x7fc63015c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62f7fe700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63015a620 0x7fc63015c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62f7fe700 1 -- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6300cc540 msgr2=0x7fc63014e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62f7fe700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6300cc540 0x7fc63014e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62f7fe700 1 -- 172.21.15.97:0/3731015598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6200094a0 con 0x7fc63006e5e0 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62ffff700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63015a620 0x7fc63015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62effd700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6300cc540 0x7fc63014e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc62f7fe700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63006e5e0 0x7fc6301e4aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6240011e0 tx=0x7fc62400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc617fff700 1 -- 172.21.15.97:0/3731015598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624004c80 con 0x7fc63006e5e0 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc617fff700 1 -- 172.21.15.97:0/3731015598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc624004e00 con 0x7fc63006e5e0 2023-11-24T06:08:57.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6301e42a0 con 0x7fc63006e5e0 2023-11-24T06:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc617fff700 1 -- 172.21.15.97:0/3731015598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624007310 con 0x7fc63006e5e0 2023-11-24T06:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.661+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63015d1e0 con 0x7fc63006e5e0 2023-11-24T06:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.662+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc630068d90 con 0x7fc63006e5e0 2023-11-24T06:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.663+0000 7fc617fff700 1 -- 172.21.15.97:0/3731015598 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc624007470 con 0x7fc63006e5e0 2023-11-24T06:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.663+0000 7fc617fff700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc61c06aed0 0x7fc61c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.663+0000 7fc617fff700 1 -- 172.21.15.97:0/3731015598 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc624089320 con 0x7fc63006e5e0 2023-11-24T06:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.663+0000 7fc62effd700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc61c06aed0 0x7fc61c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:57.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.663+0000 7fc62effd700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc61c06aed0 0x7fc61c06d390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6200090b0 tx=0x7fc620008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:57.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.668+0000 7fc617fff700 1 -- 172.21.15.97:0/3731015598 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc624054430 con 0x7fc63006e5e0 2023-11-24T06:08:57.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:57.901+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc6301598a0 con 0x7fc63006e5e0 2023-11-24T06:08:58.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.513+0000 7fc617fff700 1 -- 172.21.15.97:0/3731015598 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc624057a50 con 0x7fc63006e5e0 2023-11-24T06:08:58.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.517+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc61c06aed0 msgr2=0x7fc61c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.517+0000 7fc6360f2700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc61c06aed0 0x7fc61c06d390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc6200090b0 tx=0x7fc620008030 comp rx=0 tx=0).stop 2023-11-24T06:08:58.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.517+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63006e5e0 msgr2=0x7fc6301e4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.517+0000 7fc6360f2700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63006e5e0 0x7fc6301e4aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6240011e0 tx=0x7fc62400cae0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.518+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 shutdown_connections 2023-11-24T06:08:58.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.518+0000 7fc6360f2700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc61c06aed0 0x7fc61c06d390 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.518+0000 7fc6360f2700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc63015a620 0x7fc63015c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.518+0000 7fc6360f2700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc63006e5e0 0x7fc6301e4aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.518+0000 7fc6360f2700 1 --2- 172.21.15.97:0/3731015598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6300cc540 0x7fc63014e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.518+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 >> 172.21.15.97:0/3731015598 conn(0x7fc6300b2050 msgr2=0x7fc6300b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:58.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.518+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 shutdown_connections 2023-11-24T06:08:58.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.518+0000 7fc6360f2700 1 -- 172.21.15.97:0/3731015598 wait complete. 2023-11-24T06:08:58.522 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T06:08:58.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:08:58.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.662+0000 7f9f657ed700 1 Processor -- start 2023-11-24T06:08:58.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.662+0000 7f9f657ed700 1 -- start start 2023-11-24T06:08:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.662+0000 7f9f657ed700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f6015e1e0 0x7f9f600d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.662+0000 7f9f657ed700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f600d20b0 con 0x7f9f600d0f80 2023-11-24T06:08:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f657ed700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f600d2390 con 0x7f9f600d3a00 2023-11-24T06:08:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f657ed700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600d3150 con 0x7f9f6015e1e0 2023-11-24T06:08:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9f600d0f80 0x7f9f600b6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37402/0 (socket says 172.21.15.97:37402) 2023-11-24T06:08:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5f7fe700 1 -- 172.21.15.97:0/3673753425 learned_addr learned my addr 172.21.15.97:0/3673753425 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:08:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5effd700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f6015e1e0 0x7f9f600d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:58.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981434618 0 0) 0x7f9f600d20b0 con 0x7f9f600d0f80 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f480035b0 con 0x7f9f600d0f80 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565123857 0 0) 0x7f9f600d2390 con 0x7f9f600d3a00 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f600d20b0 con 0x7f9f600d3a00 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1481324138 0 0) 0x7f9f480035b0 con 0x7f9f600d0f80 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f6015e1e0 msgr2=0x7f9f600d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f6015e1e0 0x7f9f600d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 >> v1:172.21.15.183:6789/0 conn(0x7f9f600d3a00 legacy=0x7f9f600d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600d2670 con 0x7f9f600d0f80 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f9f580019f0 con 0x7f9f600d0f80 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 >> v1:172.21.15.97:6789/0 conn(0x7f9f600d0f80 legacy=0x7f9f600b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f48003bc0 0x7f9f48003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f48004850 0x7f9f48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48005210 0x7f9f4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600d3150 con 0x7f9f48005210 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4800bf80 con 0x7f9f48003bc0 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4800c140 con 0x7f9f48004850 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5effd700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f6015e1e0 0x7f9f600d34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5f7fe700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48005210 0x7f9f4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5e7fc700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f48004850 0x7f9f48004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.663+0000 7f9f5effd700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f48003bc0 0x7f9f48003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5f7fe700 1 -- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f48004850 msgr2=0x7f9f48004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5f7fe700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f48004850 0x7f9f48004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5f7fe700 1 -- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f48003bc0 msgr2=0x7f9f48003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:58.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5f7fe700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f48003bc0 0x7f9f48003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5f7fe700 1 -- 172.21.15.97:0/3673753425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480035b0 con 0x7f9f48005210 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5f7fe700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48005210 0x7f9f4800ba10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f6008d140 tx=0x7f9f48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243d3df9506afd72 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580072d0 con 0x7f9f48005210 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f58007450 con 0x7f9f48005210 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5dffb700 1 -- 172.21.15.97:0/3673753425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58007800 con 0x7f9f48005210 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f657ed700 1 -- 172.21.15.97:0/3673753425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48005210 msgr2=0x7f9f4800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f657ed700 1 --2- 172.21.15.97:0/3673753425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f48005210 0x7f9f4800ba10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f6008d140 tx=0x7f9f48005880 comp rx=0 tx=0).stop 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f5f7fe700 1 -- 172.21.15.97:0/3673753425 reap_dead start 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f657ed700 1 -- 172.21.15.97:0/3673753425 shutdown_connections 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f657ed700 1 -- 172.21.15.97:0/3673753425 >> 172.21.15.97:0/3673753425 conn(0x7f9f600b5b50 msgr2=0x7f9f600b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:58.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.664+0000 7f9f657ed700 1 -- 172.21.15.97:0/3673753425 shutdown_connections 2023-11-24T06:08:58.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.665+0000 7f9f657ed700 1 -- 172.21.15.97:0/3673753425 wait complete. 2023-11-24T06:08:58.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.666+0000 7f9f657ed700 1 Processor -- start 2023-11-24T06:08:58.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.666+0000 7f9f657ed700 1 -- start start 2023-11-24T06:08:58.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f657ed700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f60151bb0 0x7f9f600caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:58.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f657ed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60160960 0x7f9f600b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:58.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f657ed700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60156970 0x7f9f600b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:58.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f657ed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600cc9f0 con 0x7f9f60160960 2023-11-24T06:08:58.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f657ed700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600cb8a0 con 0x7f9f60156970 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f657ed700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600cbb80 con 0x7f9f60151bb0 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60160960 0x7f9f600b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60160960 0x7f9f600b7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44960/0 (socket says 172.21.15.97:44960) 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5e7fc700 1 -- 172.21.15.97:0/60455980 learned_addr learned my addr 172.21.15.97:0/60455980 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5effd700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f60151bb0 0x7f9f600caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5f7fe700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60156970 0x7f9f600b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5e7fc700 1 -- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f60151bb0 msgr2=0x7f9f600caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5e7fc700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f60151bb0 0x7f9f600caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5e7fc700 1 -- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60156970 msgr2=0x7f9f600b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5e7fc700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60156970 0x7f9f600b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.667+0000 7f9f5e7fc700 1 -- 172.21.15.97:0/60455980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600cbe60 con 0x7f9f60160960 2023-11-24T06:08:58.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.668+0000 7f9f5effd700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f60151bb0 0x7f9f600caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.668+0000 7f9f5e7fc700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60160960 0x7f9f600b7970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400f7b0 tx=0x7f9f5400fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.668+0000 7f9f477fe700 1 -- 172.21.15.97:0/60455980 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540052e0 con 0x7f9f60160960 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.668+0000 7f9f477fe700 1 -- 172.21.15.97:0/60455980 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54005460 con 0x7f9f60160960 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.668+0000 7f9f477fe700 1 -- 172.21.15.97:0/60455980 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540038c0 con 0x7f9f60160960 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.668+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f600cc140 con 0x7f9f60160960 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.668+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f600cb470 con 0x7f9f60160960 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.669+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f600751e0 con 0x7f9f60160960 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.669+0000 7f9f477fe700 1 -- 172.21.15.97:0/60455980 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f54003a20 con 0x7f9f60160960 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.669+0000 7f9f477fe700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f4c06aed0 0x7f9f4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.670+0000 7f9f477fe700 1 -- 172.21.15.97:0/60455980 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f540894a0 con 0x7f9f60160960 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.670+0000 7f9f5effd700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f4c06aed0 0x7f9f4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:58.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.670+0000 7f9f5effd700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f4c06aed0 0x7f9f4c06d390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f600b8600 tx=0x7f9f50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:58.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.674+0000 7f9f477fe700 1 -- 172.21.15.97:0/60455980 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f540545e0 con 0x7f9f60160960 2023-11-24T06:08:58.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.866+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f600ccb30 con 0x7f9f60160960 2023-11-24T06:08:58.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.867+0000 7f9f477fe700 1 -- 172.21.15.97:0/60455980 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16530 (secure 0 0 0) 0x7f9f600ccb30 con 0x7f9f60160960 2023-11-24T06:08:58.870 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:08:58.871 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":19,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:57.902193+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:08:58.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f4c06aed0 msgr2=0x7f9f4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f4c06aed0 0x7f9f4c06d390 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f600b8600 tx=0x7f9f50008040 comp rx=0 tx=0).stop 2023-11-24T06:08:58.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60160960 msgr2=0x7f9f600b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60160960 0x7f9f600b7970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400f7b0 tx=0x7f9f5400fb70 comp rx=0 tx=0).stop 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 shutdown_connections 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f60156970 0x7f9f600b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f4c06aed0 0x7f9f4c06d390 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f60151bb0 0x7f9f600caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 --2- 172.21.15.97:0/60455980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f60160960 0x7f9f600b7970 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 >> 172.21.15.97:0/60455980 conn(0x7f9f600b5b50 msgr2=0x7f9f6006b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 shutdown_connections 2023-11-24T06:08:58.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:58.869+0000 7f9f657ed700 1 -- 172.21.15.97:0/60455980 wait complete. 2023-11-24T06:08:58.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:08:59.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.000+0000 7f5da661d700 1 Processor -- start 2023-11-24T06:08:59.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.000+0000 7f5da661d700 1 -- start start 2023-11-24T06:08:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.001+0000 7f5da661d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5da015bdd0 0x7f5da01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.001+0000 7f5da661d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5da00b3370 0x7f5da00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.001+0000 7f5da661d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da00b2030 con 0x7f5da00d1c70 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.001+0000 7f5da661d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da00b2f50 con 0x7f5da015bdd0 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.001+0000 7f5da661d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da00b22d0 con 0x7f5da00b3370 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.001+0000 7f5d9ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5da015bdd0 0x7f5da01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.001+0000 7f5d9ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5da015bdd0 0x7f5da01604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44966/0 (socket says 172.21.15.97:44966) 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.001+0000 7f5d9ffff700 1 -- 172.21.15.97:0/3673732482 learned_addr learned my addr 172.21.15.97:0/3673732482 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9f7fe700 1 --2- 172.21.15.97:0/3673732482 >> v2:172.21.15.183:3300/0 conn(0x7f5da00b3370 0x7f5da00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9ffff700 1 -- 172.21.15.97:0/3673732482 >> v2:172.21.15.183:3300/0 conn(0x7f5da00b3370 msgr2=0x7f5da00b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9ffff700 1 --2- 172.21.15.97:0/3673732482 >> v2:172.21.15.183:3300/0 conn(0x7f5da00b3370 0x7f5da00b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9ffff700 1 -- 172.21.15.97:0/3673732482 >> v1:172.21.15.183:6789/0 conn(0x7f5da00d1c70 legacy=0x7f5da00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9ffff700 1 -- 172.21.15.97:0/3673732482 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00b25b0 con 0x7f5da015bdd0 2023-11-24T06:08:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9ffff700 1 --2- 172.21.15.97:0/3673732482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da015bdd0 0x7f5da01604e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5da00714c0 tx=0x7f5d90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839250f3e7ba5363 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9effd700 1 -- 172.21.15.97:0/3673732482 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90011830 con 0x7f5da015bdd0 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9effd700 1 -- 172.21.15.97:0/3673732482 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d90011990 con 0x7f5da015bdd0 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.002+0000 7f5d9effd700 1 -- 172.21.15.97:0/3673732482 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90011d20 con 0x7f5da015bdd0 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.003+0000 7f5da661d700 1 -- 172.21.15.97:0/3673732482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da015bdd0 msgr2=0x7f5da01604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.003+0000 7f5da661d700 1 --2- 172.21.15.97:0/3673732482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da015bdd0 0x7f5da01604e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5da00714c0 tx=0x7f5d90009bd0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.003+0000 7f5da661d700 1 -- 172.21.15.97:0/3673732482 shutdown_connections 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.003+0000 7f5da661d700 1 --2- 172.21.15.97:0/3673732482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da015bdd0 0x7f5da01604e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.003+0000 7f5da661d700 1 --2- 172.21.15.97:0/3673732482 >> v2:172.21.15.183:3300/0 conn(0x7f5da00b3370 0x7f5da00b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.003+0000 7f5da661d700 1 -- 172.21.15.97:0/3673732482 >> 172.21.15.97:0/3673732482 conn(0x7f5da00731a0 msgr2=0x7f5da009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.003+0000 7f5da661d700 1 -- 172.21.15.97:0/3673732482 shutdown_connections 2023-11-24T06:08:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.003+0000 7f5da661d700 1 -- 172.21.15.97:0/3673732482 wait complete. 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.004+0000 7f5da661d700 1 Processor -- start 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.004+0000 7f5da661d700 1 -- start start 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5da661d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da00b3370 0x7f5da006c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5da661d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da00d1c70 0x7f5da00cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5da661d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015bdd0 0x7f5da0158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5da661d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da006cf30 con 0x7f5da00d1c70 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5da661d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c8db0 con 0x7f5da00b3370 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5da661d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c9090 con 0x7f5da015bdd0 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da00d1c70 0x7f5da00cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da00d1c70 0x7f5da00cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44974/0 (socket says 172.21.15.97:44974) 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 -- 172.21.15.97:0/3549157478 learned_addr learned my addr 172.21.15.97:0/3549157478 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:59.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9ffff700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da00b3370 0x7f5da006c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5da4bba700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015bdd0 0x7f5da0158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 -- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015bdd0 msgr2=0x7f5da0158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015bdd0 0x7f5da0158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 -- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da00b3370 msgr2=0x7f5da006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da00b3370 0x7f5da006c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 -- 172.21.15.97:0/3549157478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d90009550 con 0x7f5da00d1c70 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5da4bba700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015bdd0 0x7f5da0158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9f7fe700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da00d1c70 0x7f5da00cab50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e980 tx=0x7f5d8800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9cff9700 1 -- 172.21.15.97:0/3549157478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88009710 con 0x7f5da00d1c70 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.005+0000 7f5d9cff9700 1 -- 172.21.15.97:0/3549157478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8800eed0 con 0x7f5da00d1c70 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.006+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da00c9370 con 0x7f5da00d1c70 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.006+0000 7f5d9cff9700 1 -- 172.21.15.97:0/3549157478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800ca90 con 0x7f5da00d1c70 2023-11-24T06:08:59.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.006+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da00c9930 con 0x7f5da00d1c70 2023-11-24T06:08:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.006+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da01f3ef0 con 0x7f5da00d1c70 2023-11-24T06:08:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.007+0000 7f5d9cff9700 1 -- 172.21.15.97:0/3549157478 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d8800cbf0 con 0x7f5da00d1c70 2023-11-24T06:08:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.007+0000 7f5d9cff9700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d8006aed0 0x7f5d8006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.007+0000 7f5d9cff9700 1 -- 172.21.15.97:0/3549157478 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d880895b0 con 0x7f5da00d1c70 2023-11-24T06:08:59.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.007+0000 7f5d9ffff700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d8006aed0 0x7f5d8006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.008+0000 7f5d9ffff700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d8006aed0 0x7f5d8006d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d90014040 tx=0x7f5d90008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:59.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.012+0000 7f5d9cff9700 1 -- 172.21.15.97:0/3549157478 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d880546f0 con 0x7f5da00d1c70 2023-11-24T06:08:59.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.202+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5da006d070 con 0x7f5da00d1c70 2023-11-24T06:08:59.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.204+0000 7f5d9cff9700 1 -- 172.21.15.97:0/3549157478 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16530 (secure 0 0 0) 0x7f5d88057d10 con 0x7f5da00d1c70 2023-11-24T06:08:59.207 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:08:59.207 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":19,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:57.902193+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.205+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d8006aed0 msgr2=0x7f5d8006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.205+0000 7f5da661d700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d8006aed0 0x7f5d8006d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d90014040 tx=0x7f5d90008030 comp rx=0 tx=0).stop 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.205+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da00d1c70 msgr2=0x7f5da00cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.205+0000 7f5da661d700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da00d1c70 0x7f5da00cab50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e980 tx=0x7f5d8800ec90 comp rx=0 tx=0).stop 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.206+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 shutdown_connections 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.206+0000 7f5da661d700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5d8006aed0 0x7f5d8006d390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.206+0000 7f5da661d700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5da015bdd0 0x7f5da0158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.206+0000 7f5da661d700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5da00b3370 0x7f5da006c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.206+0000 7f5da661d700 1 --2- 172.21.15.97:0/3549157478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5da00d1c70 0x7f5da00cab50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.206+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 >> 172.21.15.97:0/3549157478 conn(0x7f5da00731a0 msgr2=0x7f5da00b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.206+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 shutdown_connections 2023-11-24T06:08:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.206+0000 7f5da661d700 1 -- 172.21.15.97:0/3549157478 wait complete. 2023-11-24T06:08:59.221 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:08:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.342+0000 7f1d957b5700 1 Processor -- start 2023-11-24T06:08:59.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.342+0000 7f1d957b5700 1 -- start start 2023-11-24T06:08:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.343+0000 7f1d957b5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1d9015e160 0x7f1d900d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.343+0000 7f1d957b5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d900b1fd0 0x7f1d9006b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.343+0000 7f1d957b5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900cfe20 con 0x7f1d900d20d0 2023-11-24T06:08:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.343+0000 7f1d957b5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900d2550 con 0x7f1d900b1fd0 2023-11-24T06:08:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.343+0000 7f1d957b5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900b2bb0 con 0x7f1d9015e160 2023-11-24T06:08:59.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.343+0000 7f1d8f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d900b1fd0 0x7f1d9006b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.343+0000 7f1d8f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d900b1fd0 0x7f1d9006b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44984/0 (socket says 172.21.15.97:44984) 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.343+0000 7f1d8f7fe700 1 -- 172.21.15.97:0/1965813389 learned_addr learned my addr 172.21.15.97:0/1965813389 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8effd700 1 --2- 172.21.15.97:0/1965813389 >> v2:172.21.15.183:3300/0 conn(0x7f1d9015e160 0x7f1d900d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8f7fe700 1 -- 172.21.15.97:0/1965813389 >> v2:172.21.15.183:3300/0 conn(0x7f1d9015e160 msgr2=0x7f1d900d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8f7fe700 1 --2- 172.21.15.97:0/1965813389 >> v2:172.21.15.183:3300/0 conn(0x7f1d9015e160 0x7f1d900d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8f7fe700 1 -- 172.21.15.97:0/1965813389 >> v1:172.21.15.183:6789/0 conn(0x7f1d900d20d0 legacy=0x7f1d900b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8f7fe700 1 -- 172.21.15.97:0/1965813389 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9015e540 con 0x7f1d900b1fd0 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8f7fe700 1 --2- 172.21.15.97:0/1965813389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900b1fd0 0x7f1d9006b4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d900730c0 tx=0x7f1d8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b65aaa431b64147 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8e7fc700 1 -- 172.21.15.97:0/1965813389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800e070 con 0x7f1d900b1fd0 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8e7fc700 1 -- 172.21.15.97:0/1965813389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8800b9c0 con 0x7f1d900b1fd0 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.344+0000 7f1d8e7fc700 1 -- 172.21.15.97:0/1965813389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8800bd70 con 0x7f1d900b1fd0 2023-11-24T06:08:59.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.345+0000 7f1d957b5700 1 -- 172.21.15.97:0/1965813389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900b1fd0 msgr2=0x7f1d9006b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.345+0000 7f1d957b5700 1 --2- 172.21.15.97:0/1965813389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900b1fd0 0x7f1d9006b4c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d900730c0 tx=0x7f1d8800b330 comp rx=0 tx=0).stop 2023-11-24T06:08:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.345+0000 7f1d957b5700 1 -- 172.21.15.97:0/1965813389 shutdown_connections 2023-11-24T06:08:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.345+0000 7f1d957b5700 1 --2- 172.21.15.97:0/1965813389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900b1fd0 0x7f1d9006b4c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.345+0000 7f1d957b5700 1 --2- 172.21.15.97:0/1965813389 >> v2:172.21.15.183:3300/0 conn(0x7f1d9015e160 0x7f1d900d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.345+0000 7f1d957b5700 1 -- 172.21.15.97:0/1965813389 >> 172.21.15.97:0/1965813389 conn(0x7f1d90073130 msgr2=0x7f1d9009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:08:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.345+0000 7f1d957b5700 1 -- 172.21.15.97:0/1965813389 shutdown_connections 2023-11-24T06:08:59.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.345+0000 7f1d957b5700 1 -- 172.21.15.97:0/1965813389 wait complete. 2023-11-24T06:08:59.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.346+0000 7f1d957b5700 1 Processor -- start 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.346+0000 7f1d957b5700 1 -- start start 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d957b5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d900b1fd0 0x7f1d90152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d957b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900d20d0 0x7f1d90160df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d957b5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d9015e160 0x7f1d9015f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d957b5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900c8d60 con 0x7f1d900d20d0 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d957b5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900c8fb0 con 0x7f1d900b1fd0 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d957b5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900c9290 con 0x7f1d9015e160 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900d20d0 0x7f1d90160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900d20d0 0x7f1d90160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44988/0 (socket says 172.21.15.97:44988) 2023-11-24T06:08:59.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 -- 172.21.15.97:0/2853534485 learned_addr learned my addr 172.21.15.97:0/2853534485 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d8f7fe700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d9015e160 0x7f1d9015f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d8effd700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d900b1fd0 0x7f1d90152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 -- 172.21.15.97:0/2853534485 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d9015e160 msgr2=0x7f1d9015f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d9015e160 0x7f1d9015f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 -- 172.21.15.97:0/2853534485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d900b1fd0 msgr2=0x7f1d90152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d900b1fd0 0x7f1d90152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 -- 172.21.15.97:0/2853534485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d88009b30 con 0x7f1d900d20d0 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d87fff700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900d20d0 0x7f1d90160df0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c4e0 tx=0x7f1d7800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d877fe700 1 -- 172.21.15.97:0/2853534485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780052d0 con 0x7f1d900d20d0 2023-11-24T06:08:59.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d877fe700 1 -- 172.21.15.97:0/2853534485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78005430 con 0x7f1d900d20d0 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9015b6f0 con 0x7f1d900d20d0 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d877fe700 1 -- 172.21.15.97:0/2853534485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d780038c0 con 0x7f1d900d20d0 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.347+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d901534e0 con 0x7f1d900d20d0 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.348+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d90068fb0 con 0x7f1d900d20d0 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.349+0000 7f1d877fe700 1 -- 172.21.15.97:0/2853534485 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d78003a20 con 0x7f1d900d20d0 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.349+0000 7f1d877fe700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d7006aed0 0x7f1d7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.349+0000 7f1d8effd700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d7006aed0 0x7f1d7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.349+0000 7f1d877fe700 1 -- 172.21.15.97:0/2853534485 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d78088a20 con 0x7f1d900d20d0 2023-11-24T06:08:59.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.349+0000 7f1d8effd700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d7006aed0 0x7f1d7006d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009930 tx=0x7f1d80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:08:59.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.353+0000 7f1d877fe700 1 -- 172.21.15.97:0/2853534485 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d78053b60 con 0x7f1d900d20d0 2023-11-24T06:08:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:08:59.588+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1d900b7980 con 0x7f1d900d20d0 2023-11-24T06:09:00.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.528+0000 7f1d877fe700 1 -- 172.21.15.97:0/2853534485 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1d780170b0 con 0x7f1d900d20d0 2023-11-24T06:09:00.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d7006aed0 msgr2=0x7f1d7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:00.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d7006aed0 0x7f1d7006d390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009930 tx=0x7f1d80008040 comp rx=0 tx=0).stop 2023-11-24T06:09:00.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900d20d0 msgr2=0x7f1d90160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:00.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900d20d0 0x7f1d90160df0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800c4e0 tx=0x7f1d7800c8a0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 shutdown_connections 2023-11-24T06:09:00.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d7006aed0 0x7f1d7006d390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d9015e160 0x7f1d9015f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d900b1fd0 0x7f1d90152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 --2- 172.21.15.97:0/2853534485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d900d20d0 0x7f1d90160df0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 >> 172.21.15.97:0/2853534485 conn(0x7f1d90073130 msgr2=0x7f1d900b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:00.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 shutdown_connections 2023-11-24T06:09:00.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.531+0000 7f1d957b5700 1 -- 172.21.15.97:0/2853534485 wait complete. 2023-11-24T06:09:00.548 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:09:00.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:09:00.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.674+0000 7efcc79b5700 1 Processor -- start 2023-11-24T06:09:00.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.674+0000 7efcc79b5700 1 -- start start 2023-11-24T06:09:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcc79b5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efcc015e160 0x7efcc00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcc79b5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc00b3c60 con 0x7efcc00746a0 2023-11-24T06:09:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcc79b5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc00b58c0 con 0x7efcc00afc90 2023-11-24T06:09:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcc79b5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc00b4500 con 0x7efcc015e160 2023-11-24T06:09:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcc5f52700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efcc00746a0 0x7efcc006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37450/0 (socket says 172.21.15.97:37450) 2023-11-24T06:09:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcc5f52700 1 -- 172.21.15.97:0/1379653703 learned_addr learned my addr 172.21.15.97:0/1379653703 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcc5751700 1 --2- 172.21.15.97:0/1379653703 >> v2:172.21.15.183:3300/0 conn(0x7efcc015e160 0x7efcc00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076813549 0 0) 0x7efcc00b3c60 con 0x7efcc00746a0 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcac0035b0 con 0x7efcc00746a0 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1485851256 0 0) 0x7efcac0035b0 con 0x7efcc00746a0 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 >> v2:172.21.15.183:3300/0 conn(0x7efcc015e160 msgr2=0x7efcc00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcb7fff700 1 --2- 172.21.15.97:0/1379653703 >> v2:172.21.15.183:3300/0 conn(0x7efcc015e160 0x7efcc00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 >> v1:172.21.15.183:6789/0 conn(0x7efcc00afc90 legacy=0x7efcc0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc00b0140 con 0x7efcc00746a0 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.675+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7efcb80019f0 con 0x7efcc00746a0 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 >> v1:172.21.15.97:6789/0 conn(0x7efcc00746a0 legacy=0x7efcc006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcac003a00 0x7efcac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcac004850 0x7efcac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00b4500 con 0x7efcac005210 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac00c0b0 con 0x7efcac003a00 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac00c270 con 0x7efcac004850 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5f52700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5751700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcac003a00 0x7efcac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc4f50700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcac004850 0x7efcac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5f52700 1 -- 172.21.15.97:0/1379653703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcac004850 msgr2=0x7efcac004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5f52700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcac004850 0x7efcac004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5f52700 1 -- 172.21.15.97:0/1379653703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcac003a00 msgr2=0x7efcac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5f52700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcac003a00 0x7efcac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5f52700 1 -- 172.21.15.97:0/1379653703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0035b0 con 0x7efcac005210 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5f52700 1 -- 172.21.15.97:0/1379653703 reap_dead start 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcc5f52700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcac00ba10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efcc00d1b40 tx=0x7efcac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eafbee36da40b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb800a2f0 con 0x7efcac005210 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb8012600 con 0x7efcac005210 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.676+0000 7efcb7fff700 1 -- 172.21.15.97:0/1379653703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb80129b0 con 0x7efcac005210 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.677+0000 7efcc79b5700 1 -- 172.21.15.97:0/1379653703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 msgr2=0x7efcac00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.677+0000 7efcc79b5700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcac00ba10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efcc00d1b40 tx=0x7efcac005880 comp rx=0 tx=0).stop 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.677+0000 7efcc79b5700 1 -- 172.21.15.97:0/1379653703 shutdown_connections 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.677+0000 7efcc79b5700 1 --2- 172.21.15.97:0/1379653703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcac00ba10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.677+0000 7efcc79b5700 1 -- 172.21.15.97:0/1379653703 >> 172.21.15.97:0/1379653703 conn(0x7efcc00d0bf0 msgr2=0x7efcc00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.677+0000 7efcc79b5700 1 -- 172.21.15.97:0/1379653703 shutdown_connections 2023-11-24T06:09:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.677+0000 7efcc79b5700 1 -- 172.21.15.97:0/1379653703 wait complete. 2023-11-24T06:09:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.678+0000 7efcc79b5700 1 Processor -- start 2023-11-24T06:09:00.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.678+0000 7efcc79b5700 1 -- start start 2023-11-24T06:09:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc79b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcc0158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc79b5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc00c6c50 0x7efcc015e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:00.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc79b5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcc00c7650 0x7efcc00c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc79b5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00c8040 con 0x7efcac005210 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc79b5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc015f020 con 0x7efcc00c6c50 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc79b5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc015f300 con 0x7efcc00c7650 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc5751700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcc0158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc5751700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcc0158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45006/0 (socket says 172.21.15.97:45006) 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc5751700 1 -- 172.21.15.97:0/417494813 learned_addr learned my addr 172.21.15.97:0/417494813 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.679+0000 7efcc4f50700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc00c6c50 0x7efcc015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc5f52700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcc00c7650 0x7efcc00c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc5751700 1 -- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcc00c7650 msgr2=0x7efcc00c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc5751700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcc00c7650 0x7efcc00c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc5751700 1 -- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc00c6c50 msgr2=0x7efcc015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc5751700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc00c6c50 0x7efcc015e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc5751700 1 -- 172.21.15.97:0/417494813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc015f5e0 con 0x7efcac005210 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc4f50700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc00c6c50 0x7efcc015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc5751700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcc0158860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0119a0 tx=0x7efcbc011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcb5ffb700 1 -- 172.21.15.97:0/417494813 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00f940 con 0x7efcac005210 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcb5ffb700 1 -- 172.21.15.97:0/417494813 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc00faa0 con 0x7efcac005210 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc015f8c0 con 0x7efcac005210 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcb5ffb700 1 -- 172.21.15.97:0/417494813 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00fe30 con 0x7efcac005210 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.680+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc0158da0 con 0x7efcac005210 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.681+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc0068fb0 con 0x7efcac005210 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.681+0000 7efcb5ffb700 1 -- 172.21.15.97:0/417494813 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcbc014440 con 0x7efcac005210 2023-11-24T06:09:00.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.682+0000 7efcb5ffb700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efca406aed0 0x7efca406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.682+0000 7efcb5ffb700 1 -- 172.21.15.97:0/417494813 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcbc089d20 con 0x7efcac005210 2023-11-24T06:09:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.682+0000 7efcc4f50700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efca406aed0 0x7efca406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.682+0000 7efcc4f50700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efca406aed0 0x7efca406d390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcc00c7090 tx=0x7efcb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.686+0000 7efcb5ffb700 1 -- 172.21.15.97:0/417494813 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efcbc054dc0 con 0x7efcac005210 2023-11-24T06:09:00.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:00.920+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efcc015d100 con 0x7efcac005210 2023-11-24T06:09:01.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.529+0000 7efcb5ffb700 1 -- 172.21.15.97:0/417494813 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7efcbc0583e0 con 0x7efcac005210 2023-11-24T06:09:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efca406aed0 msgr2=0x7efca406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efca406aed0 0x7efca406d390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efcc00c7090 tx=0x7efcb0009040 comp rx=0 tx=0).stop 2023-11-24T06:09:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 msgr2=0x7efcc0158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcc0158860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0119a0 tx=0x7efcbc011d60 comp rx=0 tx=0).stop 2023-11-24T06:09:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 shutdown_connections 2023-11-24T06:09:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efca406aed0 0x7efca406d390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efcc00c7650 0x7efcc00c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcac005210 0x7efcc0158860 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 --2- 172.21.15.97:0/417494813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcc00c6c50 0x7efcc015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 >> 172.21.15.97:0/417494813 conn(0x7efcc00d0bf0 msgr2=0x7efcc006e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:01.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 shutdown_connections 2023-11-24T06:09:01.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.532+0000 7efcc79b5700 1 -- 172.21.15.97:0/417494813 wait complete. 2023-11-24T06:09:01.549 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:09:01.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:01.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.672+0000 7f69c2f70700 1 Processor -- start 2023-11-24T06:09:01.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.672+0000 7f69c2f70700 1 -- start start 2023-11-24T06:09:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c2f70700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f69bc15ea60 0x7f69bc15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c2f70700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc06c000 0x7f69bc06c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:01.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c2f70700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69bc06c980 con 0x7f69bc144660 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c2f70700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69bc0b4230 con 0x7f69bc15ea60 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c2f70700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc0b3c40 con 0x7f69bc06c000 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c150d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc06c000 0x7f69bc06c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69bbfff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f69bc15ea60 0x7f69bc15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69bbfff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f69bc15ea60 0x7f69bc15eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39176/0 (socket says 172.21.15.97:39176) 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c150d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc06c000 0x7f69bc06c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60286/0 (socket says 172.21.15.97:60286) 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c0d0c700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f69bc144660 0x7f69bc0b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46788/0 (socket says 172.21.15.97:46788) 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69bbfff700 1 -- 172.21.15.97:0/1762507426 learned_addr learned my addr 172.21.15.97:0/1762507426 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c150d700 1 -- 172.21.15.97:0/1762507426 >> v2:172.21.15.183:3300/0 conn(0x7f69bc15ea60 msgr2=0x7f69bc15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:01.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c150d700 1 --2- 172.21.15.97:0/1762507426 >> v2:172.21.15.183:3300/0 conn(0x7f69bc15ea60 0x7f69bc15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c150d700 1 -- 172.21.15.97:0/1762507426 >> v1:172.21.15.183:6789/0 conn(0x7f69bc144660 legacy=0x7f69bc0b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69c150d700 1 -- 172.21.15.97:0/1762507426 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc15f520 con 0x7f69bc06c000 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.673+0000 7f69bbfff700 1 --2- 172.21.15.97:0/1762507426 >> v2:172.21.15.183:3300/0 conn(0x7f69bc15ea60 0x7f69bc15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c150d700 1 --2- 172.21.15.97:0/1762507426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc06c000 0x7f69bc06c440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009d90 tx=0x7f69b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98315b1d3fe6d6af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69bb7fe700 1 -- 172.21.15.97:0/1762507426 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b400e070 con 0x7f69bc06c000 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69bb7fe700 1 -- 172.21.15.97:0/1762507426 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b400bb20 con 0x7f69bc06c000 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69bb7fe700 1 -- 172.21.15.97:0/1762507426 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4012430 con 0x7f69bc06c000 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c2f70700 1 -- 172.21.15.97:0/1762507426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc06c000 msgr2=0x7f69bc06c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c2f70700 1 --2- 172.21.15.97:0/1762507426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc06c000 0x7f69bc06c440 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009d90 tx=0x7f69b400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:01.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c2f70700 1 -- 172.21.15.97:0/1762507426 shutdown_connections 2023-11-24T06:09:01.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c2f70700 1 --2- 172.21.15.97:0/1762507426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc06c000 0x7f69bc06c440 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c2f70700 1 --2- 172.21.15.97:0/1762507426 >> v2:172.21.15.183:3300/0 conn(0x7f69bc15ea60 0x7f69bc15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c2f70700 1 -- 172.21.15.97:0/1762507426 >> 172.21.15.97:0/1762507426 conn(0x7f69bc071ac0 msgr2=0x7f69bc071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:01.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c2f70700 1 -- 172.21.15.97:0/1762507426 shutdown_connections 2023-11-24T06:09:01.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.674+0000 7f69c2f70700 1 -- 172.21.15.97:0/1762507426 wait complete. 2023-11-24T06:09:01.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.675+0000 7f69c2f70700 1 Processor -- start 2023-11-24T06:09:01.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.676+0000 7f69c2f70700 1 -- start start 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.676+0000 7f69c2f70700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69bc06c000 0x7f69bc0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.676+0000 7f69c2f70700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc144660 0x7f69bc0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.676+0000 7f69c2f70700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc15ea60 0x7f69bc157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.676+0000 7f69c2f70700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc1f3a30 con 0x7f69bc06c000 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.676+0000 7f69c2f70700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc1f3c80 con 0x7f69bc144660 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.676+0000 7f69c2f70700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc1f3f60 con 0x7f69bc15ea60 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69bc06c000 0x7f69bc0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69bc06c000 0x7f69bc0cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45020/0 (socket says 172.21.15.97:45020) 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 -- 172.21.15.97:0/439442654 learned_addr learned my addr 172.21.15.97:0/439442654 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:01.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c150d700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc15ea60 0x7f69bc157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69bbfff700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc144660 0x7f69bc0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 -- 172.21.15.97:0/439442654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc15ea60 msgr2=0x7f69bc157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc15ea60 0x7f69bc157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 -- 172.21.15.97:0/439442654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc144660 msgr2=0x7f69bc0cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc144660 0x7f69bc0cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 -- 172.21.15.97:0/439442654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a800b7f0 con 0x7f69bc06c000 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c0d0c700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69bc06c000 0x7f69bc0cf740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c520 tx=0x7f69b000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69b97fa700 1 -- 172.21.15.97:0/439442654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0005cb0 con 0x7f69bc06c000 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69b97fa700 1 -- 172.21.15.97:0/439442654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b0005e10 con 0x7f69bc06c000 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b4009a70 con 0x7f69bc06c000 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69b97fa700 1 -- 172.21.15.97:0/439442654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00072d0 con 0x7f69bc06c000 2023-11-24T06:09:01.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.677+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69bc1f4520 con 0x7f69bc06c000 2023-11-24T06:09:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.678+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69bc0b5690 con 0x7f69bc06c000 2023-11-24T06:09:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.679+0000 7f69b97fa700 1 -- 172.21.15.97:0/439442654 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69b0007430 con 0x7f69bc06c000 2023-11-24T06:09:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.679+0000 7f69b97fa700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f69a006aed0 0x7f69a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.679+0000 7f69b97fa700 1 -- 172.21.15.97:0/439442654 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69b0088840 con 0x7f69bc06c000 2023-11-24T06:09:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.679+0000 7f69bbfff700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f69a006aed0 0x7f69a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.679+0000 7f69bbfff700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f69a006aed0 0x7f69a006d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69a800b240 tx=0x7f69a800b4d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:01.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.683+0000 7f69b97fa700 1 -- 172.21.15.97:0/439442654 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69b00539a0 con 0x7f69bc06c000 2023-11-24T06:09:01.900 INFO:tasks.ceph.mds.d.smithi183.stderr:2023-11-24T06:09:01.897+0000 7ff973dcf700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:09:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.916+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69bc067f60 con 0x7f69bc06c000 2023-11-24T06:09:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.917+0000 7f69b97fa700 1 -- 172.21.15.97:0/439442654 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f69b0017020 con 0x7f69bc06c000 2023-11-24T06:09:01.920 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:01.920 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:08:57.902510+0000","modified":"2023-11-24T06:09:00.921422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:09:01.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f69a006aed0 msgr2=0x7f69a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:01.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f69a006aed0 0x7f69a006d390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69a800b240 tx=0x7f69a800b4d0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69bc06c000 msgr2=0x7f69bc0cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69bc06c000 0x7f69bc0cf740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69b000c520 tx=0x7f69b000c830 comp rx=0 tx=0).stop 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 shutdown_connections 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f69a006aed0 0x7f69a006d390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f69bc15ea60 0x7f69bc157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69bc06c000 0x7f69bc0cf740 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 --2- 172.21.15.97:0/439442654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc144660 0x7f69bc0cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 >> 172.21.15.97:0/439442654 conn(0x7f69bc071ac0 msgr2=0x7f69bc0b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 shutdown_connections 2023-11-24T06:09:01.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:01.919+0000 7f69c2f70700 1 -- 172.21.15.97:0/439442654 wait complete. 2023-11-24T06:09:01.924 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-11-24T06:09:01.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.050+0000 7f94f542c700 1 Processor -- start 2023-11-24T06:09:02.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94f542c700 1 -- start start 2023-11-24T06:09:02.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94f542c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94f015e140 0x7f94f00d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94f542c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94f542c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f00b80e0 con 0x7f94f00d39d0 2023-11-24T06:09:02.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94f542c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f015e520 con 0x7f94f015e140 2023-11-24T06:09:02.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94f542c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00d2930 con 0x7f94f00b98c0 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94eeffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94f015e140 0x7f94f00d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94ee7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f94f00d39d0 0x7f94f00b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37480/0 (socket says 172.21.15.97:37480) 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94eeffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94f015e140 0x7f94f00d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45028/0 (socket says 172.21.15.97:45028) 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.051+0000 7f94ee7fc700 1 -- 172.21.15.97:0/1034213221 learned_addr learned my addr 172.21.15.97:0/1034213221 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94ef7fe700 1 --2- 172.21.15.97:0/1034213221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94eeffd700 1 -- 172.21.15.97:0/1034213221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 msgr2=0x7f94f00d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94eeffd700 1 --2- 172.21.15.97:0/1034213221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94eeffd700 1 -- 172.21.15.97:0/1034213221 >> v1:172.21.15.97:6789/0 conn(0x7f94f00d39d0 legacy=0x7f94f00b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94eeffd700 1 -- 172.21.15.97:0/1034213221 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00d2c10 con 0x7f94f015e140 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94edffb700 1 -- 172.21.15.97:0/1034213221 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71958351 0 0) 0x7f94e4001110 con 0x7f94f00d39d0 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94ef7fe700 1 --2- 172.21.15.97:0/1034213221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:02.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94eeffd700 1 --2- 172.21.15.97:0/1034213221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00d3ec0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009840 tx=0x7f94e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b11e2635a06b575 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94edffb700 1 -- 172.21.15.97:0/1034213221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0011830 con 0x7f94f015e140 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94edffb700 1 -- 172.21.15.97:0/1034213221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e0011990 con 0x7f94f015e140 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.052+0000 7f94edffb700 1 -- 172.21.15.97:0/1034213221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0011d20 con 0x7f94f015e140 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.053+0000 7f94f542c700 1 -- 172.21.15.97:0/1034213221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 msgr2=0x7f94f00d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.053+0000 7f94f542c700 1 --2- 172.21.15.97:0/1034213221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00d3ec0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009840 tx=0x7f94e0009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.053+0000 7f94f542c700 1 -- 172.21.15.97:0/1034213221 shutdown_connections 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.053+0000 7f94f542c700 1 --2- 172.21.15.97:0/1034213221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.053+0000 7f94f542c700 1 --2- 172.21.15.97:0/1034213221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00d3ec0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.053+0000 7f94f542c700 1 -- 172.21.15.97:0/1034213221 >> 172.21.15.97:0/1034213221 conn(0x7f94f006b410 msgr2=0x7f94f006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.053+0000 7f94f542c700 1 -- 172.21.15.97:0/1034213221 shutdown_connections 2023-11-24T06:09:02.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.053+0000 7f94f542c700 1 -- 172.21.15.97:0/1034213221 wait complete. 2023-11-24T06:09:02.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.054+0000 7f94f542c700 1 Processor -- start 2023-11-24T06:09:02.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.054+0000 7f94f542c700 1 -- start start 2023-11-24T06:09:02.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94f542c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94f542c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94f00d39d0 0x7f94f00b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94f542c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94f542c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00b4ca0 con 0x7f94f015e140 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94f542c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00b5600 con 0x7f94f00d39d0 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94f542c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00c88d0 con 0x7f94f00b98c0 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94ef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94ef7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45038/0 (socket says 172.21.15.97:45038) 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.055+0000 7f94ef7fe700 1 -- 172.21.15.97:0/3780450757 learned_addr learned my addr 172.21.15.97:0/3780450757 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94eeffd700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94ee7fc700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94f00d39d0 0x7f94f00b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94ef7fe700 1 -- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 msgr2=0x7f94f00b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94ef7fe700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94ef7fe700 1 -- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94f00d39d0 msgr2=0x7f94f00b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94ef7fe700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94f00d39d0 0x7f94f00b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94ef7fe700 1 -- 172.21.15.97:0/3780450757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e00094a0 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94ee7fc700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94f00d39d0 0x7f94f00b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94ef7fe700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00c8390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f94e8007ef0 tx=0x7f94e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94d77fe700 1 -- 172.21.15.97:0/3780450757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800de60 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94d77fe700 1 -- 172.21.15.97:0/3780450757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e800f040 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f00b4300 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94d77fe700 1 -- 172.21.15.97:0/3780450757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8003c60 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.056+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f00b48c0 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.057+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f0067f60 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.057+0000 7f94d77fe700 1 -- 172.21.15.97:0/3780450757 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94e8004030 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.058+0000 7f94d77fe700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94dc06aed0 0x7f94dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.058+0000 7f94d77fe700 1 -- 172.21.15.97:0/3780450757 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94e8089740 con 0x7f94f015e140 2023-11-24T06:09:02.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.058+0000 7f94eeffd700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94dc06aed0 0x7f94dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.058+0000 7f94eeffd700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94dc06aed0 0x7f94dc06d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94e00090b0 tx=0x7f94e001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.062+0000 7f94d77fe700 1 -- 172.21.15.97:0/3780450757 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94e8054880 con 0x7f94f015e140 2023-11-24T06:09:02.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.255+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94f00b4de0 con 0x7f94f015e140 2023-11-24T06:09:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.256+0000 7f94d77fe700 1 -- 172.21.15.97:0/3780450757 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f94f00b4de0 con 0x7f94f015e140 2023-11-24T06:09:02.259 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:02.259 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94dc06aed0 msgr2=0x7f94dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94dc06aed0 0x7f94dc06d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94e00090b0 tx=0x7f94e001c040 comp rx=0 tx=0).stop 2023-11-24T06:09:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 msgr2=0x7f94f00c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00c8390 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f94e8007ef0 tx=0x7f94e800d3b0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 shutdown_connections 2023-11-24T06:09:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94f015e140 0x7f94f00c8390 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94dc06aed0 0x7f94dc06d390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f94f00b98c0 0x7f94f00b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 --2- 172.21.15.97:0/3780450757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94f00d39d0 0x7f94f00b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 >> 172.21.15.97:0/3780450757 conn(0x7f94f006b410 msgr2=0x7f94f00d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 shutdown_connections 2023-11-24T06:09:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.258+0000 7f94f542c700 1 -- 172.21.15.97:0/3780450757 wait complete. 2023-11-24T06:09:02.273 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:09:02.273 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:09:02.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.390+0000 7f24afca8700 1 Processor -- start 2023-11-24T06:09:02.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.391+0000 7f24afca8700 1 -- start start 2023-11-24T06:09:02.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.391+0000 7f24afca8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24a815e080 0x7f24a80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.391+0000 7f24afca8700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a80ae430 con 0x7f24a806c710 2023-11-24T06:09:02.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.391+0000 7f24afca8700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a80ae710 con 0x7f24a80af4d0 2023-11-24T06:09:02.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.391+0000 7f24afca8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80af0b0 con 0x7f24a815e080 2023-11-24T06:09:02.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24ae245700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f24a806c710 0x7f24a806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37498/0 (socket says 172.21.15.97:37498) 2023-11-24T06:09:02.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24ae245700 1 -- 172.21.15.97:0/2017670846 learned_addr learned my addr 172.21.15.97:0/2017670846 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:02.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24ada44700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24a815e080 0x7f24a80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865387422 0 0) 0x7f24a80ae430 con 0x7f24a806c710 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24940035b0 con 0x7f24a806c710 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2014474288 0 0) 0x7f24940035b0 con 0x7f24a806c710 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24a815e080 msgr2=0x7f24a80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24a815e080 0x7f24a80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 >> v1:172.21.15.183:6789/0 conn(0x7f24a80af4d0 legacy=0x7f24a806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a80ae9f0 con 0x7f24a806c710 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f24a00019f0 con 0x7f24a806c710 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 >> v1:172.21.15.97:6789/0 conn(0x7f24a806c710 legacy=0x7f24a806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24a815e080 0x7f2494003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2494003fb0 0x7f2494004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2494004d90 0x7f2494005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a80af0b0 con 0x7f2494003fb0 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249400baf0 con 0x7f2494004d90 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249400bcb0 con 0x7f24a815e080 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24ad243700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2494003fb0 0x7f2494004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24ada44700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24a815e080 0x7f2494003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.392+0000 7f24ae245700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2494004d90 0x7f2494005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24ad243700 1 -- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24a815e080 msgr2=0x7f2494003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24ad243700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f24a815e080 0x7f2494003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24ad243700 1 -- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2494004d90 msgr2=0x7f2494005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24ad243700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2494004d90 0x7f2494005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24ad243700 1 -- 172.21.15.97:0/2017670846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24940035b0 con 0x7f2494003fb0 2023-11-24T06:09:02.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24ad243700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2494003fb0 0x7f2494004850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24a80af950 tx=0x7f2494005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c15a0a4cf754d91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40052d0 con 0x7f2494003fb0 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a4005470 con 0x7f2494003fb0 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24aca42700 1 -- 172.21.15.97:0/2017670846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40038c0 con 0x7f2494003fb0 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24afca8700 1 -- 172.21.15.97:0/2017670846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2494003fb0 msgr2=0x7f2494004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24afca8700 1 --2- 172.21.15.97:0/2017670846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2494003fb0 0x7f2494004850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24a80af950 tx=0x7f2494005880 comp rx=0 tx=0).stop 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24ae245700 1 -- 172.21.15.97:0/2017670846 reap_dead start 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24afca8700 1 -- 172.21.15.97:0/2017670846 shutdown_connections 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24afca8700 1 -- 172.21.15.97:0/2017670846 >> 172.21.15.97:0/2017670846 conn(0x7f24a8071b90 msgr2=0x7f24a8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24afca8700 1 -- 172.21.15.97:0/2017670846 shutdown_connections 2023-11-24T06:09:02.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.393+0000 7f24afca8700 1 -- 172.21.15.97:0/2017670846 wait complete. 2023-11-24T06:09:02.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.395+0000 7f24afca8700 1 Processor -- start 2023-11-24T06:09:02.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.395+0000 7f24afca8700 1 -- start start 2023-11-24T06:09:02.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24afca8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2494003fb0 0x7f24a81f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24afca8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24a8155050 0x7f24a81504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24afca8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24a8150a30 0x7f24a81510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24afca8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a81f3530 con 0x7f24a8155050 2023-11-24T06:09:02.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24afca8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8153fb0 con 0x7f24a8150a30 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24afca8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8154290 con 0x7f2494003fb0 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24a8155050 0x7f24a81504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24a8155050 0x7f24a81504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45056/0 (socket says 172.21.15.97:45056) 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 -- 172.21.15.97:0/781032778 learned_addr learned my addr 172.21.15.97:0/781032778 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ae245700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24a8150a30 0x7f24a81510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ada44700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2494003fb0 0x7f24a81f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 -- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2494003fb0 msgr2=0x7f24a81f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2494003fb0 0x7f24a81f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 -- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24a8150a30 msgr2=0x7f24a81510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24a8150a30 0x7f24a81510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 -- 172.21.15.97:0/781032778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8154570 con 0x7f24a8155050 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ae245700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24a8150a30 0x7f24a81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:02.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ada44700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2494003fb0 0x7f24a81f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.396+0000 7f24ad243700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24a8155050 0x7f24a81504f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f24a4004000 tx=0x7f24a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.397+0000 7f249e7fc700 1 -- 172.21.15.97:0/781032778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4007240 con 0x7f24a8155050 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.397+0000 7f249e7fc700 1 -- 172.21.15.97:0/781032778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a40073a0 con 0x7f24a8155050 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.397+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a8154850 con 0x7f24a8155050 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.397+0000 7f249e7fc700 1 -- 172.21.15.97:0/781032778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4001290 con 0x7f24a8155050 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.397+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a8153bb0 con 0x7f24a8155050 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.397+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a80753f0 con 0x7f24a8155050 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.398+0000 7f249e7fc700 1 -- 172.21.15.97:0/781032778 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24a4003680 con 0x7f24a8155050 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.398+0000 7f249e7fc700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f248c06aed0 0x7f248c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.398+0000 7f249e7fc700 1 -- 172.21.15.97:0/781032778 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24a408a1e0 con 0x7f24a8155050 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.398+0000 7f24ada44700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f248c06aed0 0x7f248c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.399+0000 7f24ada44700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f248c06aed0 0x7f248c06d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2498009b50 tx=0x7f2498008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.403+0000 7f249e7fc700 1 -- 172.21.15.97:0/781032778 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24a4055440 con 0x7f24a8155050 2023-11-24T06:09:02.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.595+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24a80b05a0 con 0x7f248c06aed0 2023-11-24T06:09:02.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.602+0000 7f249e7fc700 1 -- 172.21.15.97:0/781032778 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264561 (secure 0 0 0) 0x7f2498004030 con 0x7f248c06aed0 2023-11-24T06:09:02.608 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f248c06aed0 msgr2=0x7f248c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f248c06aed0 0x7f248c06d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2498009b50 tx=0x7f2498008040 comp rx=0 tx=0).stop 2023-11-24T06:09:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24a8155050 msgr2=0x7f24a81504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24a8155050 0x7f24a81504f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f24a4004000 tx=0x7f24a4001120 comp rx=0 tx=0).stop 2023-11-24T06:09:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 shutdown_connections 2023-11-24T06:09:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24a8150a30 0x7f24a81510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f248c06aed0 0x7f248c06d390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2494003fb0 0x7f24a81f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 --2- 172.21.15.97:0/781032778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24a8155050 0x7f24a81504f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 >> 172.21.15.97:0/781032778 conn(0x7f24a8071b90 msgr2=0x7f24a80af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:02.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 shutdown_connections 2023-11-24T06:09:02.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.605+0000 7f24afca8700 1 -- 172.21.15.97:0/781032778 wait complete. 2023-11-24T06:09:02.612 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:09:02.620 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T06:09:02.323701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218844,"kb_used_data":2660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755876,"statfs":{"total":773094113280,"available":772870017024,"internally_reserved":0,"allocated":2723840,"data_stored":1680213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.821623"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544347+0000","last_change":"2023-11-24T06:08:58.542671+0000","last_active":"2023-11-24T06:08:58.544347+0000","last_peered":"2023-11-24T06:08:58.544347+0000","last_clean":"2023-11-24T06:08:58.544347+0000","last_became_active":"2023-11-24T06:08:58.541937+0000","last_became_peered":"2023-11-24T06:08:58.541937+0000","last_unstale":"2023-11-24T06:08:58.544347+0000","last_undegraded":"2023-11-24T06:08:58.544347+0000","last_fullsized":"2023-11-24T06:08:58.544347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524219+0000","last_change":"2023-11-24T06:08:58.522085+0000","last_active":"2023-11-24T06:08:58.524219+0000","last_peered":"2023-11-24T06:08:58.524219+0000","last_clean":"2023-11-24T06:08:58.524219+0000","last_became_active":"2023-11-24T06:08:58.521116+0000","last_became_peered":"2023-11-24T06:08:58.521116+0000","last_unstale":"2023-11-24T06:08:58.524219+0000","last_undegraded":"2023-11-24T06:08:58.524219+0000","last_fullsized":"2023-11-24T06:08:58.524219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.521057+0000","last_change":"2023-11-24T06:08:58.515201+0000","last_active":"2023-11-24T06:08:58.521057+0000","last_peered":"2023-11-24T06:08:58.521057+0000","last_clean":"2023-11-24T06:08:58.521057+0000","last_became_active":"2023-11-24T06:08:58.514911+0000","last_became_peered":"2023-11-24T06:08:58.514911+0000","last_unstale":"2023-11-24T06:08:58.521057+0000","last_undegraded":"2023-11-24T06:08:58.521057+0000","last_fullsized":"2023-11-24T06:08:58.521057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527351+0000","last_change":"2023-11-24T06:08:58.520988+0000","last_active":"2023-11-24T06:08:58.527351+0000","last_peered":"2023-11-24T06:08:58.527351+0000","last_clean":"2023-11-24T06:08:58.527351+0000","last_became_active":"2023-11-24T06:08:58.520029+0000","last_became_peered":"2023-11-24T06:08:58.520029+0000","last_unstale":"2023-11-24T06:08:58.527351+0000","last_undegraded":"2023-11-24T06:08:58.527351+0000","last_fullsized":"2023-11-24T06:08:58.527351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.542169+0000","last_change":"2023-11-24T06:08:58.535989+0000","last_active":"2023-11-24T06:08:58.542169+0000","last_peered":"2023-11-24T06:08:58.542169+0000","last_clean":"2023-11-24T06:08:58.542169+0000","last_became_active":"2023-11-24T06:08:58.535697+0000","last_became_peered":"2023-11-24T06:08:58.535697+0000","last_unstale":"2023-11-24T06:08:58.542169+0000","last_undegraded":"2023-11-24T06:08:58.542169+0000","last_fullsized":"2023-11-24T06:08:58.542169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:40.002717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532634+0000","last_change":"2023-11-24T06:08:58.523237+0000","last_active":"2023-11-24T06:08:58.532634+0000","last_peered":"2023-11-24T06:08:58.532634+0000","last_clean":"2023-11-24T06:08:58.532634+0000","last_became_active":"2023-11-24T06:08:58.522368+0000","last_became_peered":"2023-11-24T06:08:58.522368+0000","last_unstale":"2023-11-24T06:08:58.532634+0000","last_undegraded":"2023-11-24T06:08:58.532634+0000","last_fullsized":"2023-11-24T06:08:58.532634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536344+0000","last_change":"2023-11-24T06:08:58.533834+0000","last_active":"2023-11-24T06:08:58.536344+0000","last_peered":"2023-11-24T06:08:58.536344+0000","last_clean":"2023-11-24T06:08:58.536344+0000","last_became_active":"2023-11-24T06:08:58.533539+0000","last_became_peered":"2023-11-24T06:08:58.533539+0000","last_unstale":"2023-11-24T06:08:58.536344+0000","last_undegraded":"2023-11-24T06:08:58.536344+0000","last_fullsized":"2023-11-24T06:08:58.536344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534044+0000","last_change":"2023-11-24T06:08:58.531528+0000","last_active":"2023-11-24T06:08:58.534044+0000","last_peered":"2023-11-24T06:08:58.534044+0000","last_clean":"2023-11-24T06:08:58.534044+0000","last_became_active":"2023-11-24T06:08:58.531144+0000","last_became_peered":"2023-11-24T06:08:58.531144+0000","last_unstale":"2023-11-24T06:08:58.534044+0000","last_undegraded":"2023-11-24T06:08:58.534044+0000","last_fullsized":"2023-11-24T06:08:58.534044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:40.002717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544372+0000","last_change":"2023-11-24T06:08:58.542860+0000","last_active":"2023-11-24T06:08:58.544372+0000","last_peered":"2023-11-24T06:08:58.544372+0000","last_clean":"2023-11-24T06:08:58.544372+0000","last_became_active":"2023-11-24T06:08:58.542347+0000","last_became_peered":"2023-11-24T06:08:58.542347+0000","last_unstale":"2023-11-24T06:08:58.544372+0000","last_undegraded":"2023-11-24T06:08:58.544372+0000","last_fullsized":"2023-11-24T06:08:58.544372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533871+0000","last_change":"2023-11-24T06:08:58.529308+0000","last_active":"2023-11-24T06:08:58.533871+0000","last_peered":"2023-11-24T06:08:58.533871+0000","last_clean":"2023-11-24T06:08:58.533871+0000","last_became_active":"2023-11-24T06:08:58.528986+0000","last_became_peered":"2023-11-24T06:08:58.528986+0000","last_unstale":"2023-11-24T06:08:58.533871+0000","last_undegraded":"2023-11-24T06:08:58.533871+0000","last_fullsized":"2023-11-24T06:08:58.533871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534184+0000","last_change":"2023-11-24T06:08:58.530560+0000","last_active":"2023-11-24T06:08:58.534184+0000","last_peered":"2023-11-24T06:08:58.534184+0000","last_clean":"2023-11-24T06:08:58.534184+0000","last_became_active":"2023-11-24T06:08:58.530299+0000","last_became_peered":"2023-11-24T06:08:58.530299+0000","last_unstale":"2023-11-24T06:08:58.534184+0000","last_undegraded":"2023-11-24T06:08:58.534184+0000","last_fullsized":"2023-11-24T06:08:58.534184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.531369+0000","last_change":"2023-11-24T06:08:58.529565+0000","last_active":"2023-11-24T06:08:58.531369+0000","last_peered":"2023-11-24T06:08:58.531369+0000","last_clean":"2023-11-24T06:08:58.531369+0000","last_became_active":"2023-11-24T06:08:58.529170+0000","last_became_peered":"2023-11-24T06:08:58.529170+0000","last_unstale":"2023-11-24T06:08:58.531369+0000","last_undegraded":"2023-11-24T06:08:58.531369+0000","last_fullsized":"2023-11-24T06:08:58.531369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527282+0000","last_change":"2023-11-24T06:08:58.521255+0000","last_active":"2023-11-24T06:08:58.527282+0000","last_peered":"2023-11-24T06:08:58.527282+0000","last_clean":"2023-11-24T06:08:58.527282+0000","last_became_active":"2023-11-24T06:08:58.520950+0000","last_became_peered":"2023-11-24T06:08:58.520950+0000","last_unstale":"2023-11-24T06:08:58.527282+0000","last_undegraded":"2023-11-24T06:08:58.527282+0000","last_fullsized":"2023-11-24T06:08:58.527282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:18.279785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527539+0000","last_change":"2023-11-24T06:08:58.525412+0000","last_active":"2023-11-24T06:08:58.527539+0000","last_peered":"2023-11-24T06:08:58.527539+0000","last_clean":"2023-11-24T06:08:58.527539+0000","last_became_active":"2023-11-24T06:08:58.525080+0000","last_became_peered":"2023-11-24T06:08:58.525080+0000","last_unstale":"2023-11-24T06:08:58.527539+0000","last_undegraded":"2023-11-24T06:08:58.527539+0000","last_fullsized":"2023-11-24T06:08:58.527539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533529+0000","last_change":"2023-11-24T06:08:58.529101+0000","last_active":"2023-11-24T06:08:58.533529+0000","last_peered":"2023-11-24T06:08:58.533529+0000","last_clean":"2023-11-24T06:08:58.533529+0000","last_became_active":"2023-11-24T06:08:58.528814+0000","last_became_peered":"2023-11-24T06:08:58.528814+0000","last_unstale":"2023-11-24T06:08:58.533529+0000","last_undegraded":"2023-11-24T06:08:58.533529+0000","last_fullsized":"2023-11-24T06:08:58.533529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532662+0000","last_change":"2023-11-24T06:08:58.523287+0000","last_active":"2023-11-24T06:08:58.532662+0000","last_peered":"2023-11-24T06:08:58.532662+0000","last_clean":"2023-11-24T06:08:58.532662+0000","last_became_active":"2023-11-24T06:08:58.523033+0000","last_became_peered":"2023-11-24T06:08:58.523033+0000","last_unstale":"2023-11-24T06:08:58.532662+0000","last_undegraded":"2023-11-24T06:08:58.532662+0000","last_fullsized":"2023-11-24T06:08:58.532662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544356+0000","last_change":"2023-11-24T06:08:58.543046+0000","last_active":"2023-11-24T06:08:58.544356+0000","last_peered":"2023-11-24T06:08:58.544356+0000","last_clean":"2023-11-24T06:08:58.544356+0000","last_became_active":"2023-11-24T06:08:58.542703+0000","last_became_peered":"2023-11-24T06:08:58.542703+0000","last_unstale":"2023-11-24T06:08:58.544356+0000","last_undegraded":"2023-11-24T06:08:58.544356+0000","last_fullsized":"2023-11-24T06:08:58.544356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524239+0000","last_change":"2023-11-24T06:08:58.521888+0000","last_active":"2023-11-24T06:08:58.524239+0000","last_peered":"2023-11-24T06:08:58.524239+0000","last_clean":"2023-11-24T06:08:58.524239+0000","last_became_active":"2023-11-24T06:08:58.521599+0000","last_became_peered":"2023-11-24T06:08:58.521599+0000","last_unstale":"2023-11-24T06:08:58.524239+0000","last_undegraded":"2023-11-24T06:08:58.524239+0000","last_fullsized":"2023-11-24T06:08:58.524239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.542143+0000","last_change":"2023-11-24T06:08:58.536029+0000","last_active":"2023-11-24T06:08:58.542143+0000","last_peered":"2023-11-24T06:08:58.542143+0000","last_clean":"2023-11-24T06:08:58.542143+0000","last_became_active":"2023-11-24T06:08:58.535076+0000","last_became_peered":"2023-11-24T06:08:58.535076+0000","last_unstale":"2023-11-24T06:08:58.542143+0000","last_undegraded":"2023-11-24T06:08:58.542143+0000","last_fullsized":"2023-11-24T06:08:58.542143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544489+0000","last_change":"2023-11-24T06:08:57.529523+0000","last_active":"2023-11-24T06:08:58.544489+0000","last_peered":"2023-11-24T06:08:58.544489+0000","last_clean":"2023-11-24T06:08:58.544489+0000","last_became_active":"2023-11-24T06:08:57.529207+0000","last_became_peered":"2023-11-24T06:08:57.529207+0000","last_unstale":"2023-11-24T06:08:58.544489+0000","last_undegraded":"2023-11-24T06:08:58.544489+0000","last_fullsized":"2023-11-24T06:08:58.544489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:52.525711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527321+0000","last_change":"2023-11-24T06:08:58.523242+0000","last_active":"2023-11-24T06:08:58.527321+0000","last_peered":"2023-11-24T06:08:58.527321+0000","last_clean":"2023-11-24T06:08:58.527321+0000","last_became_active":"2023-11-24T06:08:58.522951+0000","last_became_peered":"2023-11-24T06:08:58.522951+0000","last_unstale":"2023-11-24T06:08:58.527321+0000","last_undegraded":"2023-11-24T06:08:58.527321+0000","last_fullsized":"2023-11-24T06:08:58.527321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534322+0000","last_change":"2023-11-24T06:08:57.523196+0000","last_active":"2023-11-24T06:08:58.534322+0000","last_peered":"2023-11-24T06:08:58.534322+0000","last_clean":"2023-11-24T06:08:58.534322+0000","last_became_active":"2023-11-24T06:08:57.522871+0000","last_became_peered":"2023-11-24T06:08:57.522871+0000","last_unstale":"2023-11-24T06:08:58.534322+0000","last_undegraded":"2023-11-24T06:08:58.534322+0000","last_fullsized":"2023-11-24T06:08:58.534322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:22.630593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536081+0000","last_change":"2023-11-24T06:08:58.530832+0000","last_active":"2023-11-24T06:08:58.536081+0000","last_peered":"2023-11-24T06:08:58.536081+0000","last_clean":"2023-11-24T06:08:58.536081+0000","last_became_active":"2023-11-24T06:08:58.530383+0000","last_became_peered":"2023-11-24T06:08:58.530383+0000","last_unstale":"2023-11-24T06:08:58.536081+0000","last_undegraded":"2023-11-24T06:08:58.536081+0000","last_fullsized":"2023-11-24T06:08:58.536081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.539542+0000","last_change":"2023-11-24T06:08:57.528776+0000","last_active":"2023-11-24T06:08:58.539542+0000","last_peered":"2023-11-24T06:08:58.539542+0000","last_clean":"2023-11-24T06:08:58.539542+0000","last_became_active":"2023-11-24T06:08:57.528480+0000","last_became_peered":"2023-11-24T06:08:57.528480+0000","last_unstale":"2023-11-24T06:08:58.539542+0000","last_undegraded":"2023-11-24T06:08:58.539542+0000","last_fullsized":"2023-11-24T06:08:58.539542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532581+0000","last_change":"2023-11-24T06:08:58.524300+0000","last_active":"2023-11-24T06:08:58.532581+0000","last_peered":"2023-11-24T06:08:58.532581+0000","last_clean":"2023-11-24T06:08:58.532581+0000","last_became_active":"2023-11-24T06:08:58.522378+0000","last_became_peered":"2023-11-24T06:08:58.522378+0000","last_unstale":"2023-11-24T06:08:58.532581+0000","last_undegraded":"2023-11-24T06:08:58.532581+0000","last_fullsized":"2023-11-24T06:08:58.532581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544105+0000","last_change":"2023-11-24T06:08:57.526272+0000","last_active":"2023-11-24T06:08:58.544105+0000","last_peered":"2023-11-24T06:08:58.544105+0000","last_clean":"2023-11-24T06:08:58.544105+0000","last_became_active":"2023-11-24T06:08:57.525998+0000","last_became_peered":"2023-11-24T06:08:57.525998+0000","last_unstale":"2023-11-24T06:08:58.544105+0000","last_undegraded":"2023-11-24T06:08:58.544105+0000","last_fullsized":"2023-11-24T06:08:58.544105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533283+0000","last_change":"2023-11-24T06:08:58.523547+0000","last_active":"2023-11-24T06:08:58.533283+0000","last_peered":"2023-11-24T06:08:58.533283+0000","last_clean":"2023-11-24T06:08:58.533283+0000","last_became_active":"2023-11-24T06:08:58.522784+0000","last_became_peered":"2023-11-24T06:08:58.522784+0000","last_unstale":"2023-11-24T06:08:58.533283+0000","last_undegraded":"2023-11-24T06:08:58.533283+0000","last_fullsized":"2023-11-24T06:08:58.533283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.539080+0000","last_change":"2023-11-24T06:08:57.528824+0000","last_active":"2023-11-24T06:08:58.539080+0000","last_peered":"2023-11-24T06:08:58.539080+0000","last_clean":"2023-11-24T06:08:58.539080+0000","last_became_active":"2023-11-24T06:08:57.528517+0000","last_became_peered":"2023-11-24T06:08:57.528517+0000","last_unstale":"2023-11-24T06:08:58.539080+0000","last_undegraded":"2023-11-24T06:08:58.539080+0000","last_fullsized":"2023-11-24T06:08:58.539080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527092+0000","last_change":"2023-11-24T06:08:58.523181+0000","last_active":"2023-11-24T06:08:58.527092+0000","last_peered":"2023-11-24T06:08:58.527092+0000","last_clean":"2023-11-24T06:08:58.527092+0000","last_became_active":"2023-11-24T06:08:58.522917+0000","last_became_peered":"2023-11-24T06:08:58.522917+0000","last_unstale":"2023-11-24T06:08:58.527092+0000","last_undegraded":"2023-11-24T06:08:58.527092+0000","last_fullsized":"2023-11-24T06:08:58.527092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536672+0000","last_change":"2023-11-24T06:08:57.524473+0000","last_active":"2023-11-24T06:08:58.536672+0000","last_peered":"2023-11-24T06:08:58.536672+0000","last_clean":"2023-11-24T06:08:58.536672+0000","last_became_active":"2023-11-24T06:08:57.523947+0000","last_became_peered":"2023-11-24T06:08:57.523947+0000","last_unstale":"2023-11-24T06:08:58.536672+0000","last_undegraded":"2023-11-24T06:08:58.536672+0000","last_fullsized":"2023-11-24T06:08:58.536672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.529518+0000","last_change":"2023-11-24T06:08:58.526170+0000","last_active":"2023-11-24T06:08:58.529518+0000","last_peered":"2023-11-24T06:08:58.529518+0000","last_clean":"2023-11-24T06:08:58.529518+0000","last_became_active":"2023-11-24T06:08:58.525874+0000","last_became_peered":"2023-11-24T06:08:58.525874+0000","last_unstale":"2023-11-24T06:08:58.529518+0000","last_undegraded":"2023-11-24T06:08:58.529518+0000","last_fullsized":"2023-11-24T06:08:58.529518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.511356+0000","last_change":"2023-11-24T06:08:57.520895+0000","last_active":"2023-11-24T06:08:58.511356+0000","last_peered":"2023-11-24T06:08:58.511356+0000","last_clean":"2023-11-24T06:08:58.511356+0000","last_became_active":"2023-11-24T06:08:57.520367+0000","last_became_peered":"2023-11-24T06:08:57.520367+0000","last_unstale":"2023-11-24T06:08:58.511356+0000","last_undegraded":"2023-11-24T06:08:58.511356+0000","last_fullsized":"2023-11-24T06:08:58.511356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:55.763227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.525781+0000","last_change":"2023-11-24T06:08:58.519434+0000","last_active":"2023-11-24T06:08:58.525781+0000","last_peered":"2023-11-24T06:08:58.525781+0000","last_clean":"2023-11-24T06:08:58.525781+0000","last_became_active":"2023-11-24T06:08:58.519148+0000","last_became_peered":"2023-11-24T06:08:58.519148+0000","last_unstale":"2023-11-24T06:08:58.525781+0000","last_undegraded":"2023-11-24T06:08:58.525781+0000","last_fullsized":"2023-11-24T06:08:58.525781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.514508+0000","last_change":"2023-11-24T06:08:57.525692+0000","last_active":"2023-11-24T06:08:58.514508+0000","last_peered":"2023-11-24T06:08:58.514508+0000","last_clean":"2023-11-24T06:08:58.514508+0000","last_became_active":"2023-11-24T06:08:57.525451+0000","last_became_peered":"2023-11-24T06:08:57.525451+0000","last_unstale":"2023-11-24T06:08:58.514508+0000","last_undegraded":"2023-11-24T06:08:58.514508+0000","last_fullsized":"2023-11-24T06:08:58.514508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.523690+0000","last_change":"2023-11-24T06:08:58.521741+0000","last_active":"2023-11-24T06:08:58.523690+0000","last_peered":"2023-11-24T06:08:58.523690+0000","last_clean":"2023-11-24T06:08:58.523690+0000","last_became_active":"2023-11-24T06:08:58.521275+0000","last_became_peered":"2023-11-24T06:08:58.521275+0000","last_unstale":"2023-11-24T06:08:58.523690+0000","last_undegraded":"2023-11-24T06:08:58.523690+0000","last_fullsized":"2023-11-24T06:08:58.523690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.519500+0000","last_change":"2023-11-24T06:08:57.526466+0000","last_active":"2023-11-24T06:08:58.519500+0000","last_peered":"2023-11-24T06:08:58.519500+0000","last_clean":"2023-11-24T06:08:58.519500+0000","last_became_active":"2023-11-24T06:08:57.526174+0000","last_became_peered":"2023-11-24T06:08:57.526174+0000","last_unstale":"2023-11-24T06:08:58.519500+0000","last_undegraded":"2023-11-24T06:08:58.519500+0000","last_fullsized":"2023-11-24T06:08:58.519500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543843+0000","last_change":"2023-11-24T06:08:58.541089+0000","last_active":"2023-11-24T06:08:58.543843+0000","last_peered":"2023-11-24T06:08:58.543843+0000","last_clean":"2023-11-24T06:08:58.543843+0000","last_became_active":"2023-11-24T06:08:58.540792+0000","last_became_peered":"2023-11-24T06:08:58.540792+0000","last_unstale":"2023-11-24T06:08:58.543843+0000","last_undegraded":"2023-11-24T06:08:58.543843+0000","last_fullsized":"2023-11-24T06:08:58.543843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.518909+0000","last_change":"2023-11-24T06:08:57.525181+0000","last_active":"2023-11-24T06:08:58.518909+0000","last_peered":"2023-11-24T06:08:58.518909+0000","last_clean":"2023-11-24T06:08:58.518909+0000","last_became_active":"2023-11-24T06:08:57.524906+0000","last_became_peered":"2023-11-24T06:08:57.524906+0000","last_unstale":"2023-11-24T06:08:58.518909+0000","last_undegraded":"2023-11-24T06:08:58.518909+0000","last_fullsized":"2023-11-24T06:08:58.518909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536042+0000","last_change":"2023-11-24T06:08:58.530664+0000","last_active":"2023-11-24T06:08:58.536042+0000","last_peered":"2023-11-24T06:08:58.536042+0000","last_clean":"2023-11-24T06:08:58.536042+0000","last_became_active":"2023-11-24T06:08:58.530068+0000","last_became_peered":"2023-11-24T06:08:58.530068+0000","last_unstale":"2023-11-24T06:08:58.536042+0000","last_undegraded":"2023-11-24T06:08:58.536042+0000","last_fullsized":"2023-11-24T06:08:58.536042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534175+0000","last_change":"2023-11-24T06:08:57.525584+0000","last_active":"2023-11-24T06:08:58.534175+0000","last_peered":"2023-11-24T06:08:58.534175+0000","last_clean":"2023-11-24T06:08:58.534175+0000","last_became_active":"2023-11-24T06:08:57.525340+0000","last_became_peered":"2023-11-24T06:08:57.525340+0000","last_unstale":"2023-11-24T06:08:58.534175+0000","last_undegraded":"2023-11-24T06:08:58.534175+0000","last_fullsized":"2023-11-24T06:08:58.534175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.539564+0000","last_change":"2023-11-24T06:08:58.537817+0000","last_active":"2023-11-24T06:08:58.539564+0000","last_peered":"2023-11-24T06:08:58.539564+0000","last_clean":"2023-11-24T06:08:58.539564+0000","last_became_active":"2023-11-24T06:08:58.537604+0000","last_became_peered":"2023-11-24T06:08:58.537604+0000","last_unstale":"2023-11-24T06:08:58.539564+0000","last_undegraded":"2023-11-24T06:08:58.539564+0000","last_fullsized":"2023-11-24T06:08:58.539564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544228+0000","last_change":"2023-11-24T06:08:57.520656+0000","last_active":"2023-11-24T06:08:58.544228+0000","last_peered":"2023-11-24T06:08:58.544228+0000","last_clean":"2023-11-24T06:08:58.544228+0000","last_became_active":"2023-11-24T06:08:57.520398+0000","last_became_peered":"2023-11-24T06:08:57.520398+0000","last_unstale":"2023-11-24T06:08:58.544228+0000","last_undegraded":"2023-11-24T06:08:58.544228+0000","last_fullsized":"2023-11-24T06:08:58.544228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536322+0000","last_change":"2023-11-24T06:08:58.533919+0000","last_active":"2023-11-24T06:08:58.536322+0000","last_peered":"2023-11-24T06:08:58.536322+0000","last_clean":"2023-11-24T06:08:58.536322+0000","last_became_active":"2023-11-24T06:08:58.533565+0000","last_became_peered":"2023-11-24T06:08:58.533565+0000","last_unstale":"2023-11-24T06:08:58.536322+0000","last_undegraded":"2023-11-24T06:08:58.536322+0000","last_fullsized":"2023-11-24T06:08:58.536322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.537707+0000","last_change":"2023-11-24T06:08:57.523298+0000","last_active":"2023-11-24T06:08:58.537707+0000","last_peered":"2023-11-24T06:08:58.537707+0000","last_clean":"2023-11-24T06:08:58.537707+0000","last_became_active":"2023-11-24T06:08:57.523000+0000","last_became_peered":"2023-11-24T06:08:57.523000+0000","last_unstale":"2023-11-24T06:08:58.537707+0000","last_undegraded":"2023-11-24T06:08:58.537707+0000","last_fullsized":"2023-11-24T06:08:58.537707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543784+0000","last_change":"2023-11-24T06:08:58.539907+0000","last_active":"2023-11-24T06:08:58.543784+0000","last_peered":"2023-11-24T06:08:58.543784+0000","last_clean":"2023-11-24T06:08:58.543784+0000","last_became_active":"2023-11-24T06:08:58.539663+0000","last_became_peered":"2023-11-24T06:08:58.539663+0000","last_unstale":"2023-11-24T06:08:58.543784+0000","last_undegraded":"2023-11-24T06:08:58.543784+0000","last_fullsized":"2023-11-24T06:08:58.543784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:51.910530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.519678+0000","last_change":"2023-11-24T06:08:57.523074+0000","last_active":"2023-11-24T06:08:58.519678+0000","last_peered":"2023-11-24T06:08:58.519678+0000","last_clean":"2023-11-24T06:08:58.519678+0000","last_became_active":"2023-11-24T06:08:57.522527+0000","last_became_peered":"2023-11-24T06:08:57.522527+0000","last_unstale":"2023-11-24T06:08:58.519678+0000","last_undegraded":"2023-11-24T06:08:58.519678+0000","last_fullsized":"2023-11-24T06:08:58.519678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:31.591882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.542190+0000","last_change":"2023-11-24T06:08:58.538966+0000","last_active":"2023-11-24T06:08:58.542190+0000","last_peered":"2023-11-24T06:08:58.542190+0000","last_clean":"2023-11-24T06:08:58.542190+0000","last_became_active":"2023-11-24T06:08:58.538575+0000","last_became_peered":"2023-11-24T06:08:58.538575+0000","last_unstale":"2023-11-24T06:08:58.542190+0000","last_undegraded":"2023-11-24T06:08:58.542190+0000","last_fullsized":"2023-11-24T06:08:58.542190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:51.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.518534+0000","last_change":"2023-11-24T06:08:57.524582+0000","last_active":"2023-11-24T06:08:58.518534+0000","last_peered":"2023-11-24T06:08:58.518534+0000","last_clean":"2023-11-24T06:08:58.518534+0000","last_became_active":"2023-11-24T06:08:57.524351+0000","last_became_peered":"2023-11-24T06:08:57.524351+0000","last_unstale":"2023-11-24T06:08:58.518534+0000","last_undegraded":"2023-11-24T06:08:58.518534+0000","last_fullsized":"2023-11-24T06:08:58.518534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:22.630593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534253+0000","last_change":"2023-11-24T06:08:58.528140+0000","last_active":"2023-11-24T06:08:58.534253+0000","last_peered":"2023-11-24T06:08:58.534253+0000","last_clean":"2023-11-24T06:08:58.534253+0000","last_became_active":"2023-11-24T06:08:58.527685+0000","last_became_peered":"2023-11-24T06:08:58.527685+0000","last_unstale":"2023-11-24T06:08:58.534253+0000","last_undegraded":"2023-11-24T06:08:58.534253+0000","last_fullsized":"2023-11-24T06:08:58.534253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.541509+0000","last_change":"2023-11-24T06:08:57.519672+0000","last_active":"2023-11-24T06:08:58.541509+0000","last_peered":"2023-11-24T06:08:58.541509+0000","last_clean":"2023-11-24T06:08:58.541509+0000","last_became_active":"2023-11-24T06:08:57.519379+0000","last_became_peered":"2023-11-24T06:08:57.519379+0000","last_unstale":"2023-11-24T06:08:58.541509+0000","last_undegraded":"2023-11-24T06:08:58.541509+0000","last_fullsized":"2023-11-24T06:08:58.541509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:43.426568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536322+0000","last_change":"2023-11-24T06:08:58.531525+0000","last_active":"2023-11-24T06:08:58.536322+0000","last_peered":"2023-11-24T06:08:58.536322+0000","last_clean":"2023-11-24T06:08:58.536322+0000","last_became_active":"2023-11-24T06:08:58.529801+0000","last_became_peered":"2023-11-24T06:08:58.529801+0000","last_unstale":"2023-11-24T06:08:58.536322+0000","last_undegraded":"2023-11-24T06:08:58.536322+0000","last_fullsized":"2023-11-24T06:08:58.536322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.539009+0000","last_change":"2023-11-24T06:08:57.523021+0000","last_active":"2023-11-24T06:08:58.539009+0000","last_peered":"2023-11-24T06:08:58.539009+0000","last_clean":"2023-11-24T06:08:58.539009+0000","last_became_active":"2023-11-24T06:08:57.522474+0000","last_became_peered":"2023-11-24T06:08:57.522474+0000","last_unstale":"2023-11-24T06:08:58.539009+0000","last_undegraded":"2023-11-24T06:08:58.539009+0000","last_fullsized":"2023-11-24T06:08:58.539009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:43.891317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.541596+0000","last_change":"2023-11-24T06:08:58.535871+0000","last_active":"2023-11-24T06:08:58.541596+0000","last_peered":"2023-11-24T06:08:58.541596+0000","last_clean":"2023-11-24T06:08:58.541596+0000","last_became_active":"2023-11-24T06:08:58.535563+0000","last_became_peered":"2023-11-24T06:08:58.535563+0000","last_unstale":"2023-11-24T06:08:58.541596+0000","last_undegraded":"2023-11-24T06:08:58.541596+0000","last_fullsized":"2023-11-24T06:08:58.541596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.517025+0000","last_change":"2023-11-24T06:08:57.519605+0000","last_active":"2023-11-24T06:08:58.517025+0000","last_peered":"2023-11-24T06:08:58.517025+0000","last_clean":"2023-11-24T06:08:58.517025+0000","last_became_active":"2023-11-24T06:08:57.519308+0000","last_became_peered":"2023-11-24T06:08:57.519308+0000","last_unstale":"2023-11-24T06:08:58.517025+0000","last_undegraded":"2023-11-24T06:08:58.517025+0000","last_fullsized":"2023-11-24T06:08:58.517025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533303+0000","last_change":"2023-11-24T06:08:58.523994+0000","last_active":"2023-11-24T06:08:58.533303+0000","last_peered":"2023-11-24T06:08:58.533303+0000","last_clean":"2023-11-24T06:08:58.533303+0000","last_became_active":"2023-11-24T06:08:58.523750+0000","last_became_peered":"2023-11-24T06:08:58.523750+0000","last_unstale":"2023-11-24T06:08:58.533303+0000","last_undegraded":"2023-11-24T06:08:58.533303+0000","last_fullsized":"2023-11-24T06:08:58.533303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.520407+0000","last_change":"2023-11-24T06:08:57.525492+0000","last_active":"2023-11-24T06:08:58.520407+0000","last_peered":"2023-11-24T06:08:58.520407+0000","last_clean":"2023-11-24T06:08:58.520407+0000","last_became_active":"2023-11-24T06:08:57.525231+0000","last_became_peered":"2023-11-24T06:08:57.525231+0000","last_unstale":"2023-11-24T06:08:58.520407+0000","last_undegraded":"2023-11-24T06:08:58.520407+0000","last_fullsized":"2023-11-24T06:08:58.520407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:35.934471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533028+0000","last_change":"2023-11-24T06:08:58.525233+0000","last_active":"2023-11-24T06:08:58.533028+0000","last_peered":"2023-11-24T06:08:58.533028+0000","last_clean":"2023-11-24T06:08:58.533028+0000","last_became_active":"2023-11-24T06:08:58.524919+0000","last_became_peered":"2023-11-24T06:08:58.524919+0000","last_unstale":"2023-11-24T06:08:58.533028+0000","last_undegraded":"2023-11-24T06:08:58.533028+0000","last_fullsized":"2023-11-24T06:08:58.533028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.523752+0000","last_change":"2023-11-24T06:08:58.521720+0000","last_active":"2023-11-24T06:08:58.523752+0000","last_peered":"2023-11-24T06:08:58.523752+0000","last_clean":"2023-11-24T06:08:58.523752+0000","last_became_active":"2023-11-24T06:08:58.521358+0000","last_became_peered":"2023-11-24T06:08:58.521358+0000","last_unstale":"2023-11-24T06:08:58.523752+0000","last_undegraded":"2023-11-24T06:08:58.523752+0000","last_fullsized":"2023-11-24T06:08:58.523752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.510094+0000","last_change":"2023-11-24T06:08:57.521087+0000","last_active":"2023-11-24T06:08:58.510094+0000","last_peered":"2023-11-24T06:08:58.510094+0000","last_clean":"2023-11-24T06:08:58.510094+0000","last_became_active":"2023-11-24T06:08:57.520758+0000","last_became_peered":"2023-11-24T06:08:57.520758+0000","last_unstale":"2023-11-24T06:08:58.510094+0000","last_undegraded":"2023-11-24T06:08:58.510094+0000","last_fullsized":"2023-11-24T06:08:58.510094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:28.366035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533816+0000","last_change":"2023-11-24T06:08:58.531268+0000","last_active":"2023-11-24T06:08:58.533816+0000","last_peered":"2023-11-24T06:08:58.533816+0000","last_clean":"2023-11-24T06:08:58.533816+0000","last_became_active":"2023-11-24T06:08:58.530982+0000","last_became_peered":"2023-11-24T06:08:58.530982+0000","last_unstale":"2023-11-24T06:08:58.533816+0000","last_undegraded":"2023-11-24T06:08:58.533816+0000","last_fullsized":"2023-11-24T06:08:58.533816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.514439+0000","last_change":"2023-11-24T06:08:57.519233+0000","last_active":"2023-11-24T06:08:58.514439+0000","last_peered":"2023-11-24T06:08:58.514439+0000","last_clean":"2023-11-24T06:08:58.514439+0000","last_became_active":"2023-11-24T06:08:57.518413+0000","last_became_peered":"2023-11-24T06:08:57.518413+0000","last_unstale":"2023-11-24T06:08:58.514439+0000","last_undegraded":"2023-11-24T06:08:58.514439+0000","last_fullsized":"2023-11-24T06:08:58.514439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-24T06:08:57.516151+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:08:57.516151+0000","last_peered":"2023-11-24T06:08:57.516151+0000","last_clean":"2023-11-24T06:08:57.516151+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:08:57.516151+0000","last_undegraded":"2023-11-24T06:08:57.516151+0000","last_fullsized":"2023-11-24T06:08:57.516151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.530681+0000","last_change":"2023-11-24T06:08:58.528408+0000","last_active":"2023-11-24T06:08:58.530681+0000","last_peered":"2023-11-24T06:08:58.530681+0000","last_clean":"2023-11-24T06:08:58.530681+0000","last_became_active":"2023-11-24T06:08:58.528125+0000","last_became_peered":"2023-11-24T06:08:58.528125+0000","last_unstale":"2023-11-24T06:08:58.530681+0000","last_undegraded":"2023-11-24T06:08:58.530681+0000","last_fullsized":"2023-11-24T06:08:58.530681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:03.307275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.516395+0000","last_change":"2023-11-24T06:08:57.527351+0000","last_active":"2023-11-24T06:08:58.516395+0000","last_peered":"2023-11-24T06:08:58.516395+0000","last_clean":"2023-11-24T06:08:58.516395+0000","last_became_active":"2023-11-24T06:08:57.527085+0000","last_became_peered":"2023-11-24T06:08:57.527085+0000","last_unstale":"2023-11-24T06:08:58.516395+0000","last_undegraded":"2023-11-24T06:08:58.516395+0000","last_fullsized":"2023-11-24T06:08:58.516395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.520674+0000","last_change":"2023-11-24T06:08:57.528763+0000","last_active":"2023-11-24T06:08:58.520674+0000","last_peered":"2023-11-24T06:08:58.520674+0000","last_clean":"2023-11-24T06:08:58.520674+0000","last_became_active":"2023-11-24T06:08:57.528469+0000","last_became_peered":"2023-11-24T06:08:57.528469+0000","last_unstale":"2023-11-24T06:08:58.520674+0000","last_undegraded":"2023-11-24T06:08:58.520674+0000","last_fullsized":"2023-11-24T06:08:58.520674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524016+0000","last_change":"2023-11-24T06:08:58.522186+0000","last_active":"2023-11-24T06:08:58.524016+0000","last_peered":"2023-11-24T06:08:58.524016+0000","last_clean":"2023-11-24T06:08:58.524016+0000","last_became_active":"2023-11-24T06:08:58.521844+0000","last_became_peered":"2023-11-24T06:08:58.521844+0000","last_unstale":"2023-11-24T06:08:58.524016+0000","last_undegraded":"2023-11-24T06:08:58.524016+0000","last_fullsized":"2023-11-24T06:08:58.524016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.517366+0000","last_change":"2023-11-24T06:08:57.519266+0000","last_active":"2023-11-24T06:08:58.517366+0000","last_peered":"2023-11-24T06:08:58.517366+0000","last_clean":"2023-11-24T06:08:58.517366+0000","last_became_active":"2023-11-24T06:08:57.518952+0000","last_became_peered":"2023-11-24T06:08:57.518952+0000","last_unstale":"2023-11-24T06:08:58.517366+0000","last_undegraded":"2023-11-24T06:08:58.517366+0000","last_fullsized":"2023-11-24T06:08:58.517366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543999+0000","last_change":"2023-11-24T06:08:58.539562+0000","last_active":"2023-11-24T06:08:58.543999+0000","last_peered":"2023-11-24T06:08:58.543999+0000","last_clean":"2023-11-24T06:08:58.543999+0000","last_became_active":"2023-11-24T06:08:58.539268+0000","last_became_peered":"2023-11-24T06:08:58.539268+0000","last_unstale":"2023-11-24T06:08:58.543999+0000","last_undegraded":"2023-11-24T06:08:58.543999+0000","last_fullsized":"2023-11-24T06:08:58.543999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543401+0000","last_change":"2023-11-24T06:08:57.525492+0000","last_active":"2023-11-24T06:08:58.543401+0000","last_peered":"2023-11-24T06:08:58.543401+0000","last_clean":"2023-11-24T06:08:58.543401+0000","last_became_active":"2023-11-24T06:08:57.525231+0000","last_became_peered":"2023-11-24T06:08:57.525231+0000","last_unstale":"2023-11-24T06:08:58.543401+0000","last_undegraded":"2023-11-24T06:08:58.543401+0000","last_fullsized":"2023-11-24T06:08:58.543401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524614+0000","last_change":"2023-11-24T06:08:58.522482+0000","last_active":"2023-11-24T06:08:58.524614+0000","last_peered":"2023-11-24T06:08:58.524614+0000","last_clean":"2023-11-24T06:08:58.524614+0000","last_became_active":"2023-11-24T06:08:58.522237+0000","last_became_peered":"2023-11-24T06:08:58.522237+0000","last_unstale":"2023-11-24T06:08:58.524614+0000","last_undegraded":"2023-11-24T06:08:58.524614+0000","last_fullsized":"2023-11-24T06:08:58.524614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.516117+0000","last_change":"2023-11-24T06:08:57.525325+0000","last_active":"2023-11-24T06:08:58.516117+0000","last_peered":"2023-11-24T06:08:58.516117+0000","last_clean":"2023-11-24T06:08:58.516117+0000","last_became_active":"2023-11-24T06:08:57.525051+0000","last_became_peered":"2023-11-24T06:08:57.525051+0000","last_unstale":"2023-11-24T06:08:58.516117+0000","last_undegraded":"2023-11-24T06:08:58.516117+0000","last_fullsized":"2023-11-24T06:08:58.516117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:16.569994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534692+0000","last_change":"2023-11-24T06:08:58.534115+0000","last_active":"2023-11-24T06:08:58.534692+0000","last_peered":"2023-11-24T06:08:58.534692+0000","last_clean":"2023-11-24T06:08:58.534692+0000","last_became_active":"2023-11-24T06:08:58.525128+0000","last_became_peered":"2023-11-24T06:08:58.525128+0000","last_unstale":"2023-11-24T06:08:58.534692+0000","last_undegraded":"2023-11-24T06:08:58.534692+0000","last_fullsized":"2023-11-24T06:08:58.534692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:03.307275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.518114+0000","last_change":"2023-11-24T06:08:57.530707+0000","last_active":"2023-11-24T06:08:58.518114+0000","last_peered":"2023-11-24T06:08:58.518114+0000","last_clean":"2023-11-24T06:08:58.518114+0000","last_became_active":"2023-11-24T06:08:57.529531+0000","last_became_peered":"2023-11-24T06:08:57.529531+0000","last_unstale":"2023-11-24T06:08:58.518114+0000","last_undegraded":"2023-11-24T06:08:58.518114+0000","last_fullsized":"2023-11-24T06:08:58.518114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527289+0000","last_change":"2023-11-24T06:08:58.523161+0000","last_active":"2023-11-24T06:08:58.527289+0000","last_peered":"2023-11-24T06:08:58.527289+0000","last_clean":"2023-11-24T06:08:58.527289+0000","last_became_active":"2023-11-24T06:08:58.522753+0000","last_became_peered":"2023-11-24T06:08:58.522753+0000","last_unstale":"2023-11-24T06:08:58.527289+0000","last_undegraded":"2023-11-24T06:08:58.527289+0000","last_fullsized":"2023-11-24T06:08:58.527289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:03.307275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.517545+0000","last_change":"2023-11-24T06:08:57.521686+0000","last_active":"2023-11-24T06:08:58.517545+0000","last_peered":"2023-11-24T06:08:58.517545+0000","last_clean":"2023-11-24T06:08:58.517545+0000","last_became_active":"2023-11-24T06:08:57.520431+0000","last_became_peered":"2023-11-24T06:08:57.520431+0000","last_unstale":"2023-11-24T06:08:58.517545+0000","last_undegraded":"2023-11-24T06:08:58.517545+0000","last_fullsized":"2023-11-24T06:08:58.517545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:31.591882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524982+0000","last_change":"2023-11-24T06:08:58.519601+0000","last_active":"2023-11-24T06:08:58.524982+0000","last_peered":"2023-11-24T06:08:58.524982+0000","last_clean":"2023-11-24T06:08:58.524982+0000","last_became_active":"2023-11-24T06:08:58.519340+0000","last_became_peered":"2023-11-24T06:08:58.519340+0000","last_unstale":"2023-11-24T06:08:58.524982+0000","last_undegraded":"2023-11-24T06:08:58.524982+0000","last_fullsized":"2023-11-24T06:08:58.524982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.541736+0000","last_change":"2023-11-24T06:08:57.529494+0000","last_active":"2023-11-24T06:08:58.541736+0000","last_peered":"2023-11-24T06:08:58.541736+0000","last_clean":"2023-11-24T06:08:58.541736+0000","last_became_active":"2023-11-24T06:08:57.529231+0000","last_became_peered":"2023-11-24T06:08:57.529231+0000","last_unstale":"2023-11-24T06:08:58.541736+0000","last_undegraded":"2023-11-24T06:08:58.541736+0000","last_fullsized":"2023-11-24T06:08:58.541736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:06.359234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533968+0000","last_change":"2023-11-24T06:08:58.529994+0000","last_active":"2023-11-24T06:08:58.533968+0000","last_peered":"2023-11-24T06:08:58.533968+0000","last_clean":"2023-11-24T06:08:58.533968+0000","last_became_active":"2023-11-24T06:08:58.529528+0000","last_became_peered":"2023-11-24T06:08:58.529528+0000","last_unstale":"2023-11-24T06:08:58.533968+0000","last_undegraded":"2023-11-24T06:08:58.533968+0000","last_fullsized":"2023-11-24T06:08:58.533968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.510177+0000","last_change":"2023-11-24T06:08:57.525323+0000","last_active":"2023-11-24T06:08:58.510177+0000","last_peered":"2023-11-24T06:08:58.510177+0000","last_clean":"2023-11-24T06:08:58.510177+0000","last_became_active":"2023-11-24T06:08:57.525087+0000","last_became_peered":"2023-11-24T06:08:57.525087+0000","last_unstale":"2023-11-24T06:08:58.510177+0000","last_undegraded":"2023-11-24T06:08:58.510177+0000","last_fullsized":"2023-11-24T06:08:58.510177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:43.891317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.542229+0000","last_change":"2023-11-24T06:08:58.539679+0000","last_active":"2023-11-24T06:08:58.542229+0000","last_peered":"2023-11-24T06:08:58.542229+0000","last_clean":"2023-11-24T06:08:58.542229+0000","last_became_active":"2023-11-24T06:08:58.539279+0000","last_became_peered":"2023-11-24T06:08:58.539279+0000","last_unstale":"2023-11-24T06:08:58.542229+0000","last_undegraded":"2023-11-24T06:08:58.542229+0000","last_fullsized":"2023-11-24T06:08:58.542229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:26.004865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534964+0000","last_change":"2023-11-24T06:08:57.523738+0000","last_active":"2023-11-24T06:08:58.534964+0000","last_peered":"2023-11-24T06:08:58.534964+0000","last_clean":"2023-11-24T06:08:58.534964+0000","last_became_active":"2023-11-24T06:08:57.523396+0000","last_became_peered":"2023-11-24T06:08:57.523396+0000","last_unstale":"2023-11-24T06:08:58.534964+0000","last_undegraded":"2023-11-24T06:08:58.534964+0000","last_fullsized":"2023-11-24T06:08:58.534964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527309+0000","last_change":"2023-11-24T06:08:58.520593+0000","last_active":"2023-11-24T06:08:58.527309+0000","last_peered":"2023-11-24T06:08:58.527309+0000","last_clean":"2023-11-24T06:08:58.527309+0000","last_became_active":"2023-11-24T06:08:58.520297+0000","last_became_peered":"2023-11-24T06:08:58.520297+0000","last_unstale":"2023-11-24T06:08:58.527309+0000","last_undegraded":"2023-11-24T06:08:58.527309+0000","last_fullsized":"2023-11-24T06:08:58.527309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536745+0000","last_change":"2023-11-24T06:08:57.521315+0000","last_active":"2023-11-24T06:08:58.536745+0000","last_peered":"2023-11-24T06:08:58.536745+0000","last_clean":"2023-11-24T06:08:58.536745+0000","last_became_active":"2023-11-24T06:08:57.520718+0000","last_became_peered":"2023-11-24T06:08:57.520718+0000","last_unstale":"2023-11-24T06:08:58.536745+0000","last_undegraded":"2023-11-24T06:08:58.536745+0000","last_fullsized":"2023-11-24T06:08:58.536745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:16.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544298+0000","last_change":"2023-11-24T06:08:58.542621+0000","last_active":"2023-11-24T06:08:58.544298+0000","last_peered":"2023-11-24T06:08:58.544298+0000","last_clean":"2023-11-24T06:08:58.544298+0000","last_became_active":"2023-11-24T06:08:58.541632+0000","last_became_peered":"2023-11-24T06:08:58.541632+0000","last_unstale":"2023-11-24T06:08:58.544298+0000","last_undegraded":"2023-11-24T06:08:58.544298+0000","last_fullsized":"2023-11-24T06:08:58.544298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:32.599128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534237+0000","last_change":"2023-11-24T06:08:58.527955+0000","last_active":"2023-11-24T06:08:58.534237+0000","last_peered":"2023-11-24T06:08:58.534237+0000","last_clean":"2023-11-24T06:08:58.534237+0000","last_became_active":"2023-11-24T06:08:58.527336+0000","last_became_peered":"2023-11-24T06:08:58.527336+0000","last_unstale":"2023-11-24T06:08:58.534237+0000","last_undegraded":"2023-11-24T06:08:58.534237+0000","last_fullsized":"2023-11-24T06:08:58.534237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.531404+0000","last_change":"2023-11-24T06:08:58.530318+0000","last_active":"2023-11-24T06:08:58.531404+0000","last_peered":"2023-11-24T06:08:58.531404+0000","last_clean":"2023-11-24T06:08:58.531404+0000","last_became_active":"2023-11-24T06:08:58.530015+0000","last_became_peered":"2023-11-24T06:08:58.530015+0000","last_unstale":"2023-11-24T06:08:58.531404+0000","last_undegraded":"2023-11-24T06:08:58.531404+0000","last_fullsized":"2023-11-24T06:08:58.531404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532611+0000","last_change":"2023-11-24T06:08:58.522901+0000","last_active":"2023-11-24T06:08:58.532611+0000","last_peered":"2023-11-24T06:08:58.532611+0000","last_clean":"2023-11-24T06:08:58.532611+0000","last_became_active":"2023-11-24T06:08:58.522620+0000","last_became_peered":"2023-11-24T06:08:58.522620+0000","last_unstale":"2023-11-24T06:08:58.532611+0000","last_undegraded":"2023-11-24T06:08:58.532611+0000","last_fullsized":"2023-11-24T06:08:58.532611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:18.279785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.523980+0000","last_change":"2023-11-24T06:08:58.519217+0000","last_active":"2023-11-24T06:08:58.523980+0000","last_peered":"2023-11-24T06:08:58.523980+0000","last_clean":"2023-11-24T06:08:58.523980+0000","last_became_active":"2023-11-24T06:08:58.518577+0000","last_became_peered":"2023-11-24T06:08:58.518577+0000","last_unstale":"2023-11-24T06:08:58.523980+0000","last_undegraded":"2023-11-24T06:08:58.523980+0000","last_fullsized":"2023-11-24T06:08:58.523980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527945+0000","last_change":"2023-11-24T06:08:58.523627+0000","last_active":"2023-11-24T06:08:58.527945+0000","last_peered":"2023-11-24T06:08:58.527945+0000","last_clean":"2023-11-24T06:08:58.527945+0000","last_became_active":"2023-11-24T06:08:58.522543+0000","last_became_peered":"2023-11-24T06:08:58.522543+0000","last_unstale":"2023-11-24T06:08:58.527945+0000","last_undegraded":"2023-11-24T06:08:58.527945+0000","last_fullsized":"2023-11-24T06:08:58.527945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.523745+0000","last_change":"2023-11-24T06:08:58.521811+0000","last_active":"2023-11-24T06:08:58.523745+0000","last_peered":"2023-11-24T06:08:58.523745+0000","last_clean":"2023-11-24T06:08:58.523745+0000","last_became_active":"2023-11-24T06:08:58.521129+0000","last_became_peered":"2023-11-24T06:08:58.521129+0000","last_unstale":"2023-11-24T06:08:58.523745+0000","last_undegraded":"2023-11-24T06:08:58.523745+0000","last_fullsized":"2023-11-24T06:08:58.523745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.541197+0000","last_change":"2023-11-24T06:08:58.539747+0000","last_active":"2023-11-24T06:08:58.541197+0000","last_peered":"2023-11-24T06:08:58.541197+0000","last_clean":"2023-11-24T06:08:58.541197+0000","last_became_active":"2023-11-24T06:08:58.539449+0000","last_became_peered":"2023-11-24T06:08:58.539449+0000","last_unstale":"2023-11-24T06:08:58.541197+0000","last_undegraded":"2023-11-24T06:08:58.541197+0000","last_fullsized":"2023-11-24T06:08:58.541197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.528315+0000","last_change":"2023-11-24T06:08:58.523632+0000","last_active":"2023-11-24T06:08:58.528315+0000","last_peered":"2023-11-24T06:08:58.528315+0000","last_clean":"2023-11-24T06:08:58.528315+0000","last_became_active":"2023-11-24T06:08:58.523365+0000","last_became_peered":"2023-11-24T06:08:58.523365+0000","last_unstale":"2023-11-24T06:08:58.528315+0000","last_undegraded":"2023-11-24T06:08:58.528315+0000","last_fullsized":"2023-11-24T06:08:58.528315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532681+0000","last_change":"2023-11-24T06:08:58.527956+0000","last_active":"2023-11-24T06:08:58.532681+0000","last_peered":"2023-11-24T06:08:58.532681+0000","last_clean":"2023-11-24T06:08:58.532681+0000","last_became_active":"2023-11-24T06:08:58.527700+0000","last_became_peered":"2023-11-24T06:08:58.527700+0000","last_unstale":"2023-11-24T06:08:58.532681+0000","last_undegraded":"2023-11-24T06:08:58.532681+0000","last_fullsized":"2023-11-24T06:08:58.532681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.530385+0000","last_change":"2023-11-24T06:08:58.529061+0000","last_active":"2023-11-24T06:08:58.530385+0000","last_peered":"2023-11-24T06:08:58.530385+0000","last_clean":"2023-11-24T06:08:58.530385+0000","last_became_active":"2023-11-24T06:08:58.528753+0000","last_became_peered":"2023-11-24T06:08:58.528753+0000","last_unstale":"2023-11-24T06:08:58.530385+0000","last_undegraded":"2023-11-24T06:08:58.530385+0000","last_fullsized":"2023-11-24T06:08:58.530385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543547+0000","last_change":"2023-11-24T06:08:58.541017+0000","last_active":"2023-11-24T06:08:58.543547+0000","last_peered":"2023-11-24T06:08:58.543547+0000","last_clean":"2023-11-24T06:08:58.543547+0000","last_became_active":"2023-11-24T06:08:58.540746+0000","last_became_peered":"2023-11-24T06:08:58.540746+0000","last_unstale":"2023-11-24T06:08:58.543547+0000","last_undegraded":"2023-11-24T06:08:58.543547+0000","last_fullsized":"2023-11-24T06:08:58.543547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:26.004865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543810+0000","last_change":"2023-11-24T06:08:58.540211+0000","last_active":"2023-11-24T06:08:58.543810+0000","last_peered":"2023-11-24T06:08:58.543810+0000","last_clean":"2023-11-24T06:08:58.543810+0000","last_became_active":"2023-11-24T06:08:58.539978+0000","last_became_peered":"2023-11-24T06:08:58.539978+0000","last_unstale":"2023-11-24T06:08:58.543810+0000","last_undegraded":"2023-11-24T06:08:58.543810+0000","last_fullsized":"2023-11-24T06:08:58.543810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534871+0000","last_change":"2023-11-24T06:08:58.529024+0000","last_active":"2023-11-24T06:08:58.534871+0000","last_peered":"2023-11-24T06:08:58.534871+0000","last_clean":"2023-11-24T06:08:58.534871+0000","last_became_active":"2023-11-24T06:08:58.528704+0000","last_became_peered":"2023-11-24T06:08:58.528704+0000","last_unstale":"2023-11-24T06:08:58.534871+0000","last_undegraded":"2023-11-24T06:08:58.534871+0000","last_fullsized":"2023-11-24T06:08:58.534871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":66222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:09:02.626 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:09:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.777+0000 7f4aaad0c700 1 Processor -- start 2023-11-24T06:09:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.778+0000 7f4aaad0c700 1 -- start start 2023-11-24T06:09:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.778+0000 7f4aaad0c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa40adf00 0x7f4aa406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.778+0000 7f4aaad0c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4aa406c540 0x7f4aa40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.778+0000 7f4aaad0c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa40af950 con 0x7f4aa41446a0 2023-11-24T06:09:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.778+0000 7f4aaad0c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa4144a80 con 0x7f4aa406c540 2023-11-24T06:09:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.778+0000 7f4aaad0c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa406c9b0 con 0x7f4aa40adf00 2023-11-24T06:09:02.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa3fff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa40adf00 0x7f4aa406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa92a9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4aa406c540 0x7f4aa40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa92a9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4aa406c540 0x7f4aa40b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39226/0 (socket says 172.21.15.97:39226) 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa8aa8700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f4aa41446a0 0x7f4aa40b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46836/0 (socket says 172.21.15.97:46836) 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa92a9700 1 -- 172.21.15.97:0/1970762919 learned_addr learned my addr 172.21.15.97:0/1970762919 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa3fff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa40adf00 0x7f4aa406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60332/0 (socket says 172.21.15.97:60332) 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa92a9700 1 -- 172.21.15.97:0/1970762919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa40adf00 msgr2=0x7f4aa406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa92a9700 1 --2- 172.21.15.97:0/1970762919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa40adf00 0x7f4aa406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa92a9700 1 -- 172.21.15.97:0/1970762919 >> v1:172.21.15.183:6789/0 conn(0x7f4aa41446a0 legacy=0x7f4aa40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa92a9700 1 -- 172.21.15.97:0/1970762919 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa40afc30 con 0x7f4aa406c540 2023-11-24T06:09:02.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa3fff700 1 --2- 172.21.15.97:0/1970762919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa40adf00 0x7f4aa406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa92a9700 1 --2- 172.21.15.97:0/1970762919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa406c540 0x7f4aa40b3d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009e80 tx=0x7f4a9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83bfd62fc0108aea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa37fe700 1 -- 172.21.15.97:0/1970762919 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00e070 con 0x7f4aa406c540 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa37fe700 1 -- 172.21.15.97:0/1970762919 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9c00b8d0 con 0x7f4aa406c540 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.779+0000 7f4aa37fe700 1 -- 172.21.15.97:0/1970762919 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9c00bc80 con 0x7f4aa406c540 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.780+0000 7f4aaad0c700 1 -- 172.21.15.97:0/1970762919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa406c540 msgr2=0x7f4aa40b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.780+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/1970762919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa406c540 0x7f4aa40b3d40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4a9c009e80 tx=0x7f4a9c00b330 comp rx=0 tx=0).stop 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.780+0000 7f4aaad0c700 1 -- 172.21.15.97:0/1970762919 shutdown_connections 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.780+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/1970762919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa406c540 0x7f4aa40b3d40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.780+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/1970762919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa40adf00 0x7f4aa406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.780+0000 7f4aaad0c700 1 -- 172.21.15.97:0/1970762919 >> 172.21.15.97:0/1970762919 conn(0x7f4aa4071b90 msgr2=0x7f4aa4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:02.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.780+0000 7f4aaad0c700 1 -- 172.21.15.97:0/1970762919 shutdown_connections 2023-11-24T06:09:02.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.780+0000 7f4aaad0c700 1 -- 172.21.15.97:0/1970762919 wait complete. 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.781+0000 7f4aaad0c700 1 Processor -- start 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.781+0000 7f4aaad0c700 1 -- start start 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aaad0c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa406c540 0x7f4aa40cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aaad0c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa40adf00 0x7f4aa409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aaad0c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa41446a0 0x7f4aa409af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aaad0c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41388b0 con 0x7f4aa40adf00 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aaad0c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa41385a0 con 0x7f4aa41446a0 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aaad0c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa40cf5a0 con 0x7f4aa406c540 2023-11-24T06:09:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa40adf00 0x7f4aa409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa40adf00 0x7f4aa409aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45068/0 (socket says 172.21.15.97:45068) 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa3fff700 1 -- 172.21.15.97:0/197100163 learned_addr learned my addr 172.21.15.97:0/197100163 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa92a9700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa41446a0 0x7f4aa409af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa8aa8700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa406c540 0x7f4aa40cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa3fff700 1 -- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa406c540 msgr2=0x7f4aa40cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa3fff700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa406c540 0x7f4aa40cf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa3fff700 1 -- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa41446a0 msgr2=0x7f4aa409af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa3fff700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa41446a0 0x7f4aa409af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa3fff700 1 -- 172.21.15.97:0/197100163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9000b740 con 0x7f4aa40adf00 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.782+0000 7f4aa92a9700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa41446a0 0x7f4aa409af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.783+0000 7f4aa3fff700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa40adf00 0x7f4aa409aa00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a90010f00 tx=0x7f4a9000e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.783+0000 7f4aa17fa700 1 -- 172.21.15.97:0/197100163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90011650 con 0x7f4aa40adf00 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.783+0000 7f4aa17fa700 1 -- 172.21.15.97:0/197100163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a900117b0 con 0x7f4aa40adf00 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.783+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c009b30 con 0x7f4aa40adf00 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.783+0000 7f4aa17fa700 1 -- 172.21.15.97:0/197100163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90011b40 con 0x7f4aa40adf00 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.783+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa40cfb00 con 0x7f4aa40adf00 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.784+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa40753f0 con 0x7f4aa40adf00 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.784+0000 7f4aa17fa700 1 -- 172.21.15.97:0/197100163 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a90011ca0 con 0x7f4aa40adf00 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.785+0000 7f4aa17fa700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a8806aed0 0x7f4a8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.785+0000 7f4aa17fa700 1 -- 172.21.15.97:0/197100163 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a9008c700 con 0x7f4aa40adf00 2023-11-24T06:09:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.785+0000 7f4aa8aa8700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a8806aed0 0x7f4a8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.785+0000 7f4aa8aa8700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a8806aed0 0x7f4a8806d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a98000f60 tx=0x7f4a98007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.789+0000 7f4aa17fa700 1 -- 172.21.15.97:0/197100163 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a90057920 con 0x7f4aa40adf00 2023-11-24T06:09:02.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.976+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4aa4069000 con 0x7f4a8806aed0 2023-11-24T06:09:02.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.983+0000 7f4aa17fa700 1 -- 172.21.15.97:0/197100163 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264561 (secure 0 0 0) 0x7f4aa4069000 con 0x7f4a8806aed0 2023-11-24T06:09:02.989 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:02.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a8806aed0 msgr2=0x7f4a8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a8806aed0 0x7f4a8806d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a98000f60 tx=0x7f4a98007d30 comp rx=0 tx=0).stop 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa40adf00 msgr2=0x7f4aa409aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa40adf00 0x7f4aa409aa00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a90010f00 tx=0x7f4a9000e640 comp rx=0 tx=0).stop 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 shutdown_connections 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4aa41446a0 0x7f4aa409af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a8806aed0 0x7f4a8806d390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4aa406c540 0x7f4aa40cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 --2- 172.21.15.97:0/197100163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa40adf00 0x7f4aa409aa00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:02.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 >> 172.21.15.97:0/197100163 conn(0x7f4aa4071b90 msgr2=0x7f4aa40b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 shutdown_connections 2023-11-24T06:09:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:02.987+0000 7f4aaad0c700 1 -- 172.21.15.97:0/197100163 wait complete. 2023-11-24T06:09:02.993 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:09:03.001 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T06:09:02.323701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218844,"kb_used_data":2660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755876,"statfs":{"total":773094113280,"available":772870017024,"internally_reserved":0,"allocated":2723840,"data_stored":1680213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.821623"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544347+0000","last_change":"2023-11-24T06:08:58.542671+0000","last_active":"2023-11-24T06:08:58.544347+0000","last_peered":"2023-11-24T06:08:58.544347+0000","last_clean":"2023-11-24T06:08:58.544347+0000","last_became_active":"2023-11-24T06:08:58.541937+0000","last_became_peered":"2023-11-24T06:08:58.541937+0000","last_unstale":"2023-11-24T06:08:58.544347+0000","last_undegraded":"2023-11-24T06:08:58.544347+0000","last_fullsized":"2023-11-24T06:08:58.544347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524219+0000","last_change":"2023-11-24T06:08:58.522085+0000","last_active":"2023-11-24T06:08:58.524219+0000","last_peered":"2023-11-24T06:08:58.524219+0000","last_clean":"2023-11-24T06:08:58.524219+0000","last_became_active":"2023-11-24T06:08:58.521116+0000","last_became_peered":"2023-11-24T06:08:58.521116+0000","last_unstale":"2023-11-24T06:08:58.524219+0000","last_undegraded":"2023-11-24T06:08:58.524219+0000","last_fullsized":"2023-11-24T06:08:58.524219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.521057+0000","last_change":"2023-11-24T06:08:58.515201+0000","last_active":"2023-11-24T06:08:58.521057+0000","last_peered":"2023-11-24T06:08:58.521057+0000","last_clean":"2023-11-24T06:08:58.521057+0000","last_became_active":"2023-11-24T06:08:58.514911+0000","last_became_peered":"2023-11-24T06:08:58.514911+0000","last_unstale":"2023-11-24T06:08:58.521057+0000","last_undegraded":"2023-11-24T06:08:58.521057+0000","last_fullsized":"2023-11-24T06:08:58.521057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527351+0000","last_change":"2023-11-24T06:08:58.520988+0000","last_active":"2023-11-24T06:08:58.527351+0000","last_peered":"2023-11-24T06:08:58.527351+0000","last_clean":"2023-11-24T06:08:58.527351+0000","last_became_active":"2023-11-24T06:08:58.520029+0000","last_became_peered":"2023-11-24T06:08:58.520029+0000","last_unstale":"2023-11-24T06:08:58.527351+0000","last_undegraded":"2023-11-24T06:08:58.527351+0000","last_fullsized":"2023-11-24T06:08:58.527351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.542169+0000","last_change":"2023-11-24T06:08:58.535989+0000","last_active":"2023-11-24T06:08:58.542169+0000","last_peered":"2023-11-24T06:08:58.542169+0000","last_clean":"2023-11-24T06:08:58.542169+0000","last_became_active":"2023-11-24T06:08:58.535697+0000","last_became_peered":"2023-11-24T06:08:58.535697+0000","last_unstale":"2023-11-24T06:08:58.542169+0000","last_undegraded":"2023-11-24T06:08:58.542169+0000","last_fullsized":"2023-11-24T06:08:58.542169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:40.002717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532634+0000","last_change":"2023-11-24T06:08:58.523237+0000","last_active":"2023-11-24T06:08:58.532634+0000","last_peered":"2023-11-24T06:08:58.532634+0000","last_clean":"2023-11-24T06:08:58.532634+0000","last_became_active":"2023-11-24T06:08:58.522368+0000","last_became_peered":"2023-11-24T06:08:58.522368+0000","last_unstale":"2023-11-24T06:08:58.532634+0000","last_undegraded":"2023-11-24T06:08:58.532634+0000","last_fullsized":"2023-11-24T06:08:58.532634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536344+0000","last_change":"2023-11-24T06:08:58.533834+0000","last_active":"2023-11-24T06:08:58.536344+0000","last_peered":"2023-11-24T06:08:58.536344+0000","last_clean":"2023-11-24T06:08:58.536344+0000","last_became_active":"2023-11-24T06:08:58.533539+0000","last_became_peered":"2023-11-24T06:08:58.533539+0000","last_unstale":"2023-11-24T06:08:58.536344+0000","last_undegraded":"2023-11-24T06:08:58.536344+0000","last_fullsized":"2023-11-24T06:08:58.536344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534044+0000","last_change":"2023-11-24T06:08:58.531528+0000","last_active":"2023-11-24T06:08:58.534044+0000","last_peered":"2023-11-24T06:08:58.534044+0000","last_clean":"2023-11-24T06:08:58.534044+0000","last_became_active":"2023-11-24T06:08:58.531144+0000","last_became_peered":"2023-11-24T06:08:58.531144+0000","last_unstale":"2023-11-24T06:08:58.534044+0000","last_undegraded":"2023-11-24T06:08:58.534044+0000","last_fullsized":"2023-11-24T06:08:58.534044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:40.002717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544372+0000","last_change":"2023-11-24T06:08:58.542860+0000","last_active":"2023-11-24T06:08:58.544372+0000","last_peered":"2023-11-24T06:08:58.544372+0000","last_clean":"2023-11-24T06:08:58.544372+0000","last_became_active":"2023-11-24T06:08:58.542347+0000","last_became_peered":"2023-11-24T06:08:58.542347+0000","last_unstale":"2023-11-24T06:08:58.544372+0000","last_undegraded":"2023-11-24T06:08:58.544372+0000","last_fullsized":"2023-11-24T06:08:58.544372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533871+0000","last_change":"2023-11-24T06:08:58.529308+0000","last_active":"2023-11-24T06:08:58.533871+0000","last_peered":"2023-11-24T06:08:58.533871+0000","last_clean":"2023-11-24T06:08:58.533871+0000","last_became_active":"2023-11-24T06:08:58.528986+0000","last_became_peered":"2023-11-24T06:08:58.528986+0000","last_unstale":"2023-11-24T06:08:58.533871+0000","last_undegraded":"2023-11-24T06:08:58.533871+0000","last_fullsized":"2023-11-24T06:08:58.533871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534184+0000","last_change":"2023-11-24T06:08:58.530560+0000","last_active":"2023-11-24T06:08:58.534184+0000","last_peered":"2023-11-24T06:08:58.534184+0000","last_clean":"2023-11-24T06:08:58.534184+0000","last_became_active":"2023-11-24T06:08:58.530299+0000","last_became_peered":"2023-11-24T06:08:58.530299+0000","last_unstale":"2023-11-24T06:08:58.534184+0000","last_undegraded":"2023-11-24T06:08:58.534184+0000","last_fullsized":"2023-11-24T06:08:58.534184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.531369+0000","last_change":"2023-11-24T06:08:58.529565+0000","last_active":"2023-11-24T06:08:58.531369+0000","last_peered":"2023-11-24T06:08:58.531369+0000","last_clean":"2023-11-24T06:08:58.531369+0000","last_became_active":"2023-11-24T06:08:58.529170+0000","last_became_peered":"2023-11-24T06:08:58.529170+0000","last_unstale":"2023-11-24T06:08:58.531369+0000","last_undegraded":"2023-11-24T06:08:58.531369+0000","last_fullsized":"2023-11-24T06:08:58.531369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527282+0000","last_change":"2023-11-24T06:08:58.521255+0000","last_active":"2023-11-24T06:08:58.527282+0000","last_peered":"2023-11-24T06:08:58.527282+0000","last_clean":"2023-11-24T06:08:58.527282+0000","last_became_active":"2023-11-24T06:08:58.520950+0000","last_became_peered":"2023-11-24T06:08:58.520950+0000","last_unstale":"2023-11-24T06:08:58.527282+0000","last_undegraded":"2023-11-24T06:08:58.527282+0000","last_fullsized":"2023-11-24T06:08:58.527282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:18.279785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527539+0000","last_change":"2023-11-24T06:08:58.525412+0000","last_active":"2023-11-24T06:08:58.527539+0000","last_peered":"2023-11-24T06:08:58.527539+0000","last_clean":"2023-11-24T06:08:58.527539+0000","last_became_active":"2023-11-24T06:08:58.525080+0000","last_became_peered":"2023-11-24T06:08:58.525080+0000","last_unstale":"2023-11-24T06:08:58.527539+0000","last_undegraded":"2023-11-24T06:08:58.527539+0000","last_fullsized":"2023-11-24T06:08:58.527539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533529+0000","last_change":"2023-11-24T06:08:58.529101+0000","last_active":"2023-11-24T06:08:58.533529+0000","last_peered":"2023-11-24T06:08:58.533529+0000","last_clean":"2023-11-24T06:08:58.533529+0000","last_became_active":"2023-11-24T06:08:58.528814+0000","last_became_peered":"2023-11-24T06:08:58.528814+0000","last_unstale":"2023-11-24T06:08:58.533529+0000","last_undegraded":"2023-11-24T06:08:58.533529+0000","last_fullsized":"2023-11-24T06:08:58.533529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532662+0000","last_change":"2023-11-24T06:08:58.523287+0000","last_active":"2023-11-24T06:08:58.532662+0000","last_peered":"2023-11-24T06:08:58.532662+0000","last_clean":"2023-11-24T06:08:58.532662+0000","last_became_active":"2023-11-24T06:08:58.523033+0000","last_became_peered":"2023-11-24T06:08:58.523033+0000","last_unstale":"2023-11-24T06:08:58.532662+0000","last_undegraded":"2023-11-24T06:08:58.532662+0000","last_fullsized":"2023-11-24T06:08:58.532662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544356+0000","last_change":"2023-11-24T06:08:58.543046+0000","last_active":"2023-11-24T06:08:58.544356+0000","last_peered":"2023-11-24T06:08:58.544356+0000","last_clean":"2023-11-24T06:08:58.544356+0000","last_became_active":"2023-11-24T06:08:58.542703+0000","last_became_peered":"2023-11-24T06:08:58.542703+0000","last_unstale":"2023-11-24T06:08:58.544356+0000","last_undegraded":"2023-11-24T06:08:58.544356+0000","last_fullsized":"2023-11-24T06:08:58.544356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524239+0000","last_change":"2023-11-24T06:08:58.521888+0000","last_active":"2023-11-24T06:08:58.524239+0000","last_peered":"2023-11-24T06:08:58.524239+0000","last_clean":"2023-11-24T06:08:58.524239+0000","last_became_active":"2023-11-24T06:08:58.521599+0000","last_became_peered":"2023-11-24T06:08:58.521599+0000","last_unstale":"2023-11-24T06:08:58.524239+0000","last_undegraded":"2023-11-24T06:08:58.524239+0000","last_fullsized":"2023-11-24T06:08:58.524239+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.542143+0000","last_change":"2023-11-24T06:08:58.536029+0000","last_active":"2023-11-24T06:08:58.542143+0000","last_peered":"2023-11-24T06:08:58.542143+0000","last_clean":"2023-11-24T06:08:58.542143+0000","last_became_active":"2023-11-24T06:08:58.535076+0000","last_became_peered":"2023-11-24T06:08:58.535076+0000","last_unstale":"2023-11-24T06:08:58.542143+0000","last_undegraded":"2023-11-24T06:08:58.542143+0000","last_fullsized":"2023-11-24T06:08:58.542143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544489+0000","last_change":"2023-11-24T06:08:57.529523+0000","last_active":"2023-11-24T06:08:58.544489+0000","last_peered":"2023-11-24T06:08:58.544489+0000","last_clean":"2023-11-24T06:08:58.544489+0000","last_became_active":"2023-11-24T06:08:57.529207+0000","last_became_peered":"2023-11-24T06:08:57.529207+0000","last_unstale":"2023-11-24T06:08:58.544489+0000","last_undegraded":"2023-11-24T06:08:58.544489+0000","last_fullsized":"2023-11-24T06:08:58.544489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:52.525711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527321+0000","last_change":"2023-11-24T06:08:58.523242+0000","last_active":"2023-11-24T06:08:58.527321+0000","last_peered":"2023-11-24T06:08:58.527321+0000","last_clean":"2023-11-24T06:08:58.527321+0000","last_became_active":"2023-11-24T06:08:58.522951+0000","last_became_peered":"2023-11-24T06:08:58.522951+0000","last_unstale":"2023-11-24T06:08:58.527321+0000","last_undegraded":"2023-11-24T06:08:58.527321+0000","last_fullsized":"2023-11-24T06:08:58.527321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534322+0000","last_change":"2023-11-24T06:08:57.523196+0000","last_active":"2023-11-24T06:08:58.534322+0000","last_peered":"2023-11-24T06:08:58.534322+0000","last_clean":"2023-11-24T06:08:58.534322+0000","last_became_active":"2023-11-24T06:08:57.522871+0000","last_became_peered":"2023-11-24T06:08:57.522871+0000","last_unstale":"2023-11-24T06:08:58.534322+0000","last_undegraded":"2023-11-24T06:08:58.534322+0000","last_fullsized":"2023-11-24T06:08:58.534322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:22.630593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536081+0000","last_change":"2023-11-24T06:08:58.530832+0000","last_active":"2023-11-24T06:08:58.536081+0000","last_peered":"2023-11-24T06:08:58.536081+0000","last_clean":"2023-11-24T06:08:58.536081+0000","last_became_active":"2023-11-24T06:08:58.530383+0000","last_became_peered":"2023-11-24T06:08:58.530383+0000","last_unstale":"2023-11-24T06:08:58.536081+0000","last_undegraded":"2023-11-24T06:08:58.536081+0000","last_fullsized":"2023-11-24T06:08:58.536081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.539542+0000","last_change":"2023-11-24T06:08:57.528776+0000","last_active":"2023-11-24T06:08:58.539542+0000","last_peered":"2023-11-24T06:08:58.539542+0000","last_clean":"2023-11-24T06:08:58.539542+0000","last_became_active":"2023-11-24T06:08:57.528480+0000","last_became_peered":"2023-11-24T06:08:57.528480+0000","last_unstale":"2023-11-24T06:08:58.539542+0000","last_undegraded":"2023-11-24T06:08:58.539542+0000","last_fullsized":"2023-11-24T06:08:58.539542+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532581+0000","last_change":"2023-11-24T06:08:58.524300+0000","last_active":"2023-11-24T06:08:58.532581+0000","last_peered":"2023-11-24T06:08:58.532581+0000","last_clean":"2023-11-24T06:08:58.532581+0000","last_became_active":"2023-11-24T06:08:58.522378+0000","last_became_peered":"2023-11-24T06:08:58.522378+0000","last_unstale":"2023-11-24T06:08:58.532581+0000","last_undegraded":"2023-11-24T06:08:58.532581+0000","last_fullsized":"2023-11-24T06:08:58.532581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544105+0000","last_change":"2023-11-24T06:08:57.526272+0000","last_active":"2023-11-24T06:08:58.544105+0000","last_peered":"2023-11-24T06:08:58.544105+0000","last_clean":"2023-11-24T06:08:58.544105+0000","last_became_active":"2023-11-24T06:08:57.525998+0000","last_became_peered":"2023-11-24T06:08:57.525998+0000","last_unstale":"2023-11-24T06:08:58.544105+0000","last_undegraded":"2023-11-24T06:08:58.544105+0000","last_fullsized":"2023-11-24T06:08:58.544105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533283+0000","last_change":"2023-11-24T06:08:58.523547+0000","last_active":"2023-11-24T06:08:58.533283+0000","last_peered":"2023-11-24T06:08:58.533283+0000","last_clean":"2023-11-24T06:08:58.533283+0000","last_became_active":"2023-11-24T06:08:58.522784+0000","last_became_peered":"2023-11-24T06:08:58.522784+0000","last_unstale":"2023-11-24T06:08:58.533283+0000","last_undegraded":"2023-11-24T06:08:58.533283+0000","last_fullsized":"2023-11-24T06:08:58.533283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.539080+0000","last_change":"2023-11-24T06:08:57.528824+0000","last_active":"2023-11-24T06:08:58.539080+0000","last_peered":"2023-11-24T06:08:58.539080+0000","last_clean":"2023-11-24T06:08:58.539080+0000","last_became_active":"2023-11-24T06:08:57.528517+0000","last_became_peered":"2023-11-24T06:08:57.528517+0000","last_unstale":"2023-11-24T06:08:58.539080+0000","last_undegraded":"2023-11-24T06:08:58.539080+0000","last_fullsized":"2023-11-24T06:08:58.539080+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527092+0000","last_change":"2023-11-24T06:08:58.523181+0000","last_active":"2023-11-24T06:08:58.527092+0000","last_peered":"2023-11-24T06:08:58.527092+0000","last_clean":"2023-11-24T06:08:58.527092+0000","last_became_active":"2023-11-24T06:08:58.522917+0000","last_became_peered":"2023-11-24T06:08:58.522917+0000","last_unstale":"2023-11-24T06:08:58.527092+0000","last_undegraded":"2023-11-24T06:08:58.527092+0000","last_fullsized":"2023-11-24T06:08:58.527092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536672+0000","last_change":"2023-11-24T06:08:57.524473+0000","last_active":"2023-11-24T06:08:58.536672+0000","last_peered":"2023-11-24T06:08:58.536672+0000","last_clean":"2023-11-24T06:08:58.536672+0000","last_became_active":"2023-11-24T06:08:57.523947+0000","last_became_peered":"2023-11-24T06:08:57.523947+0000","last_unstale":"2023-11-24T06:08:58.536672+0000","last_undegraded":"2023-11-24T06:08:58.536672+0000","last_fullsized":"2023-11-24T06:08:58.536672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.529518+0000","last_change":"2023-11-24T06:08:58.526170+0000","last_active":"2023-11-24T06:08:58.529518+0000","last_peered":"2023-11-24T06:08:58.529518+0000","last_clean":"2023-11-24T06:08:58.529518+0000","last_became_active":"2023-11-24T06:08:58.525874+0000","last_became_peered":"2023-11-24T06:08:58.525874+0000","last_unstale":"2023-11-24T06:08:58.529518+0000","last_undegraded":"2023-11-24T06:08:58.529518+0000","last_fullsized":"2023-11-24T06:08:58.529518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.511356+0000","last_change":"2023-11-24T06:08:57.520895+0000","last_active":"2023-11-24T06:08:58.511356+0000","last_peered":"2023-11-24T06:08:58.511356+0000","last_clean":"2023-11-24T06:08:58.511356+0000","last_became_active":"2023-11-24T06:08:57.520367+0000","last_became_peered":"2023-11-24T06:08:57.520367+0000","last_unstale":"2023-11-24T06:08:58.511356+0000","last_undegraded":"2023-11-24T06:08:58.511356+0000","last_fullsized":"2023-11-24T06:08:58.511356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:55.763227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.525781+0000","last_change":"2023-11-24T06:08:58.519434+0000","last_active":"2023-11-24T06:08:58.525781+0000","last_peered":"2023-11-24T06:08:58.525781+0000","last_clean":"2023-11-24T06:08:58.525781+0000","last_became_active":"2023-11-24T06:08:58.519148+0000","last_became_peered":"2023-11-24T06:08:58.519148+0000","last_unstale":"2023-11-24T06:08:58.525781+0000","last_undegraded":"2023-11-24T06:08:58.525781+0000","last_fullsized":"2023-11-24T06:08:58.525781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.514508+0000","last_change":"2023-11-24T06:08:57.525692+0000","last_active":"2023-11-24T06:08:58.514508+0000","last_peered":"2023-11-24T06:08:58.514508+0000","last_clean":"2023-11-24T06:08:58.514508+0000","last_became_active":"2023-11-24T06:08:57.525451+0000","last_became_peered":"2023-11-24T06:08:57.525451+0000","last_unstale":"2023-11-24T06:08:58.514508+0000","last_undegraded":"2023-11-24T06:08:58.514508+0000","last_fullsized":"2023-11-24T06:08:58.514508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.523690+0000","last_change":"2023-11-24T06:08:58.521741+0000","last_active":"2023-11-24T06:08:58.523690+0000","last_peered":"2023-11-24T06:08:58.523690+0000","last_clean":"2023-11-24T06:08:58.523690+0000","last_became_active":"2023-11-24T06:08:58.521275+0000","last_became_peered":"2023-11-24T06:08:58.521275+0000","last_unstale":"2023-11-24T06:08:58.523690+0000","last_undegraded":"2023-11-24T06:08:58.523690+0000","last_fullsized":"2023-11-24T06:08:58.523690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.519500+0000","last_change":"2023-11-24T06:08:57.526466+0000","last_active":"2023-11-24T06:08:58.519500+0000","last_peered":"2023-11-24T06:08:58.519500+0000","last_clean":"2023-11-24T06:08:58.519500+0000","last_became_active":"2023-11-24T06:08:57.526174+0000","last_became_peered":"2023-11-24T06:08:57.526174+0000","last_unstale":"2023-11-24T06:08:58.519500+0000","last_undegraded":"2023-11-24T06:08:58.519500+0000","last_fullsized":"2023-11-24T06:08:58.519500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543843+0000","last_change":"2023-11-24T06:08:58.541089+0000","last_active":"2023-11-24T06:08:58.543843+0000","last_peered":"2023-11-24T06:08:58.543843+0000","last_clean":"2023-11-24T06:08:58.543843+0000","last_became_active":"2023-11-24T06:08:58.540792+0000","last_became_peered":"2023-11-24T06:08:58.540792+0000","last_unstale":"2023-11-24T06:08:58.543843+0000","last_undegraded":"2023-11-24T06:08:58.543843+0000","last_fullsized":"2023-11-24T06:08:58.543843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.518909+0000","last_change":"2023-11-24T06:08:57.525181+0000","last_active":"2023-11-24T06:08:58.518909+0000","last_peered":"2023-11-24T06:08:58.518909+0000","last_clean":"2023-11-24T06:08:58.518909+0000","last_became_active":"2023-11-24T06:08:57.524906+0000","last_became_peered":"2023-11-24T06:08:57.524906+0000","last_unstale":"2023-11-24T06:08:58.518909+0000","last_undegraded":"2023-11-24T06:08:58.518909+0000","last_fullsized":"2023-11-24T06:08:58.518909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536042+0000","last_change":"2023-11-24T06:08:58.530664+0000","last_active":"2023-11-24T06:08:58.536042+0000","last_peered":"2023-11-24T06:08:58.536042+0000","last_clean":"2023-11-24T06:08:58.536042+0000","last_became_active":"2023-11-24T06:08:58.530068+0000","last_became_peered":"2023-11-24T06:08:58.530068+0000","last_unstale":"2023-11-24T06:08:58.536042+0000","last_undegraded":"2023-11-24T06:08:58.536042+0000","last_fullsized":"2023-11-24T06:08:58.536042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534175+0000","last_change":"2023-11-24T06:08:57.525584+0000","last_active":"2023-11-24T06:08:58.534175+0000","last_peered":"2023-11-24T06:08:58.534175+0000","last_clean":"2023-11-24T06:08:58.534175+0000","last_became_active":"2023-11-24T06:08:57.525340+0000","last_became_peered":"2023-11-24T06:08:57.525340+0000","last_unstale":"2023-11-24T06:08:58.534175+0000","last_undegraded":"2023-11-24T06:08:58.534175+0000","last_fullsized":"2023-11-24T06:08:58.534175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.539564+0000","last_change":"2023-11-24T06:08:58.537817+0000","last_active":"2023-11-24T06:08:58.539564+0000","last_peered":"2023-11-24T06:08:58.539564+0000","last_clean":"2023-11-24T06:08:58.539564+0000","last_became_active":"2023-11-24T06:08:58.537604+0000","last_became_peered":"2023-11-24T06:08:58.537604+0000","last_unstale":"2023-11-24T06:08:58.539564+0000","last_undegraded":"2023-11-24T06:08:58.539564+0000","last_fullsized":"2023-11-24T06:08:58.539564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544228+0000","last_change":"2023-11-24T06:08:57.520656+0000","last_active":"2023-11-24T06:08:58.544228+0000","last_peered":"2023-11-24T06:08:58.544228+0000","last_clean":"2023-11-24T06:08:58.544228+0000","last_became_active":"2023-11-24T06:08:57.520398+0000","last_became_peered":"2023-11-24T06:08:57.520398+0000","last_unstale":"2023-11-24T06:08:58.544228+0000","last_undegraded":"2023-11-24T06:08:58.544228+0000","last_fullsized":"2023-11-24T06:08:58.544228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536322+0000","last_change":"2023-11-24T06:08:58.533919+0000","last_active":"2023-11-24T06:08:58.536322+0000","last_peered":"2023-11-24T06:08:58.536322+0000","last_clean":"2023-11-24T06:08:58.536322+0000","last_became_active":"2023-11-24T06:08:58.533565+0000","last_became_peered":"2023-11-24T06:08:58.533565+0000","last_unstale":"2023-11-24T06:08:58.536322+0000","last_undegraded":"2023-11-24T06:08:58.536322+0000","last_fullsized":"2023-11-24T06:08:58.536322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.537707+0000","last_change":"2023-11-24T06:08:57.523298+0000","last_active":"2023-11-24T06:08:58.537707+0000","last_peered":"2023-11-24T06:08:58.537707+0000","last_clean":"2023-11-24T06:08:58.537707+0000","last_became_active":"2023-11-24T06:08:57.523000+0000","last_became_peered":"2023-11-24T06:08:57.523000+0000","last_unstale":"2023-11-24T06:08:58.537707+0000","last_undegraded":"2023-11-24T06:08:58.537707+0000","last_fullsized":"2023-11-24T06:08:58.537707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543784+0000","last_change":"2023-11-24T06:08:58.539907+0000","last_active":"2023-11-24T06:08:58.543784+0000","last_peered":"2023-11-24T06:08:58.543784+0000","last_clean":"2023-11-24T06:08:58.543784+0000","last_became_active":"2023-11-24T06:08:58.539663+0000","last_became_peered":"2023-11-24T06:08:58.539663+0000","last_unstale":"2023-11-24T06:08:58.543784+0000","last_undegraded":"2023-11-24T06:08:58.543784+0000","last_fullsized":"2023-11-24T06:08:58.543784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:51.910530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.519678+0000","last_change":"2023-11-24T06:08:57.523074+0000","last_active":"2023-11-24T06:08:58.519678+0000","last_peered":"2023-11-24T06:08:58.519678+0000","last_clean":"2023-11-24T06:08:58.519678+0000","last_became_active":"2023-11-24T06:08:57.522527+0000","last_became_peered":"2023-11-24T06:08:57.522527+0000","last_unstale":"2023-11-24T06:08:58.519678+0000","last_undegraded":"2023-11-24T06:08:58.519678+0000","last_fullsized":"2023-11-24T06:08:58.519678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:31.591882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.542190+0000","last_change":"2023-11-24T06:08:58.538966+0000","last_active":"2023-11-24T06:08:58.542190+0000","last_peered":"2023-11-24T06:08:58.542190+0000","last_clean":"2023-11-24T06:08:58.542190+0000","last_became_active":"2023-11-24T06:08:58.538575+0000","last_became_peered":"2023-11-24T06:08:58.538575+0000","last_unstale":"2023-11-24T06:08:58.542190+0000","last_undegraded":"2023-11-24T06:08:58.542190+0000","last_fullsized":"2023-11-24T06:08:58.542190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:51.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.518534+0000","last_change":"2023-11-24T06:08:57.524582+0000","last_active":"2023-11-24T06:08:58.518534+0000","last_peered":"2023-11-24T06:08:58.518534+0000","last_clean":"2023-11-24T06:08:58.518534+0000","last_became_active":"2023-11-24T06:08:57.524351+0000","last_became_peered":"2023-11-24T06:08:57.524351+0000","last_unstale":"2023-11-24T06:08:58.518534+0000","last_undegraded":"2023-11-24T06:08:58.518534+0000","last_fullsized":"2023-11-24T06:08:58.518534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:22.630593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534253+0000","last_change":"2023-11-24T06:08:58.528140+0000","last_active":"2023-11-24T06:08:58.534253+0000","last_peered":"2023-11-24T06:08:58.534253+0000","last_clean":"2023-11-24T06:08:58.534253+0000","last_became_active":"2023-11-24T06:08:58.527685+0000","last_became_peered":"2023-11-24T06:08:58.527685+0000","last_unstale":"2023-11-24T06:08:58.534253+0000","last_undegraded":"2023-11-24T06:08:58.534253+0000","last_fullsized":"2023-11-24T06:08:58.534253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.541509+0000","last_change":"2023-11-24T06:08:57.519672+0000","last_active":"2023-11-24T06:08:58.541509+0000","last_peered":"2023-11-24T06:08:58.541509+0000","last_clean":"2023-11-24T06:08:58.541509+0000","last_became_active":"2023-11-24T06:08:57.519379+0000","last_became_peered":"2023-11-24T06:08:57.519379+0000","last_unstale":"2023-11-24T06:08:58.541509+0000","last_undegraded":"2023-11-24T06:08:58.541509+0000","last_fullsized":"2023-11-24T06:08:58.541509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:43.426568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536322+0000","last_change":"2023-11-24T06:08:58.531525+0000","last_active":"2023-11-24T06:08:58.536322+0000","last_peered":"2023-11-24T06:08:58.536322+0000","last_clean":"2023-11-24T06:08:58.536322+0000","last_became_active":"2023-11-24T06:08:58.529801+0000","last_became_peered":"2023-11-24T06:08:58.529801+0000","last_unstale":"2023-11-24T06:08:58.536322+0000","last_undegraded":"2023-11-24T06:08:58.536322+0000","last_fullsized":"2023-11-24T06:08:58.536322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.539009+0000","last_change":"2023-11-24T06:08:57.523021+0000","last_active":"2023-11-24T06:08:58.539009+0000","last_peered":"2023-11-24T06:08:58.539009+0000","last_clean":"2023-11-24T06:08:58.539009+0000","last_became_active":"2023-11-24T06:08:57.522474+0000","last_became_peered":"2023-11-24T06:08:57.522474+0000","last_unstale":"2023-11-24T06:08:58.539009+0000","last_undegraded":"2023-11-24T06:08:58.539009+0000","last_fullsized":"2023-11-24T06:08:58.539009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:43.891317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.541596+0000","last_change":"2023-11-24T06:08:58.535871+0000","last_active":"2023-11-24T06:08:58.541596+0000","last_peered":"2023-11-24T06:08:58.541596+0000","last_clean":"2023-11-24T06:08:58.541596+0000","last_became_active":"2023-11-24T06:08:58.535563+0000","last_became_peered":"2023-11-24T06:08:58.535563+0000","last_unstale":"2023-11-24T06:08:58.541596+0000","last_undegraded":"2023-11-24T06:08:58.541596+0000","last_fullsized":"2023-11-24T06:08:58.541596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.517025+0000","last_change":"2023-11-24T06:08:57.519605+0000","last_active":"2023-11-24T06:08:58.517025+0000","last_peered":"2023-11-24T06:08:58.517025+0000","last_clean":"2023-11-24T06:08:58.517025+0000","last_became_active":"2023-11-24T06:08:57.519308+0000","last_became_peered":"2023-11-24T06:08:57.519308+0000","last_unstale":"2023-11-24T06:08:58.517025+0000","last_undegraded":"2023-11-24T06:08:58.517025+0000","last_fullsized":"2023-11-24T06:08:58.517025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533303+0000","last_change":"2023-11-24T06:08:58.523994+0000","last_active":"2023-11-24T06:08:58.533303+0000","last_peered":"2023-11-24T06:08:58.533303+0000","last_clean":"2023-11-24T06:08:58.533303+0000","last_became_active":"2023-11-24T06:08:58.523750+0000","last_became_peered":"2023-11-24T06:08:58.523750+0000","last_unstale":"2023-11-24T06:08:58.533303+0000","last_undegraded":"2023-11-24T06:08:58.533303+0000","last_fullsized":"2023-11-24T06:08:58.533303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.520407+0000","last_change":"2023-11-24T06:08:57.525492+0000","last_active":"2023-11-24T06:08:58.520407+0000","last_peered":"2023-11-24T06:08:58.520407+0000","last_clean":"2023-11-24T06:08:58.520407+0000","last_became_active":"2023-11-24T06:08:57.525231+0000","last_became_peered":"2023-11-24T06:08:57.525231+0000","last_unstale":"2023-11-24T06:08:58.520407+0000","last_undegraded":"2023-11-24T06:08:58.520407+0000","last_fullsized":"2023-11-24T06:08:58.520407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:35.934471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533028+0000","last_change":"2023-11-24T06:08:58.525233+0000","last_active":"2023-11-24T06:08:58.533028+0000","last_peered":"2023-11-24T06:08:58.533028+0000","last_clean":"2023-11-24T06:08:58.533028+0000","last_became_active":"2023-11-24T06:08:58.524919+0000","last_became_peered":"2023-11-24T06:08:58.524919+0000","last_unstale":"2023-11-24T06:08:58.533028+0000","last_undegraded":"2023-11-24T06:08:58.533028+0000","last_fullsized":"2023-11-24T06:08:58.533028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.523752+0000","last_change":"2023-11-24T06:08:58.521720+0000","last_active":"2023-11-24T06:08:58.523752+0000","last_peered":"2023-11-24T06:08:58.523752+0000","last_clean":"2023-11-24T06:08:58.523752+0000","last_became_active":"2023-11-24T06:08:58.521358+0000","last_became_peered":"2023-11-24T06:08:58.521358+0000","last_unstale":"2023-11-24T06:08:58.523752+0000","last_undegraded":"2023-11-24T06:08:58.523752+0000","last_fullsized":"2023-11-24T06:08:58.523752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.510094+0000","last_change":"2023-11-24T06:08:57.521087+0000","last_active":"2023-11-24T06:08:58.510094+0000","last_peered":"2023-11-24T06:08:58.510094+0000","last_clean":"2023-11-24T06:08:58.510094+0000","last_became_active":"2023-11-24T06:08:57.520758+0000","last_became_peered":"2023-11-24T06:08:57.520758+0000","last_unstale":"2023-11-24T06:08:58.510094+0000","last_undegraded":"2023-11-24T06:08:58.510094+0000","last_fullsized":"2023-11-24T06:08:58.510094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:28.366035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533816+0000","last_change":"2023-11-24T06:08:58.531268+0000","last_active":"2023-11-24T06:08:58.533816+0000","last_peered":"2023-11-24T06:08:58.533816+0000","last_clean":"2023-11-24T06:08:58.533816+0000","last_became_active":"2023-11-24T06:08:58.530982+0000","last_became_peered":"2023-11-24T06:08:58.530982+0000","last_unstale":"2023-11-24T06:08:58.533816+0000","last_undegraded":"2023-11-24T06:08:58.533816+0000","last_fullsized":"2023-11-24T06:08:58.533816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.514439+0000","last_change":"2023-11-24T06:08:57.519233+0000","last_active":"2023-11-24T06:08:58.514439+0000","last_peered":"2023-11-24T06:08:58.514439+0000","last_clean":"2023-11-24T06:08:58.514439+0000","last_became_active":"2023-11-24T06:08:57.518413+0000","last_became_peered":"2023-11-24T06:08:57.518413+0000","last_unstale":"2023-11-24T06:08:58.514439+0000","last_undegraded":"2023-11-24T06:08:58.514439+0000","last_fullsized":"2023-11-24T06:08:58.514439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-11-24T06:08:57.516151+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:08:57.516151+0000","last_peered":"2023-11-24T06:08:57.516151+0000","last_clean":"2023-11-24T06:08:57.516151+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:08:57.516151+0000","last_undegraded":"2023-11-24T06:08:57.516151+0000","last_fullsized":"2023-11-24T06:08:57.516151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.530681+0000","last_change":"2023-11-24T06:08:58.528408+0000","last_active":"2023-11-24T06:08:58.530681+0000","last_peered":"2023-11-24T06:08:58.530681+0000","last_clean":"2023-11-24T06:08:58.530681+0000","last_became_active":"2023-11-24T06:08:58.528125+0000","last_became_peered":"2023-11-24T06:08:58.528125+0000","last_unstale":"2023-11-24T06:08:58.530681+0000","last_undegraded":"2023-11-24T06:08:58.530681+0000","last_fullsized":"2023-11-24T06:08:58.530681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:03.307275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.516395+0000","last_change":"2023-11-24T06:08:57.527351+0000","last_active":"2023-11-24T06:08:58.516395+0000","last_peered":"2023-11-24T06:08:58.516395+0000","last_clean":"2023-11-24T06:08:58.516395+0000","last_became_active":"2023-11-24T06:08:57.527085+0000","last_became_peered":"2023-11-24T06:08:57.527085+0000","last_unstale":"2023-11-24T06:08:58.516395+0000","last_undegraded":"2023-11-24T06:08:58.516395+0000","last_fullsized":"2023-11-24T06:08:58.516395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.520674+0000","last_change":"2023-11-24T06:08:57.528763+0000","last_active":"2023-11-24T06:08:58.520674+0000","last_peered":"2023-11-24T06:08:58.520674+0000","last_clean":"2023-11-24T06:08:58.520674+0000","last_became_active":"2023-11-24T06:08:57.528469+0000","last_became_peered":"2023-11-24T06:08:57.528469+0000","last_unstale":"2023-11-24T06:08:58.520674+0000","last_undegraded":"2023-11-24T06:08:58.520674+0000","last_fullsized":"2023-11-24T06:08:58.520674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524016+0000","last_change":"2023-11-24T06:08:58.522186+0000","last_active":"2023-11-24T06:08:58.524016+0000","last_peered":"2023-11-24T06:08:58.524016+0000","last_clean":"2023-11-24T06:08:58.524016+0000","last_became_active":"2023-11-24T06:08:58.521844+0000","last_became_peered":"2023-11-24T06:08:58.521844+0000","last_unstale":"2023-11-24T06:08:58.524016+0000","last_undegraded":"2023-11-24T06:08:58.524016+0000","last_fullsized":"2023-11-24T06:08:58.524016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.517366+0000","last_change":"2023-11-24T06:08:57.519266+0000","last_active":"2023-11-24T06:08:58.517366+0000","last_peered":"2023-11-24T06:08:58.517366+0000","last_clean":"2023-11-24T06:08:58.517366+0000","last_became_active":"2023-11-24T06:08:57.518952+0000","last_became_peered":"2023-11-24T06:08:57.518952+0000","last_unstale":"2023-11-24T06:08:58.517366+0000","last_undegraded":"2023-11-24T06:08:58.517366+0000","last_fullsized":"2023-11-24T06:08:58.517366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543999+0000","last_change":"2023-11-24T06:08:58.539562+0000","last_active":"2023-11-24T06:08:58.543999+0000","last_peered":"2023-11-24T06:08:58.543999+0000","last_clean":"2023-11-24T06:08:58.543999+0000","last_became_active":"2023-11-24T06:08:58.539268+0000","last_became_peered":"2023-11-24T06:08:58.539268+0000","last_unstale":"2023-11-24T06:08:58.543999+0000","last_undegraded":"2023-11-24T06:08:58.543999+0000","last_fullsized":"2023-11-24T06:08:58.543999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:53.532958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543401+0000","last_change":"2023-11-24T06:08:57.525492+0000","last_active":"2023-11-24T06:08:58.543401+0000","last_peered":"2023-11-24T06:08:58.543401+0000","last_clean":"2023-11-24T06:08:58.543401+0000","last_became_active":"2023-11-24T06:08:57.525231+0000","last_became_peered":"2023-11-24T06:08:57.525231+0000","last_unstale":"2023-11-24T06:08:58.543401+0000","last_undegraded":"2023-11-24T06:08:58.543401+0000","last_fullsized":"2023-11-24T06:08:58.543401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524614+0000","last_change":"2023-11-24T06:08:58.522482+0000","last_active":"2023-11-24T06:08:58.524614+0000","last_peered":"2023-11-24T06:08:58.524614+0000","last_clean":"2023-11-24T06:08:58.524614+0000","last_became_active":"2023-11-24T06:08:58.522237+0000","last_became_peered":"2023-11-24T06:08:58.522237+0000","last_unstale":"2023-11-24T06:08:58.524614+0000","last_undegraded":"2023-11-24T06:08:58.524614+0000","last_fullsized":"2023-11-24T06:08:58.524614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.516117+0000","last_change":"2023-11-24T06:08:57.525325+0000","last_active":"2023-11-24T06:08:58.516117+0000","last_peered":"2023-11-24T06:08:58.516117+0000","last_clean":"2023-11-24T06:08:58.516117+0000","last_became_active":"2023-11-24T06:08:57.525051+0000","last_became_peered":"2023-11-24T06:08:57.525051+0000","last_unstale":"2023-11-24T06:08:58.516117+0000","last_undegraded":"2023-11-24T06:08:58.516117+0000","last_fullsized":"2023-11-24T06:08:58.516117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:16.569994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534692+0000","last_change":"2023-11-24T06:08:58.534115+0000","last_active":"2023-11-24T06:08:58.534692+0000","last_peered":"2023-11-24T06:08:58.534692+0000","last_clean":"2023-11-24T06:08:58.534692+0000","last_became_active":"2023-11-24T06:08:58.525128+0000","last_became_peered":"2023-11-24T06:08:58.525128+0000","last_unstale":"2023-11-24T06:08:58.534692+0000","last_undegraded":"2023-11-24T06:08:58.534692+0000","last_fullsized":"2023-11-24T06:08:58.534692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:03.307275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.518114+0000","last_change":"2023-11-24T06:08:57.530707+0000","last_active":"2023-11-24T06:08:58.518114+0000","last_peered":"2023-11-24T06:08:58.518114+0000","last_clean":"2023-11-24T06:08:58.518114+0000","last_became_active":"2023-11-24T06:08:57.529531+0000","last_became_peered":"2023-11-24T06:08:57.529531+0000","last_unstale":"2023-11-24T06:08:58.518114+0000","last_undegraded":"2023-11-24T06:08:58.518114+0000","last_fullsized":"2023-11-24T06:08:58.518114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527289+0000","last_change":"2023-11-24T06:08:58.523161+0000","last_active":"2023-11-24T06:08:58.527289+0000","last_peered":"2023-11-24T06:08:58.527289+0000","last_clean":"2023-11-24T06:08:58.527289+0000","last_became_active":"2023-11-24T06:08:58.522753+0000","last_became_peered":"2023-11-24T06:08:58.522753+0000","last_unstale":"2023-11-24T06:08:58.527289+0000","last_undegraded":"2023-11-24T06:08:58.527289+0000","last_fullsized":"2023-11-24T06:08:58.527289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:03.307275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.517545+0000","last_change":"2023-11-24T06:08:57.521686+0000","last_active":"2023-11-24T06:08:58.517545+0000","last_peered":"2023-11-24T06:08:58.517545+0000","last_clean":"2023-11-24T06:08:58.517545+0000","last_became_active":"2023-11-24T06:08:57.520431+0000","last_became_peered":"2023-11-24T06:08:57.520431+0000","last_unstale":"2023-11-24T06:08:58.517545+0000","last_undegraded":"2023-11-24T06:08:58.517545+0000","last_fullsized":"2023-11-24T06:08:58.517545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:31.591882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.524982+0000","last_change":"2023-11-24T06:08:58.519601+0000","last_active":"2023-11-24T06:08:58.524982+0000","last_peered":"2023-11-24T06:08:58.524982+0000","last_clean":"2023-11-24T06:08:58.524982+0000","last_became_active":"2023-11-24T06:08:58.519340+0000","last_became_peered":"2023-11-24T06:08:58.519340+0000","last_unstale":"2023-11-24T06:08:58.524982+0000","last_undegraded":"2023-11-24T06:08:58.524982+0000","last_fullsized":"2023-11-24T06:08:58.524982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.541736+0000","last_change":"2023-11-24T06:08:57.529494+0000","last_active":"2023-11-24T06:08:58.541736+0000","last_peered":"2023-11-24T06:08:58.541736+0000","last_clean":"2023-11-24T06:08:58.541736+0000","last_became_active":"2023-11-24T06:08:57.529231+0000","last_became_peered":"2023-11-24T06:08:57.529231+0000","last_unstale":"2023-11-24T06:08:58.541736+0000","last_undegraded":"2023-11-24T06:08:58.541736+0000","last_fullsized":"2023-11-24T06:08:58.541736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:06.359234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.533968+0000","last_change":"2023-11-24T06:08:58.529994+0000","last_active":"2023-11-24T06:08:58.533968+0000","last_peered":"2023-11-24T06:08:58.533968+0000","last_clean":"2023-11-24T06:08:58.533968+0000","last_became_active":"2023-11-24T06:08:58.529528+0000","last_became_peered":"2023-11-24T06:08:58.529528+0000","last_unstale":"2023-11-24T06:08:58.533968+0000","last_undegraded":"2023-11-24T06:08:58.533968+0000","last_fullsized":"2023-11-24T06:08:58.533968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.510177+0000","last_change":"2023-11-24T06:08:57.525323+0000","last_active":"2023-11-24T06:08:58.510177+0000","last_peered":"2023-11-24T06:08:58.510177+0000","last_clean":"2023-11-24T06:08:58.510177+0000","last_became_active":"2023-11-24T06:08:57.525087+0000","last_became_peered":"2023-11-24T06:08:57.525087+0000","last_unstale":"2023-11-24T06:08:58.510177+0000","last_undegraded":"2023-11-24T06:08:58.510177+0000","last_fullsized":"2023-11-24T06:08:58.510177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:43.891317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.542229+0000","last_change":"2023-11-24T06:08:58.539679+0000","last_active":"2023-11-24T06:08:58.542229+0000","last_peered":"2023-11-24T06:08:58.542229+0000","last_clean":"2023-11-24T06:08:58.542229+0000","last_became_active":"2023-11-24T06:08:58.539279+0000","last_became_peered":"2023-11-24T06:08:58.539279+0000","last_unstale":"2023-11-24T06:08:58.542229+0000","last_undegraded":"2023-11-24T06:08:58.542229+0000","last_fullsized":"2023-11-24T06:08:58.542229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:26.004865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534964+0000","last_change":"2023-11-24T06:08:57.523738+0000","last_active":"2023-11-24T06:08:58.534964+0000","last_peered":"2023-11-24T06:08:58.534964+0000","last_clean":"2023-11-24T06:08:58.534964+0000","last_became_active":"2023-11-24T06:08:57.523396+0000","last_became_peered":"2023-11-24T06:08:57.523396+0000","last_unstale":"2023-11-24T06:08:58.534964+0000","last_undegraded":"2023-11-24T06:08:58.534964+0000","last_fullsized":"2023-11-24T06:08:58.534964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527309+0000","last_change":"2023-11-24T06:08:58.520593+0000","last_active":"2023-11-24T06:08:58.527309+0000","last_peered":"2023-11-24T06:08:58.527309+0000","last_clean":"2023-11-24T06:08:58.527309+0000","last_became_active":"2023-11-24T06:08:58.520297+0000","last_became_peered":"2023-11-24T06:08:58.520297+0000","last_unstale":"2023-11-24T06:08:58.527309+0000","last_undegraded":"2023-11-24T06:08:58.527309+0000","last_fullsized":"2023-11-24T06:08:58.527309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.536745+0000","last_change":"2023-11-24T06:08:57.521315+0000","last_active":"2023-11-24T06:08:58.536745+0000","last_peered":"2023-11-24T06:08:58.536745+0000","last_clean":"2023-11-24T06:08:58.536745+0000","last_became_active":"2023-11-24T06:08:57.520718+0000","last_became_peered":"2023-11-24T06:08:57.520718+0000","last_unstale":"2023-11-24T06:08:58.536745+0000","last_undegraded":"2023-11-24T06:08:58.536745+0000","last_fullsized":"2023-11-24T06:08:58.536745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:16.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.544298+0000","last_change":"2023-11-24T06:08:58.542621+0000","last_active":"2023-11-24T06:08:58.544298+0000","last_peered":"2023-11-24T06:08:58.544298+0000","last_clean":"2023-11-24T06:08:58.544298+0000","last_became_active":"2023-11-24T06:08:58.541632+0000","last_became_peered":"2023-11-24T06:08:58.541632+0000","last_unstale":"2023-11-24T06:08:58.544298+0000","last_undegraded":"2023-11-24T06:08:58.544298+0000","last_fullsized":"2023-11-24T06:08:58.544298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:32.599128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534237+0000","last_change":"2023-11-24T06:08:58.527955+0000","last_active":"2023-11-24T06:08:58.534237+0000","last_peered":"2023-11-24T06:08:58.534237+0000","last_clean":"2023-11-24T06:08:58.534237+0000","last_became_active":"2023-11-24T06:08:58.527336+0000","last_became_peered":"2023-11-24T06:08:58.527336+0000","last_unstale":"2023-11-24T06:08:58.534237+0000","last_undegraded":"2023-11-24T06:08:58.534237+0000","last_fullsized":"2023-11-24T06:08:58.534237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.531404+0000","last_change":"2023-11-24T06:08:58.530318+0000","last_active":"2023-11-24T06:08:58.531404+0000","last_peered":"2023-11-24T06:08:58.531404+0000","last_clean":"2023-11-24T06:08:58.531404+0000","last_became_active":"2023-11-24T06:08:58.530015+0000","last_became_peered":"2023-11-24T06:08:58.530015+0000","last_unstale":"2023-11-24T06:08:58.531404+0000","last_undegraded":"2023-11-24T06:08:58.531404+0000","last_fullsized":"2023-11-24T06:08:58.531404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532611+0000","last_change":"2023-11-24T06:08:58.522901+0000","last_active":"2023-11-24T06:08:58.532611+0000","last_peered":"2023-11-24T06:08:58.532611+0000","last_clean":"2023-11-24T06:08:58.532611+0000","last_became_active":"2023-11-24T06:08:58.522620+0000","last_became_peered":"2023-11-24T06:08:58.522620+0000","last_unstale":"2023-11-24T06:08:58.532611+0000","last_undegraded":"2023-11-24T06:08:58.532611+0000","last_fullsized":"2023-11-24T06:08:58.532611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:18.279785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.523980+0000","last_change":"2023-11-24T06:08:58.519217+0000","last_active":"2023-11-24T06:08:58.523980+0000","last_peered":"2023-11-24T06:08:58.523980+0000","last_clean":"2023-11-24T06:08:58.523980+0000","last_became_active":"2023-11-24T06:08:58.518577+0000","last_became_peered":"2023-11-24T06:08:58.518577+0000","last_unstale":"2023-11-24T06:08:58.523980+0000","last_undegraded":"2023-11-24T06:08:58.523980+0000","last_fullsized":"2023-11-24T06:08:58.523980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.527945+0000","last_change":"2023-11-24T06:08:58.523627+0000","last_active":"2023-11-24T06:08:58.527945+0000","last_peered":"2023-11-24T06:08:58.527945+0000","last_clean":"2023-11-24T06:08:58.527945+0000","last_became_active":"2023-11-24T06:08:58.522543+0000","last_became_peered":"2023-11-24T06:08:58.522543+0000","last_unstale":"2023-11-24T06:08:58.527945+0000","last_undegraded":"2023-11-24T06:08:58.527945+0000","last_fullsized":"2023-11-24T06:08:58.527945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.523745+0000","last_change":"2023-11-24T06:08:58.521811+0000","last_active":"2023-11-24T06:08:58.523745+0000","last_peered":"2023-11-24T06:08:58.523745+0000","last_clean":"2023-11-24T06:08:58.523745+0000","last_became_active":"2023-11-24T06:08:58.521129+0000","last_became_peered":"2023-11-24T06:08:58.521129+0000","last_unstale":"2023-11-24T06:08:58.523745+0000","last_undegraded":"2023-11-24T06:08:58.523745+0000","last_fullsized":"2023-11-24T06:08:58.523745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.541197+0000","last_change":"2023-11-24T06:08:58.539747+0000","last_active":"2023-11-24T06:08:58.541197+0000","last_peered":"2023-11-24T06:08:58.541197+0000","last_clean":"2023-11-24T06:08:58.541197+0000","last_became_active":"2023-11-24T06:08:58.539449+0000","last_became_peered":"2023-11-24T06:08:58.539449+0000","last_unstale":"2023-11-24T06:08:58.541197+0000","last_undegraded":"2023-11-24T06:08:58.541197+0000","last_fullsized":"2023-11-24T06:08:58.541197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.528315+0000","last_change":"2023-11-24T06:08:58.523632+0000","last_active":"2023-11-24T06:08:58.528315+0000","last_peered":"2023-11-24T06:08:58.528315+0000","last_clean":"2023-11-24T06:08:58.528315+0000","last_became_active":"2023-11-24T06:08:58.523365+0000","last_became_peered":"2023-11-24T06:08:58.523365+0000","last_unstale":"2023-11-24T06:08:58.528315+0000","last_undegraded":"2023-11-24T06:08:58.528315+0000","last_fullsized":"2023-11-24T06:08:58.528315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:44.433815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.532681+0000","last_change":"2023-11-24T06:08:58.527956+0000","last_active":"2023-11-24T06:08:58.532681+0000","last_peered":"2023-11-24T06:08:58.532681+0000","last_clean":"2023-11-24T06:08:58.532681+0000","last_became_active":"2023-11-24T06:08:58.527700+0000","last_became_peered":"2023-11-24T06:08:58.527700+0000","last_unstale":"2023-11-24T06:08:58.532681+0000","last_undegraded":"2023-11-24T06:08:58.532681+0000","last_fullsized":"2023-11-24T06:08:58.532681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:18:14.318818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.530385+0000","last_change":"2023-11-24T06:08:58.529061+0000","last_active":"2023-11-24T06:08:58.530385+0000","last_peered":"2023-11-24T06:08:58.530385+0000","last_clean":"2023-11-24T06:08:58.530385+0000","last_became_active":"2023-11-24T06:08:58.528753+0000","last_became_peered":"2023-11-24T06:08:58.528753+0000","last_unstale":"2023-11-24T06:08:58.530385+0000","last_undegraded":"2023-11-24T06:08:58.530385+0000","last_fullsized":"2023-11-24T06:08:58.530385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:29.373282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543547+0000","last_change":"2023-11-24T06:08:58.541017+0000","last_active":"2023-11-24T06:08:58.543547+0000","last_peered":"2023-11-24T06:08:58.543547+0000","last_clean":"2023-11-24T06:08:58.543547+0000","last_became_active":"2023-11-24T06:08:58.540746+0000","last_became_peered":"2023-11-24T06:08:58.540746+0000","last_unstale":"2023-11-24T06:08:58.543547+0000","last_undegraded":"2023-11-24T06:08:58.543547+0000","last_fullsized":"2023-11-24T06:08:58.543547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:26.004865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.543810+0000","last_change":"2023-11-24T06:08:58.540211+0000","last_active":"2023-11-24T06:08:58.543810+0000","last_peered":"2023-11-24T06:08:58.543810+0000","last_clean":"2023-11-24T06:08:58.543810+0000","last_became_active":"2023-11-24T06:08:58.539978+0000","last_became_peered":"2023-11-24T06:08:58.539978+0000","last_unstale":"2023-11-24T06:08:58.543810+0000","last_undegraded":"2023-11-24T06:08:58.543810+0000","last_fullsized":"2023-11-24T06:08:58.543810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.534871+0000","last_change":"2023-11-24T06:08:58.529024+0000","last_active":"2023-11-24T06:08:58.534871+0000","last_peered":"2023-11-24T06:08:58.534871+0000","last_clean":"2023-11-24T06:08:58.534871+0000","last_became_active":"2023-11-24T06:08:58.528704+0000","last_became_peered":"2023-11-24T06:08:58.528704+0000","last_unstale":"2023-11-24T06:08:58.534871+0000","last_undegraded":"2023-11-24T06:08:58.534871+0000","last_fullsized":"2023-11-24T06:08:58.534871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:44.898563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":655616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":66222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":57835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":64700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.536}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1040000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:09:03.006 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:09:03.006 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T06:09:03.006 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T06:09:03.006 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T06:09:03.007 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T06:09:03.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.159+0000 7f8f87d21700 1 Processor -- start 2023-11-24T06:09:03.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.159+0000 7f8f87d21700 1 -- start start 2023-11-24T06:09:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f87d21700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8f8006c710 0x7f8f800b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f87d21700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f800af4d0 0x7f8f800afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f87d21700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f800ae430 con 0x7f8f8015e080 2023-11-24T06:09:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f87d21700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f800af0b0 con 0x7f8f8006c710 2023-11-24T06:09:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f87d21700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800b05a0 con 0x7f8f800af4d0 2023-11-24T06:09:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f852bc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8f8006c710 0x7f8f800b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f862be700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f800af4d0 0x7f8f800afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f852bc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8f8006c710 0x7f8f800b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39238/0 (socket says 172.21.15.97:39238) 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f862be700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f800af4d0 0x7f8f800afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60346/0 (socket says 172.21.15.97:60346) 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.160+0000 7f8f852bc700 1 -- 172.21.15.97:0/2769969319 learned_addr learned my addr 172.21.15.97:0/2769969319 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.161+0000 7f8f852bc700 1 -- 172.21.15.97:0/2769969319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f800af4d0 msgr2=0x7f8f800afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.161+0000 7f8f852bc700 1 --2- 172.21.15.97:0/2769969319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f800af4d0 0x7f8f800afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.161+0000 7f8f852bc700 1 -- 172.21.15.97:0/2769969319 >> v1:172.21.15.183:6789/0 conn(0x7f8f8015e080 legacy=0x7f8f8006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.161+0000 7f8f852bc700 1 -- 172.21.15.97:0/2769969319 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800ae710 con 0x7f8f8006c710 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.161+0000 7f8f852bc700 1 --2- 172.21.15.97:0/2769969319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f8006c710 0x7f8f800b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0098f0 tx=0x7f8f7c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296133c1b395a82d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.161+0000 7f8f84abb700 1 -- 172.21.15.97:0/2769969319 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c011830 con 0x7f8f8006c710 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.161+0000 7f8f84abb700 1 -- 172.21.15.97:0/2769969319 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c011990 con 0x7f8f8006c710 2023-11-24T06:09:03.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.161+0000 7f8f84abb700 1 -- 172.21.15.97:0/2769969319 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c011d20 con 0x7f8f8006c710 2023-11-24T06:09:03.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.162+0000 7f8f87d21700 1 -- 172.21.15.97:0/2769969319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f8006c710 msgr2=0x7f8f800b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.162+0000 7f8f87d21700 1 --2- 172.21.15.97:0/2769969319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f8006c710 0x7f8f800b5050 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0098f0 tx=0x7f8f7c009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:03.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.162+0000 7f8f87d21700 1 -- 172.21.15.97:0/2769969319 shutdown_connections 2023-11-24T06:09:03.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.162+0000 7f8f87d21700 1 --2- 172.21.15.97:0/2769969319 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f800af4d0 0x7f8f800afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.162+0000 7f8f87d21700 1 --2- 172.21.15.97:0/2769969319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f8006c710 0x7f8f800b5050 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.162+0000 7f8f87d21700 1 -- 172.21.15.97:0/2769969319 >> 172.21.15.97:0/2769969319 conn(0x7f8f80071b90 msgr2=0x7f8f80071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.162+0000 7f8f87d21700 1 -- 172.21.15.97:0/2769969319 shutdown_connections 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.162+0000 7f8f87d21700 1 -- 172.21.15.97:0/2769969319 wait complete. 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.163+0000 7f8f87d21700 1 Processor -- start 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.163+0000 7f8f87d21700 1 -- start start 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f87d21700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f8006c710 0x7f8f80156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f87d21700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800af4d0 0x7f8f801f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f87d21700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015e080 0x7f8f80160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f87d21700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8015e840 con 0x7f8f8015e080 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f87d21700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f801f3ff0 con 0x7f8f800af4d0 2023-11-24T06:09:03.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f87d21700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f801f42d0 con 0x7f8f8006c710 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015e080 0x7f8f80160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015e080 0x7f8f80160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45086/0 (socket says 172.21.15.97:45086) 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 -- 172.21.15.97:0/3144161999 learned_addr learned my addr 172.21.15.97:0/3144161999 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f85abd700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f8006c710 0x7f8f80156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f852bc700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800af4d0 0x7f8f801f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 -- 172.21.15.97:0/3144161999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f8006c710 msgr2=0x7f8f80156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f8006c710 0x7f8f80156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 -- 172.21.15.97:0/3144161999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800af4d0 msgr2=0x7f8f801f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800af4d0 0x7f8f801f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 -- 172.21.15.97:0/3144161999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c009550 con 0x7f8f8015e080 2023-11-24T06:09:03.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f862be700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015e080 0x7f8f80160db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800efc0 tx=0x7f8f7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f767fc700 1 -- 172.21.15.97:0/3144161999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78009ea0 con 0x7f8f8015e080 2023-11-24T06:09:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f767fc700 1 -- 172.21.15.97:0/3144161999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7800f040 con 0x7f8f8015e080 2023-11-24T06:09:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f801f45b0 con 0x7f8f8015e080 2023-11-24T06:09:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f767fc700 1 -- 172.21.15.97:0/3144161999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800ced0 con 0x7f8f8015e080 2023-11-24T06:09:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.164+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f801f3ca0 con 0x7f8f8015e080 2023-11-24T06:09:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.165+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f800753f0 con 0x7f8f8015e080 2023-11-24T06:09:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.166+0000 7f8f767fc700 1 -- 172.21.15.97:0/3144161999 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f78014480 con 0x7f8f8015e080 2023-11-24T06:09:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.167+0000 7f8f767fc700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8f6406af00 0x7f8f6406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.167+0000 7f8f767fc700 1 -- 172.21.15.97:0/3144161999 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f7808a4c0 con 0x7f8f8015e080 2023-11-24T06:09:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.167+0000 7f8f85abd700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8f6406af00 0x7f8f6406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.167+0000 7f8f85abd700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8f6406af00 0x7f8f6406d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f70007fd0 tx=0x7f8f70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.170+0000 7f8f767fc700 1 -- 172.21.15.97:0/3144161999 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f780555e0 con 0x7f8f8015e080 2023-11-24T06:09:03.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.415+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8f80072cf0 con 0x7f8f8015e080 2023-11-24T06:09:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.418+0000 7f8f767fc700 1 -- 172.21.15.97:0/3144161999 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171027 (secure 0 0 0) 0x7f8f78018090 con 0x7f8f8015e080 2023-11-24T06:09:03.423 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8f6406af00 msgr2=0x7f8f6406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8f6406af00 0x7f8f6406d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8f70007fd0 tx=0x7f8f70009040 comp rx=0 tx=0).stop 2023-11-24T06:09:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015e080 msgr2=0x7f8f80160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015e080 0x7f8f80160db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800efc0 tx=0x7f8f7800c3a0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 shutdown_connections 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f8015e080 0x7f8f80160db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8f6406af00 0x7f8f6406d3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8f8006c710 0x7f8f80156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 --2- 172.21.15.97:0/3144161999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f800af4d0 0x7f8f801f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 >> 172.21.15.97:0/3144161999 conn(0x7f8f80071b90 msgr2=0x7f8f8006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 shutdown_connections 2023-11-24T06:09:03.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.421+0000 7f8f87d21700 1 -- 172.21.15.97:0/3144161999 wait complete. 2023-11-24T06:09:03.436 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":830621211},{"type":"v1","addr":"172.21.15.97:6833","nonce":830621211}]},"active_addr":"172.21.15.97:6833/830621211","active_change":"2023-11-24T06:08:38.314934+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":729599028}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":415877240}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":1855023783}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":4021299502}]}]} 2023-11-24T06:09:03.442 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T06:09:03.442 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T06:09:03.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:03.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.559+0000 7f64e4735700 1 Processor -- start 2023-11-24T06:09:03.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.559+0000 7f64e4735700 1 -- start start 2023-11-24T06:09:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.560+0000 7f64e4735700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc15bdd0 0x7f64dc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.560+0000 7f64e4735700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f64dc0d1c70 0x7f64dc0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.560+0000 7f64e4735700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64dc0b22d0 con 0x7f64dc0b3370 2023-11-24T06:09:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.560+0000 7f64e4735700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64dc0b2f50 con 0x7f64dc0d1c70 2023-11-24T06:09:03.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.560+0000 7f64e4735700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0d2640 con 0x7f64dc15bdd0 2023-11-24T06:09:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e2cd2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f64dc0d1c70 0x7f64dc0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e24d1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc15bdd0 0x7f64dc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e1cd0700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f64dc0b3370 0x7f64dc0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46864/0 (socket says 172.21.15.97:46864) 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e1cd0700 1 -- 172.21.15.97:0/2040714114 learned_addr learned my addr 172.21.15.97:0/2040714114 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e14cf700 1 -- 172.21.15.97:0/2040714114 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844324864 0 0) 0x7f64dc0b22d0 con 0x7f64dc0b3370 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e14cf700 1 -- 172.21.15.97:0/2040714114 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64c80035b0 con 0x7f64dc0b3370 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e2cd2700 1 -- 172.21.15.97:0/2040714114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc15bdd0 msgr2=0x7f64dc1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e2cd2700 1 --2- 172.21.15.97:0/2040714114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc15bdd0 0x7f64dc1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e2cd2700 1 -- 172.21.15.97:0/2040714114 >> v1:172.21.15.183:6789/0 conn(0x7f64dc0b3370 legacy=0x7f64dc0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e2cd2700 1 -- 172.21.15.97:0/2040714114 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0b25b0 con 0x7f64dc0d1c70 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e24d1700 1 --2- 172.21.15.97:0/2040714114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc15bdd0 0x7f64dc1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e2cd2700 1 --2- 172.21.15.97:0/2040714114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 0x7f64dc0d2100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009d90 tx=0x7f64d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1063568f120284cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.561+0000 7f64e14cf700 1 -- 172.21.15.97:0/2040714114 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d400e070 con 0x7f64dc0d1c70 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e14cf700 1 -- 172.21.15.97:0/2040714114 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d400bb20 con 0x7f64dc0d1c70 2023-11-24T06:09:03.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e14cf700 1 -- 172.21.15.97:0/2040714114 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4012430 con 0x7f64dc0d1c70 2023-11-24T06:09:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e4735700 1 -- 172.21.15.97:0/2040714114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 msgr2=0x7f64dc0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e4735700 1 --2- 172.21.15.97:0/2040714114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 0x7f64dc0d2100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009d90 tx=0x7f64d400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e4735700 1 -- 172.21.15.97:0/2040714114 shutdown_connections 2023-11-24T06:09:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e4735700 1 --2- 172.21.15.97:0/2040714114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 0x7f64dc0d2100 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e4735700 1 --2- 172.21.15.97:0/2040714114 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc15bdd0 0x7f64dc1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e4735700 1 -- 172.21.15.97:0/2040714114 >> 172.21.15.97:0/2040714114 conn(0x7f64dc0731a0 msgr2=0x7f64dc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e4735700 1 -- 172.21.15.97:0/2040714114 shutdown_connections 2023-11-24T06:09:03.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.562+0000 7f64e4735700 1 -- 172.21.15.97:0/2040714114 wait complete. 2023-11-24T06:09:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.563+0000 7f64e4735700 1 Processor -- start 2023-11-24T06:09:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.563+0000 7f64e4735700 1 -- start start 2023-11-24T06:09:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e4735700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc0b3370 0x7f64dc158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e4735700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 0x7f64dc1f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e4735700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64dc15bdd0 0x7f64dc15d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e4735700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc1f4180 con 0x7f64dc15bdd0 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e4735700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc06b640 con 0x7f64dc0d1c70 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e4735700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0b73a0 con 0x7f64dc0b3370 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e2cd2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64dc15bdd0 0x7f64dc15d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e2cd2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64dc15bdd0 0x7f64dc15d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45100/0 (socket says 172.21.15.97:45100) 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e2cd2700 1 -- 172.21.15.97:0/288690320 learned_addr learned my addr 172.21.15.97:0/288690320 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e1cd0700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 0x7f64dc1f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.564+0000 7f64e24d1700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc0b3370 0x7f64dc158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64e2cd2700 1 -- 172.21.15.97:0/288690320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc0b3370 msgr2=0x7f64dc158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64e2cd2700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc0b3370 0x7f64dc158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64e2cd2700 1 -- 172.21.15.97:0/288690320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 msgr2=0x7f64dc1f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64e2cd2700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 0x7f64dc1f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64e2cd2700 1 -- 172.21.15.97:0/288690320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0094a0 con 0x7f64dc15bdd0 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64e2cd2700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64dc15bdd0 0x7f64dc15d6f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64d4014040 tx=0x7f64d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64d2ffd700 1 -- 172.21.15.97:0/288690320 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d40129f0 con 0x7f64dc15bdd0 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64d2ffd700 1 -- 172.21.15.97:0/288690320 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d4020070 con 0x7f64dc15bdd0 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64d2ffd700 1 -- 172.21.15.97:0/288690320 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4012db0 con 0x7f64dc15bdd0 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d4009a70 con 0x7f64dc15bdd0 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.565+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc06aff0 con 0x7f64dc15bdd0 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.566+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc06b7d0 con 0x7f64dc15bdd0 2023-11-24T06:09:03.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.566+0000 7f64d2ffd700 1 -- 172.21.15.97:0/288690320 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64d4004040 con 0x7f64dc15bdd0 2023-11-24T06:09:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.567+0000 7f64d2ffd700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64c006aed0 0x7f64c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.567+0000 7f64d2ffd700 1 -- 172.21.15.97:0/288690320 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64d4022070 con 0x7f64dc15bdd0 2023-11-24T06:09:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.567+0000 7f64e24d1700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64c006aed0 0x7f64c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.567+0000 7f64e24d1700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64c006aed0 0x7f64c006d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009cd0 tx=0x7f64cc009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.571+0000 7f64d2ffd700 1 -- 172.21.15.97:0/288690320 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64d4056230 con 0x7f64dc15bdd0 2023-11-24T06:09:03.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.764+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64dc067f60 con 0x7f64dc15bdd0 2023-11-24T06:09:03.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.765+0000 7f64d2ffd700 1 -- 172.21.15.97:0/288690320 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f64d4059850 con 0x7f64dc15bdd0 2023-11-24T06:09:03.768 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:03.768 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:03.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64c006aed0 msgr2=0x7f64c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64c006aed0 0x7f64c006d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009cd0 tx=0x7f64cc009210 comp rx=0 tx=0).stop 2023-11-24T06:09:03.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64dc15bdd0 msgr2=0x7f64dc15d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64dc15bdd0 0x7f64dc15d6f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64d4014040 tx=0x7f64d4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 shutdown_connections 2023-11-24T06:09:03.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64dc15bdd0 0x7f64dc15d6f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64c006aed0 0x7f64c006d390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64dc0b3370 0x7f64dc158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 --2- 172.21.15.97:0/288690320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64dc0d1c70 0x7f64dc1f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 >> 172.21.15.97:0/288690320 conn(0x7f64dc0731a0 msgr2=0x7f64dc06ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:03.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 shutdown_connections 2023-11-24T06:09:03.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.767+0000 7f64e4735700 1 -- 172.21.15.97:0/288690320 wait complete. 2023-11-24T06:09:03.782 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T06:09:03.782 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.899+0000 7f9a2062a700 1 Processor -- start 2023-11-24T06:09:03.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.899+0000 7f9a2062a700 1 -- start start 2023-11-24T06:09:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a2062a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 0x7f9a181655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a2062a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1815bb60 con 0x7f9a180b0ba0 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a2062a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a180b2bd0 con 0x7f9a180b0f80 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a2062a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180b1fc0 con 0x7f9a180cd770 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1e3c6700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9a180b0ba0 0x7f9a180af5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37556/0 (socket says 172.21.15.97:37556) 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1e3c6700 1 -- 172.21.15.97:0/4106535472 learned_addr learned my addr 172.21.15.97:0/4106535472 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1ebc7700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 0x7f9a181655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988870887 0 0) 0x7f9a08001110 con 0x7f9a180b0ba0 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a040035b0 con 0x7f9a180b0ba0 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868548903 0 0) 0x7f9a180b2bd0 con 0x7f9a180b0f80 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a08001110 con 0x7f9a180b0f80 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2346866802 0 0) 0x7f9a040035b0 con 0x7f9a180b0ba0 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 msgr2=0x7f9a181655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 0x7f9a181655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 >> v1:172.21.15.183:6789/0 conn(0x7f9a180b0f80 legacy=0x7f9a1815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180b22a0 con 0x7f9a180b0ba0 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f9a08001780 con 0x7f9a180b0ba0 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1ebc7700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 0x7f9a181655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 >> v1:172.21.15.97:6789/0 conn(0x7f9a180b0ba0 legacy=0x7f9a180af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a04003bc0 0x7f9a04003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a04004850 0x7f9a04004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 0x7f9a04005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180b1fc0 con 0x7f9a04003bc0 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0400bac0 con 0x7f9a04004850 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0400bc50 con 0x7f9a180cd770 2023-11-24T06:09:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.900+0000 7f9a1e3c6700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a04003bc0 0x7f9a04003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1dbc5700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a04004850 0x7f9a04004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1ebc7700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 0x7f9a04005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1e3c6700 1 -- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 msgr2=0x7f9a04005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1e3c6700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a180cd770 0x7f9a04005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1e3c6700 1 -- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a04004850 msgr2=0x7f9a04004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1e3c6700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a04004850 0x7f9a04004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1e3c6700 1 -- 172.21.15.97:0/4106535472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a10009a70 con 0x7f9a04003bc0 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1e3c6700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a04003bc0 0x7f9a04003fc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a08007f60 tx=0x7f9a04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3269e13140d823d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08007bf0 con 0x7f9a04003bc0 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0800cbf0 con 0x7f9a04003bc0 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.901+0000 7f9a1d3c4700 1 -- 172.21.15.97:0/4106535472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08005960 con 0x7f9a04003bc0 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.902+0000 7f9a2062a700 1 -- 172.21.15.97:0/4106535472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a04003bc0 msgr2=0x7f9a04003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.902+0000 7f9a2062a700 1 --2- 172.21.15.97:0/4106535472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a04003bc0 0x7f9a04003fc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9a08007f60 tx=0x7f9a04005880 comp rx=0 tx=0).stop 2023-11-24T06:09:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.902+0000 7f9a1ebc7700 1 -- 172.21.15.97:0/4106535472 reap_dead start 2023-11-24T06:09:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.902+0000 7f9a2062a700 1 -- 172.21.15.97:0/4106535472 shutdown_connections 2023-11-24T06:09:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.902+0000 7f9a2062a700 1 -- 172.21.15.97:0/4106535472 >> 172.21.15.97:0/4106535472 conn(0x7f9a180b57b0 msgr2=0x7f9a180b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.902+0000 7f9a2062a700 1 -- 172.21.15.97:0/4106535472 shutdown_connections 2023-11-24T06:09:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.902+0000 7f9a2062a700 1 -- 172.21.15.97:0/4106535472 wait complete. 2023-11-24T06:09:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.903+0000 7f9a2062a700 1 Processor -- start 2023-11-24T06:09:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.903+0000 7f9a2062a700 1 -- start start 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a2062a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a04003bc0 0x7f9a18148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a2062a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1814fc00 0x7f9a18151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a2062a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a18150080 0x7f9a18151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a2062a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1814e490 con 0x7f9a1814fc00 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a2062a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1815b300 con 0x7f9a18150080 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a2062a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1815b5a0 con 0x7f9a04003bc0 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a1dbc5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1814fc00 0x7f9a18151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a1dbc5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1814fc00 0x7f9a18151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45118/0 (socket says 172.21.15.97:45118) 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a1dbc5700 1 -- 172.21.15.97:0/1915724600 learned_addr learned my addr 172.21.15.97:0/1915724600 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a1dbc5700 1 -- 172.21.15.97:0/1915724600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a04003bc0 msgr2=0x7f9a18148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a1e3c6700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a04003bc0 0x7f9a18148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a1ebc7700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a18150080 0x7f9a18151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.904+0000 7f9a1dbc5700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a04003bc0 0x7f9a18148090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a1dbc5700 1 -- 172.21.15.97:0/1915724600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a18150080 msgr2=0x7f9a18151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a1dbc5700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a18150080 0x7f9a18151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a1dbc5700 1 -- 172.21.15.97:0/1915724600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1815b880 con 0x7f9a1814fc00 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a1dbc5700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1814fc00 0x7f9a18151770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400f800 tx=0x7f9a1400fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a0effd700 1 -- 172.21.15.97:0/1915724600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14005320 con 0x7f9a1814fc00 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a0effd700 1 -- 172.21.15.97:0/1915724600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a14005480 con 0x7f9a1814fc00 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a180aee90 con 0x7f9a1814fc00 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a0effd700 1 -- 172.21.15.97:0/1915724600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14006a80 con 0x7f9a1814fc00 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.905+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a18148b70 con 0x7f9a1814fc00 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.906+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a18148d90 con 0x7f9a1814fc00 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.906+0000 7f9a0effd700 1 -- 172.21.15.97:0/1915724600 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9a14006be0 con 0x7f9a1814fc00 2023-11-24T06:09:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.907+0000 7f9a0effd700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99fc06aed0 0x7f99fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:03.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.907+0000 7f9a0effd700 1 -- 172.21.15.97:0/1915724600 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a14088ee0 con 0x7f9a1814fc00 2023-11-24T06:09:03.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.907+0000 7f9a1e3c6700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99fc06aed0 0x7f99fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:03.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.907+0000 7f9a1e3c6700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99fc06aed0 0x7f99fc06d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9a18148b20 tx=0x7f9a0800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:03.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:03.911+0000 7f9a0effd700 1 -- 172.21.15.97:0/1915724600 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a14054020 con 0x7f9a1814fc00 2023-11-24T06:09:04.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.099+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a180b1d70 con 0x7f9a1814fc00 2023-11-24T06:09:04.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.100+0000 7f9a0effd700 1 -- 172.21.15.97:0/1915724600 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f9a14057640 con 0x7f9a1814fc00 2023-11-24T06:09:04.103 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:04.103 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99fc06aed0 msgr2=0x7f99fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99fc06aed0 0x7f99fc06d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9a18148b20 tx=0x7f9a0800a000 comp rx=0 tx=0).stop 2023-11-24T06:09:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1814fc00 msgr2=0x7f9a18151770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1814fc00 0x7f9a18151770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400f800 tx=0x7f9a1400fbc0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 shutdown_connections 2023-11-24T06:09:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a18150080 0x7f9a18151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99fc06aed0 0x7f99fc06d390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a04003bc0 0x7f9a18148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 --2- 172.21.15.97:0/1915724600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1814fc00 0x7f9a18151770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 >> 172.21.15.97:0/1915724600 conn(0x7f9a180b57b0 msgr2=0x7f9a180726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 shutdown_connections 2023-11-24T06:09:04.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.102+0000 7f9a2062a700 1 -- 172.21.15.97:0/1915724600 wait complete. 2023-11-24T06:09:04.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T06:09:04.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T06:09:04.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T06:09:04.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T06:09:04.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T06:09:04.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T06:09:04.120 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T06:09:04.120 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T06:09:04.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.329+0000 7f452e050700 1 Processor -- start 2023-11-24T06:09:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.330+0000 7f452e050700 1 -- start start 2023-11-24T06:09:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.330+0000 7f125305c700 1 Processor -- start 2023-11-24T06:09:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.330+0000 7fd480e6c700 1 Processor -- start 2023-11-24T06:09:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.330+0000 7fd480e6c700 1 -- start start 2023-11-24T06:09:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f452e050700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f452806d4d0 0x7f452806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f452e050700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280714e0 con 0x7f45280b4260 2023-11-24T06:09:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f452e050700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452814c1f0 con 0x7f452814be10 2023-11-24T06:09:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f452e050700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4528070bf0 con 0x7f452806d4d0 2023-11-24T06:09:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.330+0000 7f125305c700 1 -- start start 2023-11-24T06:09:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f125305c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f124c15dfc0 0x7f124c0d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f125305c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0d44e0 0x7f124c0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f125305c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0d2d10 con 0x7f124c0b2fc0 2023-11-24T06:09:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f125305c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c0cefd0 con 0x7f124c15dfc0 2023-11-24T06:09:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f125305c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0cf2b0 con 0x7f124c0d44e0 2023-11-24T06:09:04.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.331+0000 7f45277fe700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f452814be10 0x7f45280b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46900/0 (socket says 172.21.15.97:46900) 2023-11-24T06:09:04.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45277fe700 1 -- 172.21.15.97:0/3196600397 learned_addr learned my addr 172.21.15.97:0/3196600397 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7fd480e6c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c0b3b50 0x7fd47c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f124bfff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f124c0b2fc0 0x7f124c161b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46902/0 (socket says 172.21.15.97:46902) 2023-11-24T06:09:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f124bfff700 1 -- 172.21.15.97:0/172978648 learned_addr learned my addr 172.21.15.97:0/172978648 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7fd480e6c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c06ced0 con 0x7fd47c06d4d0 2023-11-24T06:09:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7fd480e6c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c06da70 con 0x7fd47c164a90 2023-11-24T06:09:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7fd480e6c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0b4470 con 0x7fd47c0b3b50 2023-11-24T06:09:04.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f12515f9700 1 --2- 172.21.15.97:0/172978648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0d44e0 0x7f124c0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212745326 0 0) 0x7f452814c1f0 con 0x7f452814be10 2023-11-24T06:09:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45100035b0 con 0x7f452814be10 2023-11-24T06:09:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7fd47ad9d700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fd47c164a90 0x7fd47c16bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46910/0 (socket says 172.21.15.97:46910) 2023-11-24T06:09:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7fd47ad9d700 1 -- 172.21.15.97:0/3432489881 learned_addr learned my addr 172.21.15.97:0/3432489881 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f02ee929700 1 Processor -- start 2023-11-24T06:09:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1103226312 0 0) 0x7f45100035b0 con 0x7f452814be10 2023-11-24T06:09:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 >> v2:172.21.15.97:3300/0 conn(0x7f452806d4d0 msgr2=0x7f452806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 --2- 172.21.15.97:0/3196600397 >> v2:172.21.15.97:3300/0 conn(0x7f452806d4d0 0x7f452806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 >> v1:172.21.15.97:6789/0 conn(0x7f45280b4260 legacy=0x7f4528071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528070e90 con 0x7f452814be10 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f45200019f0 con 0x7f452814be10 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 >> v1:172.21.15.183:6789/0 conn(0x7f452814be10 legacy=0x7f45280b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4510003a00 0x7f4510003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4510004850 0x7f4510004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510005410 0x7f451000ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528070bf0 con 0x7f4510005410 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451000c350 con 0x7f4510004850 2023-11-24T06:09:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451000c510 con 0x7f4510003a00 2023-11-24T06:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f12515f9700 1 -- 172.21.15.97:0/172978648 >> v2:172.21.15.97:3300/0 conn(0x7f124c15dfc0 msgr2=0x7f124c0d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f12515f9700 1 --2- 172.21.15.97:0/172978648 >> v2:172.21.15.97:3300/0 conn(0x7f124c15dfc0 0x7f124c0d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f12515f9700 1 -- 172.21.15.97:0/172978648 >> v1:172.21.15.183:6789/0 conn(0x7f124c0b2fc0 legacy=0x7f124c161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f12515f9700 1 -- 172.21.15.97:0/172978648 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0d2fb0 con 0x7f124c0d44e0 2023-11-24T06:09:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f45277fe700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4510003a00 0x7f4510003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f4527fff700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510005410 0x7f451000ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f4526ffd700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4510004850 0x7f4510004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f12515f9700 1 --2- 172.21.15.97:0/172978648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0d44e0 0x7f124c0d13a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1244009d90 tx=0x7f124400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=704e55110c2e055b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f45277fe700 1 -- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4510004850 msgr2=0x7f4510004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f45277fe700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4510004850 0x7f4510004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f45277fe700 1 -- 172.21.15.97:0/3196600397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510005410 msgr2=0x7f451000ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f45277fe700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510005410 0x7f451000ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f45277fe700 1 -- 172.21.15.97:0/3196600397 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528070e90 con 0x7f4510003a00 2023-11-24T06:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f4527fff700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510005410 0x7f451000ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f4527fff700 1 -- 172.21.15.97:0/3196600397 reap_dead start 2023-11-24T06:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7fada18ad700 1 Processor -- start 2023-11-24T06:09:04.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7fada18ad700 1 -- start start 2023-11-24T06:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.332+0000 7f02ee929700 1 -- start start 2023-11-24T06:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f124b7fe700 1 -- 172.21.15.97:0/172978648 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400e070 con 0x7f124c0d44e0 2023-11-24T06:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f124b7fe700 1 -- 172.21.15.97:0/172978648 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124400bad0 con 0x7f124c0d44e0 2023-11-24T06:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f124b7fe700 1 -- 172.21.15.97:0/172978648 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400be80 con 0x7f124c0d44e0 2023-11-24T06:09:04.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f02ee929700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f02e80cf440 0x7f02e806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f02ee929700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02e8070fd0 0x7f02e8165e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f02ee929700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e81663d0 0x7f02e81628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f02ee929700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e8166880 con 0x7f02e8070fd0 2023-11-24T06:09:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f02ee929700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e8072110 con 0x7f02e80cf440 2023-11-24T06:09:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f02ee929700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e806ee00 con 0x7f02e81663d0 2023-11-24T06:09:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7fd47a59c700 1 --2- 172.21.15.97:0/3432489881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c0b3b50 0x7fd47c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7fd47959a700 1 -- 172.21.15.97:0/3432489881 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338460774 0 0) 0x7fd47c06da70 con 0x7fd47c164a90 2023-11-24T06:09:04.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7fd47959a700 1 -- 172.21.15.97:0/3432489881 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4640035b0 con 0x7fd47c164a90 2023-11-24T06:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f02e80cf440 0x7f02e806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f02e80cf440 0x7f02e806e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39308/0 (socket says 172.21.15.97:39308) 2023-11-24T06:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 -- 172.21.15.97:0/1979847202 learned_addr learned my addr 172.21.15.97:0/1979847202 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e77fe700 1 --2- 172.21.15.97:0/1979847202 >> v2:172.21.15.97:3300/0 conn(0x7f02e8070fd0 0x7f02e8165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02ecec6700 1 --2- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e81663d0 0x7f02e81628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 -- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e81663d0 msgr2=0x7f02e81628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 --2- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e81663d0 0x7f02e81628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 -- 172.21.15.97:0/1979847202 >> v2:172.21.15.97:3300/0 conn(0x7f02e8070fd0 msgr2=0x7f02e8165e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 --2- 172.21.15.97:0/1979847202 >> v2:172.21.15.97:3300/0 conn(0x7f02e8070fd0 0x7f02e8165e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 -- 172.21.15.97:0/1979847202 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e8071500 con 0x7f02e80cf440 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02ecec6700 1 --2- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e81663d0 0x7f02e81628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f02e7fff700 1 --2- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 0x7f02e806e8c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02e80b5cb0 tx=0x7f02d8009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7af39da435be9f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.333+0000 7f45277fe700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4510003a00 0x7f4510003e00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f452000a8b0 tx=0x7f4510005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3da690208de74f14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520012600 con 0x7f4510003a00 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45200127a0 con 0x7f4510003a00 2023-11-24T06:09:04.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.334+0000 7f45267fc700 1 -- 172.21.15.97:0/3196600397 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520012b50 con 0x7f4510003a00 2023-11-24T06:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fd47a59c700 1 -- 172.21.15.97:0/3432489881 >> v1:172.21.15.183:6789/0 conn(0x7fd47c164a90 legacy=0x7fd47c16bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fd47a59c700 1 -- 172.21.15.97:0/3432489881 >> v1:172.21.15.97:6789/0 conn(0x7fd47c06d4d0 legacy=0x7fd47c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fd47a59c700 1 -- 172.21.15.97:0/3432489881 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c1634e0 con 0x7fd47c0b3b50 2023-11-24T06:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f125305c700 1 -- 172.21.15.97:0/172978648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0d44e0 msgr2=0x7f124c0d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f125305c700 1 --2- 172.21.15.97:0/172978648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0d44e0 0x7f124c0d13a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1244009d90 tx=0x7f124400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fada18ad700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c0b4260 0x7fad9c15d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fada18ad700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c15d970 con 0x7fad9c0714d0 2023-11-24T06:09:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fada18ad700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c0723c0 con 0x7fad9c0d0520 2023-11-24T06:09:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fada18ad700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c072d60 con 0x7fad9c0b4260 2023-11-24T06:09:04.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f125305c700 1 -- 172.21.15.97:0/172978648 shutdown_connections 2023-11-24T06:09:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f125305c700 1 --2- 172.21.15.97:0/172978648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0d44e0 0x7f124c0d13a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f125305c700 1 --2- 172.21.15.97:0/172978648 >> v2:172.21.15.97:3300/0 conn(0x7f124c15dfc0 0x7f124c0d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f125305c700 1 -- 172.21.15.97:0/172978648 >> 172.21.15.97:0/172978648 conn(0x7f124c06b410 msgr2=0x7f124c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f125305c700 1 -- 172.21.15.97:0/172978648 shutdown_connections 2023-11-24T06:09:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f125305c700 1 -- 172.21.15.97:0/172978648 wait complete. 2023-11-24T06:09:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fad9affd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fad9c0d0520 0x7fad9c0b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46932/0 (socket says 172.21.15.97:46932) 2023-11-24T06:09:04.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7fad9affd700 1 -- 172.21.15.97:0/1217818198 learned_addr learned my addr 172.21.15.97:0/1217818198 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652911234 0 0) 0x7fad9c0723c0 con 0x7fad9c0d0520 2023-11-24T06:09:04.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad840035b0 con 0x7fad9c0d0520 2023-11-24T06:09:04.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fd47a59c700 1 --2- 172.21.15.97:0/3432489881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c0b3b50 0x7fd47c0b3f30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009870 tx=0x7fd46c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60fb9720c74af2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f02e6ffd700 1 -- 172.21.15.97:0/1979847202 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8011830 con 0x7f02e80cf440 2023-11-24T06:09:04.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f02e6ffd700 1 -- 172.21.15.97:0/1979847202 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d8011990 con 0x7f02e80cf440 2023-11-24T06:09:04.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f02e6ffd700 1 -- 172.21.15.97:0/1979847202 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d8011d20 con 0x7f02e80cf440 2023-11-24T06:09:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 -- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 msgr2=0x7f02e806e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 --2- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 0x7f02e806e8c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02e80b5cb0 tx=0x7f02d8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f452e050700 1 -- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4510003a00 msgr2=0x7f4510003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.335+0000 7f452e050700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4510003a00 0x7f4510003e00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f452000a8b0 tx=0x7f4510005880 comp rx=0 tx=0).stop 2023-11-24T06:09:04.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 -- 172.21.15.97:0/1979847202 shutdown_connections 2023-11-24T06:09:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 --2- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e81663d0 0x7f02e81628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 --2- 172.21.15.97:0/1979847202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 0x7f02e806e8c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 --2- 172.21.15.97:0/1979847202 >> v2:172.21.15.97:3300/0 conn(0x7f02e8070fd0 0x7f02e8165e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 -- 172.21.15.97:0/1979847202 >> 172.21.15.97:0/1979847202 conn(0x7f02e80b1740 msgr2=0x7f02e80b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 -- 172.21.15.97:0/1979847202 shutdown_connections 2023-11-24T06:09:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fd47959a700 1 -- 172.21.15.97:0/3432489881 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c011830 con 0x7fd47c0b3b50 2023-11-24T06:09:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fd47959a700 1 -- 172.21.15.97:0/3432489881 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c011990 con 0x7fd47c0b3b50 2023-11-24T06:09:04.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fd47959a700 1 -- 172.21.15.97:0/3432489881 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c011d20 con 0x7fd47c0b3b50 2023-11-24T06:09:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f02ee929700 1 -- 172.21.15.97:0/1979847202 wait complete. 2023-11-24T06:09:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1319595043 0 0) 0x7fad840035b0 con 0x7fad9c0d0520 2023-11-24T06:09:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c0b4260 msgr2=0x7fad9c15d430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:09:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c0b4260 0x7fad9c15d430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 >> v1:172.21.15.97:6789/0 conn(0x7fad9c0714d0 legacy=0x7fad9c140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c160ba0 con 0x7fad9c0d0520 2023-11-24T06:09:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fad900016b0 con 0x7fad9c0d0520 2023-11-24T06:09:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 >> v1:172.21.15.183:6789/0 conn(0x7fad9c0d0520 legacy=0x7fad9c0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad84003a00 0x7fad84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad84004850 0x7fad84004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f452e050700 1 -- 172.21.15.97:0/3196600397 shutdown_connections 2023-11-24T06:09:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f452e050700 1 --2- 172.21.15.97:0/3196600397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4510003a00 0x7f4510003e00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f452e050700 1 -- 172.21.15.97:0/3196600397 >> 172.21.15.97:0/3196600397 conn(0x7f45280cfa30 msgr2=0x7f45280cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c0b4260 0x7fad84005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c072d60 con 0x7fad84004850 2023-11-24T06:09:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8400bba0 con 0x7fad84003a00 2023-11-24T06:09:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8400bd60 con 0x7fad9c0b4260 2023-11-24T06:09:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f452e050700 1 -- 172.21.15.97:0/3196600397 shutdown_connections 2023-11-24T06:09:04.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.336+0000 7f452e050700 1 -- 172.21.15.97:0/3196600397 wait complete. 2023-11-24T06:09:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.337+0000 7fad9a7fc700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad84004850 0x7fad84004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.337+0000 7fad9b7fe700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c0b4260 0x7fad84005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.337+0000 7fad9affd700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad84003a00 0x7fad84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.337+0000 7fd480e6c700 1 -- 172.21.15.97:0/3432489881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c0b3b50 msgr2=0x7fd47c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.337+0000 7fd480e6c700 1 --2- 172.21.15.97:0/3432489881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c0b3b50 0x7fd47c0b3f30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009870 tx=0x7fd46c009b80 comp rx=0 tx=0).stop 2023-11-24T06:09:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.337+0000 7f125305c700 1 Processor -- start 2023-11-24T06:09:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.337+0000 7f125305c700 1 -- start start 2023-11-24T06:09:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fb3a40e3700 1 Processor -- start 2023-11-24T06:09:04.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fad9a7fc700 1 -- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c0b4260 msgr2=0x7fad84005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fad9a7fc700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c0b4260 0x7fad84005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fad9a7fc700 1 -- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad84003a00 msgr2=0x7fad84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fad9a7fc700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad84003a00 0x7fad84003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fad9a7fc700 1 -- 172.21.15.97:0/1217818198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c160ba0 con 0x7fad84004850 2023-11-24T06:09:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fad9a7fc700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad84004850 0x7fad84004cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad94009e60 tx=0x7fad84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef453719953e6c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fb3a40e3700 1 -- start start 2023-11-24T06:09:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fd480e6c700 1 -- 172.21.15.97:0/3432489881 shutdown_connections 2023-11-24T06:09:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fd480e6c700 1 --2- 172.21.15.97:0/3432489881 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c0b3b50 0x7fd47c0b3f30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7fd480e6c700 1 -- 172.21.15.97:0/3432489881 >> 172.21.15.97:0/3432489881 conn(0x7fd47c0cfa30 msgr2=0x7fd47c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7f452e050700 1 Processor -- start 2023-11-24T06:09:04.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7f125305c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c0b2fc0 0x7f124c14b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7f125305c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0d44e0 0x7f124c15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7f125305c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c15dfc0 0x7f124c160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7f125305c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0c93e0 con 0x7f124c0d44e0 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7f125305c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c159b10 con 0x7f124c0b2fc0 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7f125305c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0c87d0 con 0x7f124c15dfc0 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c0b2fc0 0x7f124c14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c0b2fc0 0x7f124c14b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39326/0 (socket says 172.21.15.97:39326) 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 -- 172.21.15.97:0/594583739 learned_addr learned my addr 172.21.15.97:0/594583739 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7fd480e6c700 1 -- 172.21.15.97:0/3432489881 shutdown_connections 2023-11-24T06:09:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7fd480e6c700 1 -- 172.21.15.97:0/3432489881 wait complete. 2023-11-24T06:09:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f124bfff700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0d44e0 0x7f124c15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 -- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c15dfc0 msgr2=0x7f124c160530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:09:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c15dfc0 0x7f124c160530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 -- 172.21.15.97:0/594583739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0d44e0 msgr2=0x7f124c15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0d44e0 0x7f124c15fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 -- 172.21.15.97:0/594583739 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009a70 con 0x7f124c0b2fc0 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f1250df8700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c0b2fc0 0x7f124c14b190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f124000b530 tx=0x7f124000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7fb3a40e3700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb39c0cf440 0x7fb39c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7fb3a40e3700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb39c070fd0 0x7fb39c165e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7fb3a40e3700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39c072110 con 0x7fb39c1663d0 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7fb3a40e3700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c166860 con 0x7fb39c070fd0 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7fb3a40e3700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c06ee00 con 0x7fb39c0cf440 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7fb3a2680700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fb39c1663d0 0x7fb39c1628a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:46950/0 (socket says 172.21.15.97:46950) 2023-11-24T06:09:04.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7fb3a2680700 1 -- 172.21.15.97:0/1778785465 learned_addr learned my addr 172.21.15.97:0/1778785465 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7fb3a1e7f700 1 --2- 172.21.15.97:0/1778785465 >> v2:172.21.15.183:3300/0 conn(0x7fb39c0cf440 0x7fb39c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7fb3a0e7d700 1 -- 172.21.15.97:0/1778785465 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915490018 0 0) 0x7fb39c072110 con 0x7fb39c1663d0 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7fb3a0e7d700 1 -- 172.21.15.97:0/1778785465 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3880035b0 con 0x7fb39c1663d0 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7fb3a167e700 1 --2- 172.21.15.97:0/1778785465 >> v2:172.21.15.97:3300/0 conn(0x7fb39c070fd0 0x7fb39c165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.339+0000 7f452e050700 1 -- start start 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f452e050700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510003a00 0x7f452814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f452e050700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281f52a0 0x7f45281f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f452e050700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4528135cb0 0x7f4528136160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f452e050700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45281f5fd0 con 0x7f4510003a00 2023-11-24T06:09:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f452e050700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528133d30 con 0x7f45281f52a0 2023-11-24T06:09:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f452e050700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4528134010 con 0x7f4528135cb0 2023-11-24T06:09:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.338+0000 7f02ee929700 1 Processor -- start 2023-11-24T06:09:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12400118f0 con 0x7f124c0b2fc0 2023-11-24T06:09:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c0c8ab0 con 0x7f124c0b2fc0 2023-11-24T06:09:04.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1240011a70 con 0x7f124c0b2fc0 2023-11-24T06:09:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c0c9070 con 0x7f124c0b2fc0 2023-11-24T06:09:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1240011e20 con 0x7f124c0b2fc0 2023-11-24T06:09:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f122c000fb0 con 0x7f124c0b2fc0 2023-11-24T06:09:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7fb3a1e7f700 1 -- 172.21.15.97:0/1778785465 >> v2:172.21.15.97:3300/0 conn(0x7fb39c070fd0 msgr2=0x7fb39c165e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7fb3a1e7f700 1 --2- 172.21.15.97:0/1778785465 >> v2:172.21.15.97:3300/0 conn(0x7fb39c070fd0 0x7fb39c165e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7fb3a1e7f700 1 -- 172.21.15.97:0/1778785465 >> v1:172.21.15.183:6789/0 conn(0x7fb39c1663d0 legacy=0x7fb39c1628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7fb3a1e7f700 1 -- 172.21.15.97:0/1778785465 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c071500 con 0x7fb39c0cf440 2023-11-24T06:09:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f45277fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510003a00 0x7f452814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f45277fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510003a00 0x7f452814aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45182/0 (socket says 172.21.15.97:45182) 2023-11-24T06:09:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f45277fe700 1 -- 172.21.15.97:0/3467582375 learned_addr learned my addr 172.21.15.97:0/3467582375 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f4526ffd700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281f52a0 0x7f45281f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f4527fff700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4528135cb0 0x7f4528136160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f45277fe700 1 -- 172.21.15.97:0/3467582375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4528135cb0 msgr2=0x7f4528136160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f45277fe700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4528135cb0 0x7f4528136160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f45277fe700 1 -- 172.21.15.97:0/3467582375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281f52a0 msgr2=0x7f45281f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f45277fe700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45281f52a0 0x7f45281f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7fd480e6c700 1 Processor -- start 2023-11-24T06:09:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f45277fe700 1 -- 172.21.15.97:0/3467582375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45281342f0 con 0x7f4510003a00 2023-11-24T06:09:04.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7fb3a1e7f700 1 --2- 172.21.15.97:0/1778785465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c0cf440 0x7fb39c06e8c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0b5cb0 tx=0x7fb39400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab81411db7b1a823 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7fd480e6c700 1 -- start start 2023-11-24T06:09:04.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.340+0000 7f02ee929700 1 -- start start 2023-11-24T06:09:04.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f02ee929700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e8070fd0 0x7f02e815e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f02ee929700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 0x7f02e815f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f02ee929700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e81663d0 0x7f02e80ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f02ee929700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e815d000 con 0x7f02e81663d0 2023-11-24T06:09:04.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f02ee929700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e815d2e0 con 0x7f02e80cf440 2023-11-24T06:09:04.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.341+0000 7f02ee929700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e815d5c0 con 0x7f02e8070fd0 2023-11-24T06:09:04.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02ecec6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e81663d0 0x7f02e80ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02ecec6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e81663d0 0x7f02e80ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45186/0 (socket says 172.21.15.97:45186) 2023-11-24T06:09:04.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02ecec6700 1 -- 172.21.15.97:0/3072003496 learned_addr learned my addr 172.21.15.97:0/3072003496 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02e77fe700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 0x7f02e815f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f45277fe700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510003a00 0x7f452814aa30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4520012000 tx=0x7f4520001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7fb3a0e7d700 1 -- 172.21.15.97:0/1778785465 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39400e070 con 0x7fb39c0cf440 2023-11-24T06:09:04.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7fb3a0e7d700 1 -- 172.21.15.97:0/1778785465 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39400bb20 con 0x7fb39c0cf440 2023-11-24T06:09:04.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7fb3a0e7d700 1 -- 172.21.15.97:0/1778785465 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394012430 con 0x7fb39c0cf440 2023-11-24T06:09:04.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02e7fff700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e8070fd0 0x7f02e815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02e77fe700 1 -- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e8070fd0 msgr2=0x7f02e815e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02e77fe700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02e8070fd0 0x7f02e815e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02e77fe700 1 -- 172.21.15.97:0/3072003496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e81663d0 msgr2=0x7f02e80ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02e77fe700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02e81663d0 0x7f02e80ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f02e77fe700 1 -- 172.21.15.97:0/3072003496 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d8009550 con 0x7f02e80cf440 2023-11-24T06:09:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fb3a40e3700 1 -- 172.21.15.97:0/1778785465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c0cf440 msgr2=0x7fb39c06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fb3a40e3700 1 --2- 172.21.15.97:0/1778785465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c0cf440 0x7fb39c06e8c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0b5cb0 tx=0x7fb39400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd480e6c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c06d4d0 0x7fd47c1fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd480e6c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd47c0b3b50 0x7fd47c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd480e6c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47c164a90 0x7fd47c152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd480e6c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c2009e0 con 0x7fd47c164a90 2023-11-24T06:09:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd480e6c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c1fe1d0 con 0x7fd47c0b3b50 2023-11-24T06:09:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd480e6c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c200c80 con 0x7fd47c06d4d0 2023-11-24T06:09:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f450ffff700 1 -- 172.21.15.97:0/3467582375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452001d620 con 0x7f4510003a00 2023-11-24T06:09:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f450ffff700 1 -- 172.21.15.97:0/3467582375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452001d780 con 0x7f4510003a00 2023-11-24T06:09:04.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f450ffff700 1 -- 172.21.15.97:0/3467582375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452001daf0 con 0x7f4510003a00 2023-11-24T06:09:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fb3a40e3700 1 -- 172.21.15.97:0/1778785465 shutdown_connections 2023-11-24T06:09:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fb3a40e3700 1 --2- 172.21.15.97:0/1778785465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c0cf440 0x7fb39c06e8c0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0b5cb0 tx=0x7fb39400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fb3a40e3700 1 --2- 172.21.15.97:0/1778785465 >> v2:172.21.15.97:3300/0 conn(0x7fb39c070fd0 0x7fb39c165e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fb3a40e3700 1 -- 172.21.15.97:0/1778785465 >> 172.21.15.97:0/1778785465 conn(0x7fb39c0b1740 msgr2=0x7fb39c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fb3a40e3700 1 -- 172.21.15.97:0/1778785465 shutdown_connections 2023-11-24T06:09:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fb3a40e3700 1 -- 172.21.15.97:0/1778785465 wait complete. 2023-11-24T06:09:04.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd479d9b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd47c0b3b50 0x7fd47c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd47a59c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c06d4d0 0x7fd47c1fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45281345d0 con 0x7f4510003a00 2023-11-24T06:09:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452814af70 con 0x7f4510003a00 2023-11-24T06:09:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.342+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f124001a400 con 0x7f124c0b2fc0 2023-11-24T06:09:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f12497fa700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f123006aed0 0x7f123006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12400896f0 con 0x7f124c0b2fc0 2023-11-24T06:09:04.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f12497fa700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f12300730f0 0x7f1230075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 --> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1230075bc0 con 0x7f12300730f0 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f124008bd10 con 0x7f124c0b2fc0 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f124bfff700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f123006aed0 0x7f123006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f12515f9700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f12300730f0 0x7f1230075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7f12515f9700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f12300730f0 0x7f1230075510 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7f124bfff700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f123006aed0 0x7f123006d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f12380012e0 tx=0x7f1238001270 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9400baa0 con 0x7fad84004850 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9400bc20 con 0x7fad84004850 2023-11-24T06:09:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fad99ffb700 1 -- 172.21.15.97:0/1217818198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94015420 con 0x7fad84004850 2023-11-24T06:09:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd47a59c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c06d4d0 0x7fd47c1fdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60458/0 (socket says 172.21.15.97:60458) 2023-11-24T06:09:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7fd47a59c700 1 -- 172.21.15.97:0/2475055909 learned_addr learned my addr 172.21.15.97:0/2475055909 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fd47ad9d700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47c164a90 0x7fd47c152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fd47ad9d700 1 -- 172.21.15.97:0/2475055909 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c06d4d0 msgr2=0x7fd47c1fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fd47ad9d700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd47c06d4d0 0x7fd47c1fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fd47ad9d700 1 -- 172.21.15.97:0/2475055909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd47c0b3b50 msgr2=0x7fd47c1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fd47ad9d700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd47c0b3b50 0x7fd47c1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fd47ad9d700 1 -- 172.21.15.97:0/2475055909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c0094a0 con 0x7fd47c164a90 2023-11-24T06:09:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fada18ad700 1 -- 172.21.15.97:0/1217818198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad84004850 msgr2=0x7fad84004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fd47ad9d700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47c164a90 0x7fd47c152af0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd47c164f20 tx=0x7fd47400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fada18ad700 1 --2- 172.21.15.97:0/1217818198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad84004850 0x7fad84004cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad94009e60 tx=0x7fad84005880 comp rx=0 tx=0).stop 2023-11-24T06:09:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fad9b7fe700 1 -- 172.21.15.97:0/1217818198 reap_dead start 2023-11-24T06:09:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.343+0000 7f02e77fe700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 0x7f02e815f3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e870 tx=0x7f02dc00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc013680 con 0x7f02e80cf440 2023-11-24T06:09:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc0137e0 con 0x7f02e80cf440 2023-11-24T06:09:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc013b50 con 0x7f02e80cf440 2023-11-24T06:09:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fada18ad700 1 -- 172.21.15.97:0/1217818198 shutdown_connections 2023-11-24T06:09:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7fada18ad700 1 -- 172.21.15.97:0/1217818198 >> 172.21.15.97:0/1217818198 conn(0x7fad9c0cfa30 msgr2=0x7fad9c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.344+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e815d8a0 con 0x7f02e80cf440 2023-11-24T06:09:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.345+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e815f8e0 con 0x7f02e80cf440 2023-11-24T06:09:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.345+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 <== osd.6 v2:172.21.15.183:6816/3075593088 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1230075bc0 con 0x7f12300730f0 2023-11-24T06:09:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.345+0000 7fb3a40e3700 1 Processor -- start 2023-11-24T06:09:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.345+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47400a9e0 con 0x7fd47c164a90 2023-11-24T06:09:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.345+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47400ab40 con 0x7fd47c164a90 2023-11-24T06:09:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.345+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47400ce80 con 0x7fd47c164a90 2023-11-24T06:09:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.345+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c200f60 con 0x7fd47c164a90 2023-11-24T06:09:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.345+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c201520 con 0x7fd47c164a90 2023-11-24T06:09:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.346+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd47c165bf0 con 0x7fd47c164a90 2023-11-24T06:09:04.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7f450ffff700 1 -- 172.21.15.97:0/3467582375 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f452001dc50 con 0x7f4510003a00 2023-11-24T06:09:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02e8071da0 con 0x7f02e80cf440 2023-11-24T06:09:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.346+0000 7fb3a40e3700 1 -- start start 2023-11-24T06:09:04.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7fada18ad700 1 -- 172.21.15.97:0/1217818198 shutdown_connections 2023-11-24T06:09:04.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7fb3a40e3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb39c070fd0 0x7fb39c1fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7fb3a40e3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb39c1663d0 0x7fb39c1550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02dc013cb0 con 0x7f02e80cf440 2023-11-24T06:09:04.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7fb3a40e3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c204560 0x7fb39c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7fb3a40e3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c155780 con 0x7fb39c1663d0 2023-11-24T06:09:04.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7fb3a40e3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c161ca0 con 0x7fb39c204560 2023-11-24T06:09:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7fb3a40e3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c161f80 con 0x7fb39c070fd0 2023-11-24T06:09:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7f5bdf64e700 1 Processor -- start 2023-11-24T06:09:04.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7f5bdf64e700 1 -- start start 2023-11-24T06:09:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7f02e4ff9700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f02d406afa0 0x7f02d406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02dc089460 con 0x7f02e80cf440 2023-11-24T06:09:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7f02e4ff9700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f02d40731c0 0x7f02d40755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 --> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02d4075c90 con 0x7f02d40731c0 2023-11-24T06:09:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02dc08ba80 con 0x7f02e80cf440 2023-11-24T06:09:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7f02ecec6700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f02d40731c0 0x7f02d40755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7fb3a2680700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c204560 0x7fb39c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7fb3a2680700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c204560 0x7fb39c152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39370/0 (socket says 172.21.15.97:39370) 2023-11-24T06:09:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7fb3a2680700 1 -- 172.21.15.97:0/4050060443 learned_addr learned my addr 172.21.15.97:0/4050060443 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7fb3a1e7f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb39c070fd0 0x7fb39c1fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7f02e7fff700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f02d406afa0 0x7f02d406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3a1e7f700 1 -- 172.21.15.97:0/4050060443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c204560 msgr2=0x7fb39c152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3a1e7f700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb39c204560 0x7fb39c152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3a1e7f700 1 -- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb39c1663d0 msgr2=0x7fb39c1550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3a1e7f700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb39c1663d0 0x7fb39c1550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3a1e7f700 1 -- 172.21.15.97:0/4050060443 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb394009a70 con 0x7fb39c070fd0 2023-11-24T06:09:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7f450ffff700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f451406af50 0x7f451406d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7f450ffff700 1 -- 172.21.15.97:0/3467582375 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f452008b820 con 0x7f4510003a00 2023-11-24T06:09:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7f4526ffd700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f451406af50 0x7f451406d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7f4526ffd700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f451406af50 0x7f451406d410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f452814a790 tx=0x7f4518009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f02e7fff700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f02d406afa0 0x7f02d406d460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009c10 tx=0x7f02d801d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f5bdf64e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5bd80b4260 0x7f5bd80714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.347+0000 7fada18ad700 1 -- 172.21.15.97:0/1217818198 wait complete. 2023-11-24T06:09:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fada18ad700 1 Processor -- start 2023-11-24T06:09:04.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f02ecec6700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f02d40731c0 0x7f02d40755e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f5bdf64e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd80720e0 con 0x7f5bd815d6a0 2023-11-24T06:09:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f5bdf64e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd80723c0 con 0x7f5bd8071a10 2023-11-24T06:09:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f5bdf64e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd8072d60 con 0x7f5bd80b4260 2023-11-24T06:09:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f5bdd3ea700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5bd815d6a0 0x7f5bd80b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37664/0 (socket says 172.21.15.97:37664) 2023-11-24T06:09:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f5bdd3ea700 1 -- 172.21.15.97:0/3356483830 learned_addr learned my addr 172.21.15.97:0/3356483830 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3a1e7f700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb39c070fd0 0x7fb39c1fdbd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb394016000 tx=0x7fb394004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7f5bdcbe9700 1 --2- 172.21.15.97:0/3356483830 >> v2:172.21.15.97:3300/0 conn(0x7f5bd80b4260 0x7f5bd80714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940048b0 con 0x7fb39c070fd0 2023-11-24T06:09:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394020070 con 0x7fb39c070fd0 2023-11-24T06:09:04.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39c200650 con 0x7fb39c070fd0 2023-11-24T06:09:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.348+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd47401b3b0 con 0x7fd47c164a90 2023-11-24T06:09:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fd46affd700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd45c06afa0 0x7fd45c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fd47a59c700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd45c06afa0 0x7fd45c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4740890f0 con 0x7fd47c164a90 2023-11-24T06:09:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fd46affd700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fd45c0731c0 0x7fd45c0755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 --> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd45c075c90 con 0x7fd45c0731c0 2023-11-24T06:09:04.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd47408b710 con 0x7fd47c164a90 2023-11-24T06:09:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940276c0 con 0x7fb39c070fd0 2023-11-24T06:09:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c162260 con 0x7fb39c070fd0 2023-11-24T06:09:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f452e050700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f45281638a0 0x7f4528203ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 --> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45280b15a0 con 0x7f45281638a0 2023-11-24T06:09:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f5bcffff700 1 -- 172.21.15.97:0/3356483830 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287369373 0 0) 0x7f5bd80720e0 con 0x7f5bd815d6a0 2023-11-24T06:09:04.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f5bdcbe9700 1 -- 172.21.15.97:0/3356483830 >> v1:172.21.15.183:6789/0 conn(0x7f5bd8071a10 legacy=0x7f5bd8140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fd47a59c700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd45c06afa0 0x7fd45c06d460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009bc0 tx=0x7fd46c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f5bdcbe9700 1 -- 172.21.15.97:0/3356483830 >> v1:172.21.15.97:6789/0 conn(0x7f5bd815d6a0 legacy=0x7f5bd80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f5bdcbe9700 1 -- 172.21.15.97:0/3356483830 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd80726a0 con 0x7f5bd80b4260 2023-11-24T06:09:04.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fd479d9b700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fd45c0731c0 0x7fd45c0755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f4527fff700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f45281638a0 0x7f4528203ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fd479d9b700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fd45c0731c0 0x7fd45c0755e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f5bdcbe9700 1 --2- 172.21.15.97:0/3356483830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bd80b4260 0x7f5bd80714d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4008840 tx=0x7f5bd400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6e032a376f2ed8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7f4527fff700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f45281638a0 0x7f4528203ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 <== osd.5 v2:172.21.15.183:6817/4148254452 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f02d4075c90 con 0x7f02d40731c0 2023-11-24T06:09:04.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb384000fb0 con 0x7fb39c070fd0 2023-11-24T06:09:04.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb394012430 con 0x7fb39c070fd0 2023-11-24T06:09:04.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fb3927fc700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb38006af20 0x7fb38006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb394022070 con 0x7fb39c070fd0 2023-11-24T06:09:04.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fb3927fc700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7fb3800730c0 0x7fb3800754e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 --> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb380075b90 con 0x7fb3800730c0 2023-11-24T06:09:04.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb3a2680700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7fb3800730c0 0x7fb3800754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7f450ffff700 1 -- 172.21.15.97:0/3467582375 <== osd.2 v2:172.21.15.97:6816/1638045649 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f45280b15a0 con 0x7f45281638a0 2023-11-24T06:09:04.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.349+0000 7fada18ad700 1 -- start start 2023-11-24T06:09:04.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fada18ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c200250 0x7fad9c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fada18ad700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c0a9510 0x7fad9c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fada18ad700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c15c930 0x7fad9c151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fada18ad700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c1544e0 con 0x7fad9c200250 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fada18ad700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c15cdc0 con 0x7fad9c0a9510 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.350+0000 7fada18ad700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c153860 con 0x7fad9c15c930 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9a7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c0a9510 0x7fad9c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9a7fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c0a9510 0x7fad9c15c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39386/0 (socket says 172.21.15.97:39386) 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c200250 0x7fad9c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9a7fc700 1 -- 172.21.15.97:0/2831911494 learned_addr learned my addr 172.21.15.97:0/2831911494 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9affd700 1 -- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c15c930 msgr2=0x7fad9c151ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9affd700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad9c15c930 0x7fad9c151ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9affd700 1 -- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c0a9510 msgr2=0x7fad9c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9affd700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c0a9510 0x7fad9c15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9affd700 1 -- 172.21.15.97:0/2831911494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c153b40 con 0x7fad9c200250 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9a7fc700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c0a9510 0x7fad9c15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fad9affd700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c200250 0x7fad9c152ef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fad9000e7b0 tx=0x7fad9000eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90035630 con 0x7fad9c200250 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad90035790 con 0x7fad9c200250 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9c153e20 con 0x7fad9c200250 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90035b40 con 0x7fad9c200250 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.352+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c154900 con 0x7fad9c200250 2023-11-24T06:09:04.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3940098f0 con 0x7fb39c070fd0 2023-11-24T06:09:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad7c000fb0 con 0x7fad9c200250 2023-11-24T06:09:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad90035ca0 con 0x7fad9c200250 2023-11-24T06:09:04.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 <== osd.1 v2:172.21.15.97:6808/3026562872 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd45c075c90 con 0x7fd45c0731c0 2023-11-24T06:09:04.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fb3a167e700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb38006af20 0x7fb38006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7f5bcffff700 1 -- 172.21.15.97:0/3356483830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4010070 con 0x7f5bd80b4260 2023-11-24T06:09:04.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7f5bcffff700 1 -- 172.21.15.97:0/3356483830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd4004030 con 0x7f5bd80b4260 2023-11-24T06:09:04.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7f5bcffff700 1 -- 172.21.15.97:0/3356483830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd40043c0 con 0x7f5bd80b4260 2023-11-24T06:09:04.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fad83fff700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad8406c630 0x7fad8406eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fad9a7fc700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad8406c630 0x7fad8406eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad900ab2b0 con 0x7fad9c200250 2023-11-24T06:09:04.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fad83fff700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7fad840747d0 0x7fad84076bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 --> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad840772a0 con 0x7fad840747d0 2023-11-24T06:09:04.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad900ad970 con 0x7fad9c200250 2023-11-24T06:09:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fad9a7fc700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad8406c630 0x7fad8406eaf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad94009fd0 tx=0x7fad94011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7f5bdf64e700 1 -- 172.21.15.97:0/3356483830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bd80b4260 msgr2=0x7f5bd80714d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7f5bdf64e700 1 --2- 172.21.15.97:0/3356483830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bd80b4260 0x7f5bd80714d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4008840 tx=0x7f5bd400d330 comp rx=0 tx=0).stop 2023-11-24T06:09:04.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fad9b7fe700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7fad840747d0 0x7fad84076bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fad9b7fe700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7fad840747d0 0x7fad84076bf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.356+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 <== osd.7 v2:172.21.15.183:6808/4128955131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fad840772a0 con 0x7fad840747d0 2023-11-24T06:09:04.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.358+0000 7f5bdf64e700 1 -- 172.21.15.97:0/3356483830 shutdown_connections 2023-11-24T06:09:04.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.358+0000 7f5bdf64e700 1 --2- 172.21.15.97:0/3356483830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bd80b4260 0x7f5bd80714d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.358+0000 7f5bdf64e700 1 -- 172.21.15.97:0/3356483830 >> 172.21.15.97:0/3356483830 conn(0x7f5bd80cfa30 msgr2=0x7f5bd80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fb171600700 1 Processor -- start 2023-11-24T06:09:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.351+0000 7fb171600700 1 -- start start 2023-11-24T06:09:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb171600700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c0b3b50 0x7fb16c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb171600700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb16c06d4d0 0x7fb16c06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb171600700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16c06ced0 con 0x7fb16c163a80 2023-11-24T06:09:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb171600700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16c06da70 con 0x7fb16c06d4d0 2023-11-24T06:09:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb171600700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c0b4470 con 0x7fb16c0b3b50 2023-11-24T06:09:04.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb16affd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c0b3b50 0x7fb16c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb16affd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c0b3b50 0x7fb16c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60498/0 (socket says 172.21.15.97:60498) 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb16affd700 1 -- 172.21.15.97:0/1477138100 learned_addr learned my addr 172.21.15.97:0/1477138100 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb16affd700 1 -- 172.21.15.97:0/1477138100 >> v2:172.21.15.97:3300/0 conn(0x7fb16c06d4d0 msgr2=0x7fb16c06eee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb16affd700 1 --2- 172.21.15.97:0/1477138100 >> v2:172.21.15.97:3300/0 conn(0x7fb16c06d4d0 0x7fb16c06eee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb16affd700 1 -- 172.21.15.97:0/1477138100 >> v1:172.21.15.183:6789/0 conn(0x7fb16c163a80 legacy=0x7fb16c163f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.353+0000 7fb16affd700 1 -- 172.21.15.97:0/1477138100 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c1634e0 con 0x7fb16c0b3b50 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.354+0000 7fb16affd700 1 --2- 172.21.15.97:0/1477138100 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c0b3b50 0x7fb16c0b3f30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb1600098f0 tx=0x7fb160009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd267c13fed965 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb169ffb700 1 -- 172.21.15.97:0/1477138100 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000b8a0 con 0x7fb16c0b3b50 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb169ffb700 1 -- 172.21.15.97:0/1477138100 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16000ba00 con 0x7fb16c0b3b50 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb169ffb700 1 -- 172.21.15.97:0/1477138100 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000bd90 con 0x7fb16c0b3b50 2023-11-24T06:09:04.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb171600700 1 -- 172.21.15.97:0/1477138100 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c0b3b50 msgr2=0x7fb16c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb171600700 1 --2- 172.21.15.97:0/1477138100 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c0b3b50 0x7fb16c0b3f30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb1600098f0 tx=0x7fb160009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb171600700 1 -- 172.21.15.97:0/1477138100 shutdown_connections 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb171600700 1 --2- 172.21.15.97:0/1477138100 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c0b3b50 0x7fb16c0b3f30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb171600700 1 --2- 172.21.15.97:0/1477138100 >> v2:172.21.15.97:3300/0 conn(0x7fb16c06d4d0 0x7fb16c06eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb171600700 1 -- 172.21.15.97:0/1477138100 >> 172.21.15.97:0/1477138100 conn(0x7fb16c0cfa30 msgr2=0x7fb16c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.355+0000 7fb171600700 1 -- 172.21.15.97:0/1477138100 shutdown_connections 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.356+0000 7fb171600700 1 -- 172.21.15.97:0/1477138100 wait complete. 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.357+0000 7fb171600700 1 Processor -- start 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.357+0000 7fb171600700 1 -- start start 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb171600700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c06d4d0 0x7fb16c13e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb171600700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c0b3b50 0x7fb16c133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb171600700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c163a80 0x7fb16c14c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb171600700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c134090 con 0x7fb16c0b3b50 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb171600700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c134370 con 0x7fb16c163a80 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb171600700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c134650 con 0x7fb16c06d4d0 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c0b3b50 0x7fb16c133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16affd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c06d4d0 0x7fb16c13e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c0b3b50 0x7fb16c133690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45242/0 (socket says 172.21.15.97:45242) 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 -- 172.21.15.97:0/1967537396 learned_addr learned my addr 172.21.15.97:0/1967537396 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16b7fe700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c163a80 0x7fb16c14c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 -- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c06d4d0 msgr2=0x7fb16c13e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c06d4d0 0x7fb16c13e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 -- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c163a80 msgr2=0x7fb16c14c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c163a80 0x7fb16c14c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 -- 172.21.15.97:0/1967537396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb160009550 con 0x7fb16c0b3b50 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16b7fe700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb16c163a80 0x7fb16c14c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16affd700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb16c06d4d0 0x7fb16c13e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.359+0000 7fb16a7fc700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c0b3b50 0x7fb16c133690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb16400ba50 tx=0x7fb16400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.360+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400c930 con 0x7fb16c0b3b50 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.360+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16400caf0 con 0x7fb16c0b3b50 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.360+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c134930 con 0x7fb16c0b3b50 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.360+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400cea0 con 0x7fb16c0b3b50 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.360+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c13ecc0 con 0x7fb16c0b3b50 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.361+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb16c165bf0 con 0x7fb16c0b3b50 2023-11-24T06:09:04.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.362+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb16401d3b0 con 0x7fb16c0b3b50 2023-11-24T06:09:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.362+0000 7fb3a167e700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb38006af20 0x7fb38006d3e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb398006b20 tx=0x7fb398006ab0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.362+0000 7fb1537fe700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb15806aed0 0x7fb15806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.362+0000 7fb16affd700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb15806aed0 0x7fb15806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.363+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb164089b10 con 0x7fb16c0b3b50 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.363+0000 7fb1537fe700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7fb158073070 0x7fb158075490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.363+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 --> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb158075b40 con 0x7fb158073070 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.363+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb164089ec0 con 0x7fb16c0b3b50 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.363+0000 7fb16affd700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb15806aed0 0x7fb15806d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb160009c40 tx=0x7fb160008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.363+0000 7fb16b7fe700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7fb158073070 0x7fb158075490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.364+0000 7fb16b7fe700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7fb158073070 0x7fb158075490 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.364+0000 7f5bdf64e700 1 -- 172.21.15.97:0/3356483830 shutdown_connections 2023-11-24T06:09:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.364+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 <== osd.4 v2:172.21.15.183:6800/1408518032 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb158075b40 con 0x7fb158073070 2023-11-24T06:09:04.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.364+0000 7fb3a2680700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7fb3800730c0 0x7fb3800754e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.364+0000 7f5bdf64e700 1 -- 172.21.15.97:0/3356483830 wait complete. 2023-11-24T06:09:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.366+0000 7f5bdf64e700 1 Processor -- start 2023-11-24T06:09:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.366+0000 7f5bdf64e700 1 -- start start 2023-11-24T06:09:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdf64e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8071a10 0x7f5bd81f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdf64e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bd80b4260 0x7f5bd81fb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdf64e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5bd815d6a0 0x7f5bd80b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdf64e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd8153650 con 0x7f5bd80b4260 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdf64e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81fc590 con 0x7f5bd8071a10 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdf64e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd81fbfd0 con 0x7f5bd815d6a0 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdd3ea700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8071a10 0x7f5bd81f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdd3ea700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8071a10 0x7f5bd81f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39410/0 (socket says 172.21.15.97:39410) 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.367+0000 7f5bdd3ea700 1 -- 172.21.15.97:0/1117404206 learned_addr learned my addr 172.21.15.97:0/1117404206 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bdd3ea700 1 -- 172.21.15.97:0/1117404206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5bd815d6a0 msgr2=0x7f5bd80b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bdd3ea700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5bd815d6a0 0x7f5bd80b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bdd3ea700 1 -- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bd80b4260 msgr2=0x7f5bd81fb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bdd3ea700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bd80b4260 0x7f5bd81fb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bdd3ea700 1 -- 172.21.15.97:0/1117404206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4008520 con 0x7f5bd8071a10 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bdd3ea700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8071a10 0x7f5bd81f7860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c550 tx=0x7f5bc800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bcdffb700 1 -- 172.21.15.97:0/1117404206 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8004c80 con 0x7f5bd8071a10 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd81fc2b0 con 0x7f5bd8071a10 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.368+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd8153b60 con 0x7f5bd8071a10 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.372+0000 7f5bcdffb700 1 -- 172.21.15.97:0/1117404206 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc8004e00 con 0x7f5bd8071a10 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.372+0000 7f5bcdffb700 1 -- 172.21.15.97:0/1117404206 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8007310 con 0x7f5bd8071a10 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.374+0000 7f5bcdffb700 1 -- 172.21.15.97:0/1117404206 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5bc8007470 con 0x7f5bd8071a10 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.374+0000 7f5bcdffb700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.374+0000 7f5bcdffb700 1 -- 172.21.15.97:0/1117404206 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bc8089240 con 0x7f5bd8071a10 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.374+0000 7f5bdcbe9700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.375+0000 7f5bdcbe9700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80b40a0 tx=0x7f5bd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.376+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 <== osd.0 v2:172.21.15.97:6800/3900364081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb380075b90 con 0x7fb3800730c0 2023-11-24T06:09:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.376+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 --> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f452806dab0 con 0x7f45281638a0 2023-11-24T06:09:04.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.377+0000 7f450ffff700 1 -- 172.21.15.97:0/3467582375 <== osd.2 v2:172.21.15.97:6816/1638045649 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f451c009900 con 0x7f45281638a0 2023-11-24T06:09:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.377+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f45281638a0 msgr2=0x7f4528203ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.377+0000 7f452e050700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f45281638a0 0x7f4528203ec0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.377+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f451406af50 msgr2=0x7f451406d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.377+0000 7f452e050700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f451406af50 0x7f451406d410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f452814a790 tx=0x7f4518009040 comp rx=0 tx=0).stop 2023-11-24T06:09:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.377+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510003a00 msgr2=0x7f452814aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.377+0000 7f452e050700 1 --2- 172.21.15.97:0/3467582375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4510003a00 0x7f452814aa30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4520012000 tx=0x7f4520001cb0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.378+0000 7f4527fff700 1 -- 172.21.15.97:0/3467582375 reap_dead start 2023-11-24T06:09:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.378+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 shutdown_connections 2023-11-24T06:09:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.378+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 >> 172.21.15.97:0/3467582375 conn(0x7f45280cfa30 msgr2=0x7f4528162f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.378+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 shutdown_connections 2023-11-24T06:09:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.378+0000 7f452e050700 1 -- 172.21.15.97:0/3467582375 wait complete. 2023-11-24T06:09:04.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.379+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 --> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad7c002d30 con 0x7fad840747d0 2023-11-24T06:09:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.379+0000 7f5bdf64e700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f5bb80015d0 0x7f5bb8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.379+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 --> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5bb8006bc0 con 0x7f5bb80015d0 2023-11-24T06:09:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.379+0000 7f5bddbeb700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f5bb80015d0 0x7f5bb8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.379+0000 7f5bddbeb700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f5bb80015d0 0x7f5bb8003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.379+0000 7fad83fff700 1 -- 172.21.15.97:0/2831911494 <== osd.7 v2:172.21.15.183:6808/4128955131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad7c002d30 con 0x7fad840747d0 2023-11-24T06:09:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.380+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 --> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f122c002d30 con 0x7f12300730f0 2023-11-24T06:09:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.380+0000 7f5bcdffb700 1 -- 172.21.15.97:0/1117404206 <== osd.3 v2:172.21.15.97:6817/4004566867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5bb8006bc0 con 0x7f5bb80015d0 2023-11-24T06:09:04.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.382+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 --> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f02e8071ac0 con 0x7f02d40731c0 2023-11-24T06:09:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.382+0000 7f12497fa700 1 -- 172.21.15.97:0/594583739 <== osd.6 v2:172.21.15.183:6816/3075593088 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f122c002d30 con 0x7f12300730f0 2023-11-24T06:09:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.382+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f12300730f0 msgr2=0x7f1230075510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.382+0000 7f125305c700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f12300730f0 0x7f1230075510 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.384+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7fad840747d0 msgr2=0x7fad84076bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.384+0000 7fada18ad700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7fad840747d0 0x7fad84076bf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.384+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad8406c630 msgr2=0x7fad8406eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.384+0000 7fada18ad700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad8406c630 0x7fad8406eaf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fad94009fd0 tx=0x7fad94011000 comp rx=0 tx=0).stop 2023-11-24T06:09:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.384+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c200250 msgr2=0x7fad9c152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.384+0000 7fada18ad700 1 --2- 172.21.15.97:0/2831911494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c200250 0x7fad9c152ef0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fad9000e7b0 tx=0x7fad9000eac0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.384+0000 7fad9b7fe700 1 -- 172.21.15.97:0/2831911494 reap_dead start 2023-11-24T06:09:04.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.387+0000 7f02e4ff9700 1 -- 172.21.15.97:0/3072003496 <== osd.5 v2:172.21.15.183:6817/4148254452 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f02e0007d50 con 0x7f02d40731c0 2023-11-24T06:09:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.387+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f02d40731c0 msgr2=0x7f02d40755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.387+0000 7f02ee929700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f02d40731c0 0x7f02d40755e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.387+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f02d406afa0 msgr2=0x7f02d406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.387+0000 7f02ee929700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f02d406afa0 0x7f02d406d460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009c10 tx=0x7f02d801d040 comp rx=0 tx=0).stop 2023-11-24T06:09:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.387+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 msgr2=0x7f02e815f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.387+0000 7f02ee929700 1 --2- 172.21.15.97:0/3072003496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02e80cf440 0x7f02e815f3a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00e870 tx=0x7f02dc00eb80 comp rx=0 tx=0).stop 2023-11-24T06:09:04.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.387+0000 7f02ecec6700 1 -- 172.21.15.97:0/3072003496 reap_dead start 2023-11-24T06:09:04.424 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-11-24T06:09:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f123006aed0 msgr2=0x7f123006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7f125305c700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f123006aed0 0x7f123006d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f12380012e0 tx=0x7f1238001270 comp rx=0 tx=0).stop 2023-11-24T06:09:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c0b2fc0 msgr2=0x7f124c14b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7f125305c700 1 --2- 172.21.15.97:0/594583739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c0b2fc0 0x7f124c14b190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f124000b530 tx=0x7f124000b840 comp rx=0 tx=0).stop 2023-11-24T06:09:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7f12515f9700 1 -- 172.21.15.97:0/594583739 reap_dead start 2023-11-24T06:09:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 shutdown_connections 2023-11-24T06:09:04.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 >> 172.21.15.97:0/594583739 conn(0x7f124c06b410 msgr2=0x7f124c06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 shutdown_connections 2023-11-24T06:09:04.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 >> 172.21.15.97:0/2831911494 conn(0x7fad9c0cfa30 msgr2=0x7fad9c15e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 shutdown_connections 2023-11-24T06:09:04.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.391+0000 7fada18ad700 1 -- 172.21.15.97:0/2831911494 wait complete. 2023-11-24T06:09:04.426 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:09:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.392+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 shutdown_connections 2023-11-24T06:09:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.392+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 >> 172.21.15.97:0/3072003496 conn(0x7f02e80b1740 msgr2=0x7f02e8164000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.392+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 shutdown_connections 2023-11-24T06:09:04.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.392+0000 7f02ee929700 1 -- 172.21.15.97:0/3072003496 wait complete. 2023-11-24T06:09:04.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.392+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 shutdown_connections 2023-11-24T06:09:04.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.392+0000 7f125305c700 1 -- 172.21.15.97:0/594583739 wait complete. 2023-11-24T06:09:04.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.397+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 --> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd47c06cb60 con 0x7fd45c0731c0 2023-11-24T06:09:04.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.397+0000 7fd46affd700 1 -- 172.21.15.97:0/2475055909 <== osd.1 v2:172.21.15.97:6808/3026562872 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd470001560 con 0x7fd45c0731c0 2023-11-24T06:09:04.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fd45c0731c0 msgr2=0x7fd45c0755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fd45c0731c0 0x7fd45c0755e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd45c06afa0 msgr2=0x7fd45c06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd45c06afa0 0x7fd45c06d460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009bc0 tx=0x7fd46c008030 comp rx=0 tx=0).stop 2023-11-24T06:09:04.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47c164a90 msgr2=0x7fd47c152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 --2- 172.21.15.97:0/2475055909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd47c164a90 0x7fd47c152af0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd47c164f20 tx=0x7fd47400cd50 comp rx=0 tx=0).stop 2023-11-24T06:09:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 --> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb16c06cb60 con 0x7fb158073070 2023-11-24T06:09:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd47ad9d700 1 -- 172.21.15.97:0/2475055909 reap_dead start 2023-11-24T06:09:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 shutdown_connections 2023-11-24T06:09:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 >> 172.21.15.97:0/2475055909 conn(0x7fd47c0cfa30 msgr2=0x7fd47c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 shutdown_connections 2023-11-24T06:09:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.398+0000 7fd480e6c700 1 -- 172.21.15.97:0/2475055909 wait complete. 2023-11-24T06:09:04.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.399+0000 7fb1537fe700 1 -- 172.21.15.97:0/1967537396 <== osd.4 v2:172.21.15.183:6800/1408518032 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb15c001830 con 0x7fb158073070 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.399+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7fb158073070 msgr2=0x7fb158075490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.399+0000 7fb171600700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7fb158073070 0x7fb158075490 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb15806aed0 msgr2=0x7fb15806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb171600700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb15806aed0 0x7fb15806d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb160009c40 tx=0x7fb160008030 comp rx=0 tx=0).stop 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c0b3b50 msgr2=0x7fb16c133690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb171600700 1 --2- 172.21.15.97:0/1967537396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c0b3b50 0x7fb16c133690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb16400ba50 tx=0x7fb16400bd60 comp rx=0 tx=0).stop 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb16b7fe700 1 -- 172.21.15.97:0/1967537396 reap_dead start 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 shutdown_connections 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 >> 172.21.15.97:0/1967537396 conn(0x7fb16c0cfa30 msgr2=0x7fb16c164c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 shutdown_connections 2023-11-24T06:09:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.402+0000 7fb171600700 1 -- 172.21.15.97:0/1967537396 wait complete. 2023-11-24T06:09:04.433 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-11-24T06:09:04.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:09:04.434 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-11-24T06:09:04.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:09:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.413+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 --> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb384002d30 con 0x7fb3800730c0 2023-11-24T06:09:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.413+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 --> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5bb8005ca0 con 0x7f5bb80015d0 2023-11-24T06:09:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.413+0000 7fb3927fc700 1 -- 172.21.15.97:0/4050060443 <== osd.0 v2:172.21.15.97:6800/3900364081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb384002d30 con 0x7fb3800730c0 2023-11-24T06:09:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bcdffb700 1 -- 172.21.15.97:0/1117404206 <== osd.3 v2:172.21.15.97:6817/4004566867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5bb8005ca0 con 0x7f5bb80015d0 2023-11-24T06:09:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7fb3800730c0 msgr2=0x7fb3800754e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7fb3a40e3700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7fb3800730c0 0x7fb3800754e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f5bb80015d0 msgr2=0x7f5bb8003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f5bb80015d0 0x7f5bb8003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb38006af20 msgr2=0x7fb38006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7fb3a40e3700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb38006af20 0x7fb38006d3e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb398006b20 tx=0x7fb398006ab0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb39c070fd0 msgr2=0x7fb39c1fdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7fb3a40e3700 1 --2- 172.21.15.97:0/4050060443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb39c070fd0 0x7fb39c1fdbd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb394016000 tx=0x7fb394004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7fb3a2680700 1 -- 172.21.15.97:0/4050060443 reap_dead start 2023-11-24T06:09:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5bbc06af00 msgr2=0x7f5bbc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5bbc06af00 0x7f5bbc06d3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80b40a0 tx=0x7f5bd4008040 comp rx=0 tx=0).stop 2023-11-24T06:09:04.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8071a10 msgr2=0x7f5bd81f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 --2- 172.21.15.97:0/1117404206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd8071a10 0x7f5bd81f7860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c550 tx=0x7f5bc800c910 comp rx=0 tx=0).stop 2023-11-24T06:09:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bddbeb700 1 -- 172.21.15.97:0/1117404206 reap_dead start 2023-11-24T06:09:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 shutdown_connections 2023-11-24T06:09:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 >> 172.21.15.97:0/1117404206 conn(0x7f5bd80cfa30 msgr2=0x7f5bd80b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 shutdown_connections 2023-11-24T06:09:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.414+0000 7f5bdf64e700 1 -- 172.21.15.97:0/1117404206 wait complete. 2023-11-24T06:09:04.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.416+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 shutdown_connections 2023-11-24T06:09:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.416+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 >> 172.21.15.97:0/4050060443 conn(0x7fb39c0b1740 msgr2=0x7fb39c0b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.416+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 shutdown_connections 2023-11-24T06:09:04.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.416+0000 7fb3a40e3700 1 -- 172.21.15.97:0/4050060443 wait complete. 2023-11-24T06:09:04.440 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-11-24T06:09:04.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:09:04.442 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-11-24T06:09:04.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:09:04.443 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-11-24T06:09:04.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:09:04.444 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-11-24T06:09:04.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:09:04.445 INFO:teuthology.orchestra.run.smithi097.stdout:55834574857 2023-11-24T06:09:04.445 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:09:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.687+0000 7f49ee090700 1 Processor -- start 2023-11-24T06:09:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.687+0000 7f3a8a04f700 1 Processor -- start 2023-11-24T06:09:04.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.687+0000 7f49ee090700 1 -- start start 2023-11-24T06:09:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f49ee090700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49e815bd40 0x7f49e80cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f49ee090700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f49e80afb20 0x7f49e815c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f49ee090700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e806d540 con 0x7f49e815c660 2023-11-24T06:09:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f49ee090700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e806d820 con 0x7f49e815bd40 2023-11-24T06:09:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f49ee090700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e806db00 con 0x7f49e80afb20 2023-11-24T06:09:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.687+0000 7f3a8a04f700 1 -- start start 2023-11-24T06:09:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.687+0000 7f4744a2a700 1 Processor -- start 2023-11-24T06:09:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.687+0000 7f4744a2a700 1 -- start start 2023-11-24T06:09:04.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f4744a2a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f474006b0d0 0x7f47401635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f4744a2a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47400c4c40 0x7f47400c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f4744a2a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47400c55e0 con 0x7f4740160030 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f4744a2a700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4740160410 con 0x7f474006b0d0 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f4744a2a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400cf1a0 con 0x7f47400c4c40 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f3a8a04f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a840b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f3a8a04f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3a840adf00 0x7f3a8406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f3a8a04f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3a8406c540 0x7f3a8406da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f3a8a04f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a84144a80 con 0x7f3a8406c540 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f3a8a04f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8406c9b0 con 0x7f3a840adf00 2023-11-24T06:09:04.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.688+0000 7f3a8a04f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840af950 con 0x7f3a841446a0 2023-11-24T06:09:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a7affd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3a840adf00 0x7f3a8406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a7affd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3a840adf00 0x7f3a8406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39432/0 (socket says 172.21.15.97:39432) 2023-11-24T06:09:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a7affd700 1 -- 172.21.15.97:0/1779541849 learned_addr learned my addr 172.21.15.97:0/1779541849 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473ed9d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47400c4c40 0x7f47400c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473ed9d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47400c4c40 0x7f47400c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60544/0 (socket says 172.21.15.97:60544) 2023-11-24T06:09:04.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473ed9d700 1 -- 172.21.15.97:0/127777591 learned_addr learned my addr 172.21.15.97:0/127777591 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473dd9b700 1 --2- 172.21.15.97:0/127777591 >> v2:172.21.15.97:3300/0 conn(0x7f474006b0d0 0x7f47401635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f49e77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49e815bd40 0x7f49e80cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f49e77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f49e815bd40 0x7f49e80cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45260/0 (socket says 172.21.15.97:45260) 2023-11-24T06:09:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f49e77fe700 1 -- 172.21.15.97:0/2339826460 learned_addr learned my addr 172.21.15.97:0/2339826460 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a83fff700 1 --2- 172.21.15.97:0/1779541849 >> v2:172.21.15.97:3300/0 conn(0x7f3a8406c540 0x7f3a8406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a837fe700 1 --2- 172.21.15.97:0/1779541849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a840b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473ed9d700 1 -- 172.21.15.97:0/127777591 >> v2:172.21.15.97:3300/0 conn(0x7f474006b0d0 msgr2=0x7f47401635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473ed9d700 1 --2- 172.21.15.97:0/127777591 >> v2:172.21.15.97:3300/0 conn(0x7f474006b0d0 0x7f47401635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473ed9d700 1 -- 172.21.15.97:0/127777591 >> v1:172.21.15.97:6789/0 conn(0x7f4740160030 legacy=0x7f47400ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473ed9d700 1 -- 172.21.15.97:0/127777591 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740162af0 con 0x7f47400c4c40 2023-11-24T06:09:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a83fff700 1 -- 172.21.15.97:0/1779541849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 msgr2=0x7f3a840b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a83fff700 1 --2- 172.21.15.97:0/1779541849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a840b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a83fff700 1 -- 172.21.15.97:0/1779541849 >> v2:172.21.15.183:3300/0 conn(0x7f3a840adf00 msgr2=0x7f3a8406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a83fff700 1 --2- 172.21.15.97:0/1779541849 >> v2:172.21.15.183:3300/0 conn(0x7f3a840adf00 0x7f3a8406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f3a83fff700 1 -- 172.21.15.97:0/1779541849 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a840afc30 con 0x7f3a8406c540 2023-11-24T06:09:04.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.689+0000 7f473ed9d700 1 --2- 172.21.15.97:0/127777591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47400c4c40 0x7f47400c50a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4738009e50 tx=0x7f473800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c45287befef290 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f49e77fe700 1 -- 172.21.15.97:0/2339826460 >> v2:172.21.15.183:3300/0 conn(0x7f49e80afb20 msgr2=0x7f49e815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f49e77fe700 1 --2- 172.21.15.97:0/2339826460 >> v2:172.21.15.183:3300/0 conn(0x7f49e80afb20 0x7f49e815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f49e77fe700 1 -- 172.21.15.97:0/2339826460 >> v1:172.21.15.97:6789/0 conn(0x7f49e815c660 legacy=0x7f49e8071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f49e77fe700 1 -- 172.21.15.97:0/2339826460 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e806dde0 con 0x7f49e815bd40 2023-11-24T06:09:04.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f3a83fff700 1 --2- 172.21.15.97:0/1779541849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a8406c540 0x7f3a8406da30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3a8406d470 tx=0x7f3a7c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5298bc0c88621c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f473d59a700 1 -- 172.21.15.97:0/127777591 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473800e070 con 0x7f47400c4c40 2023-11-24T06:09:04.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f473d59a700 1 -- 172.21.15.97:0/127777591 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4738003a20 con 0x7f47400c4c40 2023-11-24T06:09:04.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f473d59a700 1 -- 172.21.15.97:0/127777591 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738003dd0 con 0x7f47400c4c40 2023-11-24T06:09:04.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f3a82ffd700 1 -- 172.21.15.97:0/1779541849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c010a30 con 0x7f3a8406c540 2023-11-24T06:09:04.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f3a82ffd700 1 -- 172.21.15.97:0/1779541849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c010b90 con 0x7f3a8406c540 2023-11-24T06:09:04.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.690+0000 7f3a82ffd700 1 -- 172.21.15.97:0/1779541849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c018440 con 0x7f3a8406c540 2023-11-24T06:09:04.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f4744a2a700 1 -- 172.21.15.97:0/127777591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47400c4c40 msgr2=0x7f47400c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f4744a2a700 1 --2- 172.21.15.97:0/127777591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47400c4c40 0x7f47400c50a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4738009e50 tx=0x7f473800b330 comp rx=0 tx=0).stop 2023-11-24T06:09:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f49e77fe700 1 --2- 172.21.15.97:0/2339826460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e815bd40 0x7f49e80cf9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49e80b30a0 tx=0x7f49d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8752251826bc5749 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1779541849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a8406c540 msgr2=0x7f3a8406da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1779541849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a8406c540 0x7f3a8406da30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3a8406d470 tx=0x7f3a7c009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:04.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f4744a2a700 1 -- 172.21.15.97:0/127777591 shutdown_connections 2023-11-24T06:09:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f4744a2a700 1 --2- 172.21.15.97:0/127777591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f47400c4c40 0x7f47400c50a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f4744a2a700 1 --2- 172.21.15.97:0/127777591 >> v2:172.21.15.97:3300/0 conn(0x7f474006b0d0 0x7f47401635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f4744a2a700 1 -- 172.21.15.97:0/127777591 >> 172.21.15.97:0/127777591 conn(0x7f4740073130 msgr2=0x7f474009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1779541849 shutdown_connections 2023-11-24T06:09:04.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1779541849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a8406c540 0x7f3a8406da30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1779541849 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a840b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1779541849 >> v2:172.21.15.183:3300/0 conn(0x7f3a840adf00 0x7f3a8406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.691+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1779541849 >> 172.21.15.97:0/1779541849 conn(0x7f3a84071b90 msgr2=0x7f3a84071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.692+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1779541849 shutdown_connections 2023-11-24T06:09:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.692+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1779541849 wait complete. 2023-11-24T06:09:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.692+0000 7f4744a2a700 1 -- 172.21.15.97:0/127777591 shutdown_connections 2023-11-24T06:09:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.692+0000 7f49e67fc700 1 -- 172.21.15.97:0/2339826460 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8004230 con 0x7f49e815bd40 2023-11-24T06:09:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.692+0000 7f49e67fc700 1 -- 172.21.15.97:0/2339826460 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d8004390 con 0x7f49e815bd40 2023-11-24T06:09:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.692+0000 7f49e67fc700 1 -- 172.21.15.97:0/2339826460 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80036a0 con 0x7f49e815bd40 2023-11-24T06:09:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.692+0000 7f4744a2a700 1 -- 172.21.15.97:0/127777591 wait complete. 2023-11-24T06:09:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f49ee090700 1 -- 172.21.15.97:0/2339826460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e815bd40 msgr2=0x7f49e80cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f49ee090700 1 --2- 172.21.15.97:0/2339826460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e815bd40 0x7f49e80cf9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49e80b30a0 tx=0x7f49d8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f49ee090700 1 -- 172.21.15.97:0/2339826460 shutdown_connections 2023-11-24T06:09:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f49ee090700 1 --2- 172.21.15.97:0/2339826460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e815bd40 0x7f49e80cf9c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f49ee090700 1 --2- 172.21.15.97:0/2339826460 >> v2:172.21.15.183:3300/0 conn(0x7f49e80afb20 0x7f49e815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f49ee090700 1 -- 172.21.15.97:0/2339826460 >> 172.21.15.97:0/2339826460 conn(0x7f49e80743b0 msgr2=0x7f49e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f6166b10700 1 Processor -- start 2023-11-24T06:09:04.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f6166b10700 1 -- start start 2023-11-24T06:09:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f3a8a04f700 1 Processor -- start 2023-11-24T06:09:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.694+0000 7f3a8a04f700 1 -- start start 2023-11-24T06:09:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f6166b10700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600d05c0 0x7f61600b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f6166b10700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616006dc20 con 0x7f61600b5cc0 2023-11-24T06:09:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f6166b10700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61600d09a0 con 0x7f61600746a0 2023-11-24T06:09:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.694+0000 7f4744a2a700 1 Processor -- start 2023-11-24T06:09:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.694+0000 7f4744a2a700 1 -- start start 2023-11-24T06:09:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f6166b10700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6160166bf0 con 0x7f61600d05c0 2023-11-24T06:09:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f3a8a04f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a8406c540 0x7f3a8415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f3a8a04f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a840adf00 0x7f3a840cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f3a8a04f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a841fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f61650ad700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f61600746a0 0x7f616006b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:47062/0 (socket says 172.21.15.97:47062) 2023-11-24T06:09:04.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f61650ad700 1 -- 172.21.15.97:0/2535142213 learned_addr learned my addr 172.21.15.97:0/2535142213 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f4744a2a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474006b0d0 0x7f47400c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f4744a2a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47400c4c40 0x7f47400caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f4744a2a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4740160030 0x7f47400caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f4744a2a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474015f930 con 0x7f4740160030 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f4744a2a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47400c9330 con 0x7f47400c4c40 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f4744a2a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47400cbe70 con 0x7f474006b0d0 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f473e59c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474006b0d0 0x7f47400c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f473e59c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474006b0d0 0x7f47400c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60564/0 (socket says 172.21.15.97:60564) 2023-11-24T06:09:04.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f61648ac700 1 --2- 172.21.15.97:0/2535142213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600d05c0 0x7f61600b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f473e59c700 1 -- 172.21.15.97:0/1036762899 learned_addr learned my addr 172.21.15.97:0/1036762899 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f3a8a04f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840cdb40 con 0x7f3a840adf00 2023-11-24T06:09:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f3a8a04f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840cdda0 con 0x7f3a8406c540 2023-11-24T06:09:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f3a8a04f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a840ce080 con 0x7f3a841446a0 2023-11-24T06:09:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f3a83fff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a841fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a83fff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a841fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60566/0 (socket says 172.21.15.97:60566) 2023-11-24T06:09:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a83fff700 1 -- 172.21.15.97:0/1421590204 learned_addr learned my addr 172.21.15.97:0/1421590204 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a837fe700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a8406c540 0x7f3a8415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f61648ac700 1 -- 172.21.15.97:0/2535142213 >> v1:172.21.15.183:6789/0 conn(0x7f61600746a0 legacy=0x7f616006b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f61648ac700 1 -- 172.21.15.97:0/2535142213 >> v1:172.21.15.97:6789/0 conn(0x7f61600b5cc0 legacy=0x7f6160073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f61648ac700 1 -- 172.21.15.97:0/2535142213 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160165f70 con 0x7f61600d05c0 2023-11-24T06:09:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a7affd700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a840adf00 0x7f3a840cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f90571a5700 1 Processor -- start 2023-11-24T06:09:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.693+0000 7f90571a5700 1 -- start start 2023-11-24T06:09:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f90571a5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f905016ae60 0x7f905006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f90571a5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9050072bb0 0x7f90500b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f90571a5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905008bdb0 con 0x7f90500b4090 2023-11-24T06:09:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f90571a5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90500b4540 con 0x7f9050072bb0 2023-11-24T06:09:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f90571a5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905006ce30 con 0x7f905016ae60 2023-11-24T06:09:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f9054f41700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f905016ae60 0x7f905006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f9054f41700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f905016ae60 0x7f905006efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39438/0 (socket says 172.21.15.97:39438) 2023-11-24T06:09:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f9054f41700 1 -- 172.21.15.97:0/940267491 learned_addr learned my addr 172.21.15.97:0/940267491 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.695+0000 7f904ffff700 1 --2- 172.21.15.97:0/940267491 >> v2:172.21.15.97:3300/0 conn(0x7f9050072bb0 0x7f90500b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f473ed9d700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4740160030 0x7f47400caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a7affd700 1 -- 172.21.15.97:0/1421590204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 msgr2=0x7f3a841fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a7affd700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a841fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a7affd700 1 -- 172.21.15.97:0/1421590204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a8406c540 msgr2=0x7f3a8415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a7affd700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a8406c540 0x7f3a8415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a7affd700 1 -- 172.21.15.97:0/1421590204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0094a0 con 0x7f3a840adf00 2023-11-24T06:09:04.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f61648ac700 1 --2- 172.21.15.97:0/2535142213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600d05c0 0x7f61600b3f10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f614c009870 tx=0x7f614c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac47d3088dfd4e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f473dd9b700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47400c4c40 0x7f47400caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f473ed9d700 1 -- 172.21.15.97:0/1036762899 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474006b0d0 msgr2=0x7f47400c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f473ed9d700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474006b0d0 0x7f47400c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f473ed9d700 1 -- 172.21.15.97:0/1036762899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47400c4c40 msgr2=0x7f47400caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f473ed9d700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47400c4c40 0x7f47400caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f473ed9d700 1 -- 172.21.15.97:0/1036762899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738009b30 con 0x7f4740160030 2023-11-24T06:09:04.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f9054f41700 1 -- 172.21.15.97:0/940267491 >> v2:172.21.15.97:3300/0 conn(0x7f9050072bb0 msgr2=0x7f90500b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f9054f41700 1 --2- 172.21.15.97:0/940267491 >> v2:172.21.15.97:3300/0 conn(0x7f9050072bb0 0x7f90500b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f9054f41700 1 -- 172.21.15.97:0/940267491 >> v1:172.21.15.97:6789/0 conn(0x7f90500b4090 legacy=0x7f905016e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f9054f41700 1 -- 172.21.15.97:0/940267491 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905006d110 con 0x7f905016ae60 2023-11-24T06:09:04.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f3a7affd700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a840adf00 0x7f3a840cd600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3a74007fd0 tx=0x7f3a7400d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f9054f41700 1 --2- 172.21.15.97:0/940267491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905016ae60 0x7f905006efc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90500b2780 tx=0x7f9040009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcdbf543261980a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f473ed9d700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4740160030 0x7f47400caf90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4738014000 tx=0x7f4738004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f3a80ff9700 1 -- 172.21.15.97:0/1421590204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400b4e0 con 0x7f3a840adf00 2023-11-24T06:09:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f904f7fe700 1 -- 172.21.15.97:0/940267491 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9040012b80 con 0x7f905016ae60 2023-11-24T06:09:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f904f7fe700 1 -- 172.21.15.97:0/940267491 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9040012ce0 con 0x7f905016ae60 2023-11-24T06:09:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f3a80ff9700 1 -- 172.21.15.97:0/1421590204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7400d9a0 con 0x7f3a840adf00 2023-11-24T06:09:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f3a80ff9700 1 -- 172.21.15.97:0/1421590204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400dd30 con 0x7f3a840adf00 2023-11-24T06:09:04.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f904f7fe700 1 -- 172.21.15.97:0/940267491 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904001b620 con 0x7f905016ae60 2023-11-24T06:09:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a840ce360 con 0x7f3a840adf00 2023-11-24T06:09:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.697+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a840ce920 con 0x7f3a840adf00 2023-11-24T06:09:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f90571a5700 1 -- 172.21.15.97:0/940267491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905016ae60 msgr2=0x7f905006efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f90571a5700 1 --2- 172.21.15.97:0/940267491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905016ae60 0x7f905006efc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90500b2780 tx=0x7f9040009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f49ee090700 1 -- 172.21.15.97:0/2339826460 shutdown_connections 2023-11-24T06:09:04.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f472effd700 1 -- 172.21.15.97:0/1036762899 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738004440 con 0x7f4740160030 2023-11-24T06:09:04.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f472effd700 1 -- 172.21.15.97:0/1036762899 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47380045a0 con 0x7f4740160030 2023-11-24T06:09:04.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47400cb8b0 con 0x7f4740160030 2023-11-24T06:09:04.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f472effd700 1 -- 172.21.15.97:0/1036762899 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4738004950 con 0x7f4740160030 2023-11-24T06:09:04.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a840753f0 con 0x7f3a840adf00 2023-11-24T06:09:04.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.698+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47400c6860 con 0x7f4740160030 2023-11-24T06:09:04.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.696+0000 7f615f7fe700 1 -- 172.21.15.97:0/2535142213 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c004230 con 0x7f61600d05c0 2023-11-24T06:09:04.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.699+0000 7f615f7fe700 1 -- 172.21.15.97:0/2535142213 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c004390 con 0x7f61600d05c0 2023-11-24T06:09:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.699+0000 7f615f7fe700 1 -- 172.21.15.97:0/2535142213 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c0036a0 con 0x7f61600d05c0 2023-11-24T06:09:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.699+0000 7f3a80ff9700 1 -- 172.21.15.97:0/1421590204 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a74009050 con 0x7f3a840adf00 2023-11-24T06:09:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.699+0000 7f3a80ff9700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3a6406aed0 0x7f3a6406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f3a80ff9700 1 -- 172.21.15.97:0/1421590204 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a740890d0 con 0x7f3a840adf00 2023-11-24T06:09:04.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f6166b10700 1 -- 172.21.15.97:0/2535142213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600d05c0 msgr2=0x7f61600b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f6166b10700 1 --2- 172.21.15.97:0/2535142213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600d05c0 0x7f61600b3f10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f614c009870 tx=0x7f614c009b80 comp rx=0 tx=0).stop 2023-11-24T06:09:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f90571a5700 1 -- 172.21.15.97:0/940267491 shutdown_connections 2023-11-24T06:09:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f90571a5700 1 --2- 172.21.15.97:0/940267491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905016ae60 0x7f905006efc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f90571a5700 1 --2- 172.21.15.97:0/940267491 >> v2:172.21.15.97:3300/0 conn(0x7f9050072bb0 0x7f90500b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f90571a5700 1 -- 172.21.15.97:0/940267491 >> 172.21.15.97:0/940267491 conn(0x7f90500cfa30 msgr2=0x7f90500cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f90571a5700 1 -- 172.21.15.97:0/940267491 shutdown_connections 2023-11-24T06:09:04.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f6166b10700 1 -- 172.21.15.97:0/2535142213 shutdown_connections 2023-11-24T06:09:04.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f6166b10700 1 --2- 172.21.15.97:0/2535142213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600d05c0 0x7f61600b3f10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f6166b10700 1 -- 172.21.15.97:0/2535142213 >> 172.21.15.97:0/2535142213 conn(0x7f61600b3130 msgr2=0x7f61600d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.699+0000 7f49ee090700 1 -- 172.21.15.97:0/2339826460 wait complete. 2023-11-24T06:09:04.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.701+0000 7f49ee090700 1 Processor -- start 2023-11-24T06:09:04.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f6166b10700 1 -- 172.21.15.97:0/2535142213 shutdown_connections 2023-11-24T06:09:04.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f3a837fe700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3a6406aed0 0x7f3a6406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f90571a5700 1 -- 172.21.15.97:0/940267491 wait complete. 2023-11-24T06:09:04.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f472effd700 1 -- 172.21.15.97:0/1036762899 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4738019070 con 0x7f4740160030 2023-11-24T06:09:04.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.701+0000 7f472effd700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f472006aed0 0x7f472006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.701+0000 7f473e59c700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f472006aed0 0x7f472006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.701+0000 7f473e59c700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f472006aed0 0x7f472006d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4740152ec0 tx=0x7f4730008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f90571a5700 1 Processor -- start 2023-11-24T06:09:04.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f90571a5700 1 -- start start 2023-11-24T06:09:04.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.701+0000 7f17c6fc5700 1 Processor -- start 2023-11-24T06:09:04.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.701+0000 7f17c6fc5700 1 -- start start 2023-11-24T06:09:04.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f2654a84700 1 Processor -- start 2023-11-24T06:09:04.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f2654a84700 1 -- start start 2023-11-24T06:09:04.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f2654a84700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f265006d4d0 0x7f265016cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f2654a84700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265006ced0 con 0x7f2650164a90 2023-11-24T06:09:04.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f2654a84700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265006da70 con 0x7f26500b3b50 2023-11-24T06:09:04.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f2654a84700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26501634e0 con 0x7f265006d4d0 2023-11-24T06:09:04.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.701+0000 7f3a837fe700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3a6406aed0 0x7f3a6406d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00b9b0 tx=0x7f3a6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.700+0000 7f6166b10700 1 -- 172.21.15.97:0/2535142213 wait complete. 2023-11-24T06:09:04.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f6166b10700 1 Processor -- start 2023-11-24T06:09:04.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f6166b10700 1 -- start start 2023-11-24T06:09:04.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f17c6fc5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c00b3b50 0x7f17c00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f17c6fc5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17c006d4d0 0x7f17c006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f17c6fc5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c006ced0 con 0x7f17c0163a80 2023-11-24T06:09:04.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f17c6fc5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c006da70 con 0x7f17c006d4d0 2023-11-24T06:09:04.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.702+0000 7f17c6fc5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c00b4470 con 0x7f17c00b3b50 2023-11-24T06:09:04.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f90571a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90500b4090 0x7f90500a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f90571a5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9050169d80 0x7f90500aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f90571a5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905008a530 0x7f905008a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f90571a5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905008aef0 con 0x7f90500b4090 2023-11-24T06:09:04.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f90571a5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90500aa970 con 0x7f905008a530 2023-11-24T06:09:04.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f90571a5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90500a9d60 con 0x7f9050169d80 2023-11-24T06:09:04.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f6166b10700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61600746a0 0x7f616014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f6166b10700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600b5cc0 0x7f616015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f6166b10700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61600d05c0 0x7f6160149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f6166b10700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616015d280 con 0x7f61600746a0 2023-11-24T06:09:04.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f6166b10700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616014bdf0 con 0x7f61600d05c0 2023-11-24T06:09:04.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f6166b10700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616014b420 con 0x7f61600b5cc0 2023-11-24T06:09:04.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f9055742700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905008a530 0x7f905008a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f9055742700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905008a530 0x7f905008a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39478/0 (socket says 172.21.15.97:39478) 2023-11-24T06:09:04.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f9055742700 1 -- 172.21.15.97:0/1569735783 learned_addr learned my addr 172.21.15.97:0/1569735783 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f904ffff700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9050169d80 0x7f90500aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f615ffff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600b5cc0 0x7f616015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f615ffff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600b5cc0 0x7f616015b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60592/0 (socket says 172.21.15.97:60592) 2023-11-24T06:09:04.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f615ffff700 1 -- 172.21.15.97:0/3000632919 learned_addr learned my addr 172.21.15.97:0/3000632919 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.703+0000 7f61650ad700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61600d05c0 0x7f6160149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904ffff700 1 -- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905008a530 msgr2=0x7f905008a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904ffff700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905008a530 0x7f905008a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904ffff700 1 -- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90500b4090 msgr2=0x7f90500a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904ffff700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90500b4090 0x7f90500a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904ffff700 1 -- 172.21.15.97:0/1569735783 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040009550 con 0x7f9050169d80 2023-11-24T06:09:04.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f9055742700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905008a530 0x7f905008a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:04.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615ffff700 1 -- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61600d05c0 msgr2=0x7f6160149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615ffff700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61600d05c0 0x7f6160149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615ffff700 1 -- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61600746a0 msgr2=0x7f616014b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904ffff700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9050169d80 0x7f90500aaef0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f904400a700 tx=0x7f904400aa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615ffff700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61600746a0 0x7f616014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615ffff700 1 -- 172.21.15.97:0/3000632919 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c0094a0 con 0x7f61600b5cc0 2023-11-24T06:09:04.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f61650ad700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61600d05c0 0x7f6160149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:04.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f264e59c700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f26500b3b50 0x7f26500b3f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:47080/0 (socket says 172.21.15.97:47080) 2023-11-24T06:09:04.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f264e59c700 1 -- 172.21.15.97:0/517748635 learned_addr learned my addr 172.21.15.97:0/517748635 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904d7fa700 1 -- 172.21.15.97:0/1569735783 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044018070 con 0x7f9050169d80 2023-11-24T06:09:04.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615ffff700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600b5cc0 0x7f616015b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f615400c780 tx=0x7f615400ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f17c4d61700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c00b3b50 0x7f17c00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f17c4d61700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c00b3b50 0x7f17c00b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60580/0 (socket says 172.21.15.97:60580) 2023-11-24T06:09:04.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f17c4d61700 1 -- 172.21.15.97:0/1529323051 learned_addr learned my addr 172.21.15.97:0/1529323051 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615d7fa700 1 -- 172.21.15.97:0/3000632919 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61540054e0 con 0x7f61600b5cc0 2023-11-24T06:09:04.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615d7fa700 1 -- 172.21.15.97:0/3000632919 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6154003680 con 0x7f61600b5cc0 2023-11-24T06:09:04.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f615d7fa700 1 -- 172.21.15.97:0/3000632919 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154003a30 con 0x7f61600b5cc0 2023-11-24T06:09:04.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616014a810 con 0x7f61600b5cc0 2023-11-24T06:09:04.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948317381 0 0) 0x7f265006da70 con 0x7f26500b3b50 2023-11-24T06:09:04.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26380035b0 con 0x7f26500b3b50 2023-11-24T06:09:04.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3581862040 0 0) 0x7f26380035b0 con 0x7f26500b3b50 2023-11-24T06:09:04.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 >> v2:172.21.15.97:3300/0 conn(0x7f265006d4d0 msgr2=0x7f265016cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 --2- 172.21.15.97:0/517748635 >> v2:172.21.15.97:3300/0 conn(0x7f265006d4d0 0x7f265016cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 >> v1:172.21.15.97:6789/0 conn(0x7f2650164a90 legacy=0x7f265006eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 --> v1:172.21.15.183:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26501637c0 con 0x7f26500b3b50 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 <== mon.1 v1:172.21.15.183:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f2644001670 con 0x7f26500b3b50 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 >> v1:172.21.15.183:6789/0 conn(0x7f26500b3b50 legacy=0x7f26500b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638003a00 0x7f2638003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2638004850 0x7f2638004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2638005410 0x7f263800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26501634e0 con 0x7f2638003a00 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263800c2b0 con 0x7f2638005410 2023-11-24T06:09:04.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263800c470 con 0x7f2638004850 2023-11-24T06:09:04.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f17c4d61700 1 -- 172.21.15.97:0/1529323051 >> v2:172.21.15.97:3300/0 conn(0x7f17c006d4d0 msgr2=0x7f17c006eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f17c4d61700 1 --2- 172.21.15.97:0/1529323051 >> v2:172.21.15.97:3300/0 conn(0x7f17c006d4d0 0x7f17c006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f17c4d61700 1 -- 172.21.15.97:0/1529323051 >> v1:172.21.15.183:6789/0 conn(0x7f17c0163a80 legacy=0x7f17c0163f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:09:04.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f17c4d61700 1 -- 172.21.15.97:0/1529323051 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c01634e0 con 0x7f17c00b3b50 2023-11-24T06:09:04.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264ed9d700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2638005410 0x7f263800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f264dd9b700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2638004850 0x7f2638004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f49ee090700 1 -- start start 2023-11-24T06:09:04.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904d7fa700 1 -- 172.21.15.97:0/1569735783 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9044010040 con 0x7f9050169d80 2023-11-24T06:09:04.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f904d7fa700 1 -- 172.21.15.97:0/1569735783 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400e620 con 0x7f9050169d80 2023-11-24T06:09:04.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.705+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616014add0 con 0x7f61600b5cc0 2023-11-24T06:09:04.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90500aa040 con 0x7f9050169d80 2023-11-24T06:09:04.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.704+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90500aa600 con 0x7f9050169d80 2023-11-24T06:09:04.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f904d7fa700 1 -- 172.21.15.97:0/1569735783 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f904400e780 con 0x7f9050169d80 2023-11-24T06:09:04.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90300052e0 con 0x7f9050169d80 2023-11-24T06:09:04.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f904d7fa700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f903c06aed0 0x7f903c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f904d7fa700 1 -- 172.21.15.97:0/1569735783 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9044089120 con 0x7f9050169d80 2023-11-24T06:09:04.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f9054f41700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f903c06aed0 0x7f903c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49ee090700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e80afb20 0x7f49e81f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49ee090700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e815bd40 0x7f49e80b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49ee090700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e815c660 0x7f49e80b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49ee090700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e8155a90 con 0x7f49e80afb20 2023-11-24T06:09:04.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49ee090700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e8155d70 con 0x7f49e815c660 2023-11-24T06:09:04.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49ee090700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e8156050 con 0x7f49e815bd40 2023-11-24T06:09:04.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49e6ffd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e815bd40 0x7f49e80b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49e6ffd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e815bd40 0x7f49e80b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60606/0 (socket says 172.21.15.97:60606) 2023-11-24T06:09:04.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f49e6ffd700 1 -- 172.21.15.97:0/3554369106 learned_addr learned my addr 172.21.15.97:0/3554369106 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.707+0000 7f9054f41700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f903c06aed0 0x7f903c06d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9040000c00 tx=0x7f9040014000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264ed9d700 1 -- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2638004850 msgr2=0x7f2638004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264ed9d700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2638004850 0x7f2638004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264ed9d700 1 -- 172.21.15.97:0/517748635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638003a00 msgr2=0x7f2638003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264ed9d700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638003a00 0x7f2638003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264ed9d700 1 -- 172.21.15.97:0/517748635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26501637c0 con 0x7f2638005410 2023-11-24T06:09:04.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264ed9d700 1 -- 172.21.15.97:0/517748635 reap_dead start 2023-11-24T06:09:04.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264ed9d700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2638005410 0x7f263800ba10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2640009910 tx=0x7f2638005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90f288ce0222ab4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2640004220 con 0x7f2638005410 2023-11-24T06:09:04.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26400043c0 con 0x7f2638005410 2023-11-24T06:09:04.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.706+0000 7f264d59a700 1 -- 172.21.15.97:0/517748635 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400038c0 con 0x7f2638005410 2023-11-24T06:09:04.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4740067f60 con 0x7f4740160030 2023-11-24T06:09:04.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f472effd700 1 -- 172.21.15.97:0/1036762899 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4738058f00 con 0x7f4740160030 2023-11-24T06:09:04.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f49e77fe700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e80afb20 0x7f49e81f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f2654a84700 1 -- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2638005410 msgr2=0x7f263800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f2654a84700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2638005410 0x7f263800ba10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2640009910 tx=0x7f2638005880 comp rx=0 tx=0).stop 2023-11-24T06:09:04.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f2654a84700 1 -- 172.21.15.97:0/517748635 shutdown_connections 2023-11-24T06:09:04.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f2654a84700 1 --2- 172.21.15.97:0/517748635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2638005410 0x7f263800ba10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f2654a84700 1 -- 172.21.15.97:0/517748635 >> 172.21.15.97:0/517748635 conn(0x7f26500cfa30 msgr2=0x7f26500cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f49e6ffd700 1 -- 172.21.15.97:0/3554369106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e815c660 msgr2=0x7f49e80b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f49e6ffd700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e815c660 0x7f49e80b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f49e6ffd700 1 -- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e80afb20 msgr2=0x7f49e81f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f49e6ffd700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e80afb20 0x7f49e81f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f49e6ffd700 1 -- 172.21.15.97:0/3554369106 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d8009550 con 0x7f49e815bd40 2023-11-24T06:09:04.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f2654a84700 1 -- 172.21.15.97:0/517748635 shutdown_connections 2023-11-24T06:09:04.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.708+0000 7f2654a84700 1 -- 172.21.15.97:0/517748635 wait complete. 2023-11-24T06:09:04.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.709+0000 7f8cf625b700 1 Processor -- start 2023-11-24T06:09:04.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.709+0000 7f17c4d61700 1 --2- 172.21.15.97:0/1529323051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c00b3b50 0x7f17c00b3f30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f17b40098f0 tx=0x7f17b4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a44196070d07e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.709+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6160067f60 con 0x7f61600b5cc0 2023-11-24T06:09:04.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.709+0000 7f49e6ffd700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e815bd40 0x7f49e80b4170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00b500 tx=0x7f49dc00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.709+0000 7f615d7fa700 1 -- 172.21.15.97:0/3000632919 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6154003b90 con 0x7f61600b5cc0 2023-11-24T06:09:04.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.709+0000 7f3a80ff9700 1 -- 172.21.15.97:0/1421590204 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a74054230 con 0x7f3a840adf00 2023-11-24T06:09:04.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.709+0000 7f8cf625b700 1 -- start start 2023-11-24T06:09:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.709+0000 7f615d7fa700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f614406af00 0x7f614406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f615d7fa700 1 -- 172.21.15.97:0/3000632919 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61540886c0 con 0x7f61600b5cc0 2023-11-24T06:09:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f61648ac700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f614406af00 0x7f614406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8cf625b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00d1570 0x7f8cf00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8cf625b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8cf00b2fc0 0x7f8cf00d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8cf625b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf00d2d10 con 0x7f8cf015dfc0 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8cf625b700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf00cefd0 con 0x7f8cf00b2fc0 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8cf625b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf00cf2b0 con 0x7f8cf00d1570 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8cef7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8cf015dfc0 0x7f8cf00d0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37798/0 (socket says 172.21.15.97:37798) 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8cef7fe700 1 -- 172.21.15.97:0/3745069786 learned_addr learned my addr 172.21.15.97:0/3745069786 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8ceffff700 1 --2- 172.21.15.97:0/3745069786 >> v2:172.21.15.183:3300/0 conn(0x7f8cf00b2fc0 0x7f8cf00d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8ce6ffd700 1 --2- 172.21.15.97:0/3745069786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00d1570 0x7f8cf00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f61648ac700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f614406af00 0x7f614406d3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f614c009bc0 tx=0x7f614c00f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8ceffff700 1 -- 172.21.15.97:0/3745069786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00d1570 msgr2=0x7f8cf00d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8ceffff700 1 --2- 172.21.15.97:0/3745069786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00d1570 0x7f8cf00d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8ceffff700 1 -- 172.21.15.97:0/3745069786 >> v1:172.21.15.97:6789/0 conn(0x7f8cf015dfc0 legacy=0x7f8cf00d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f8ceffff700 1 -- 172.21.15.97:0/3745069786 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf00d2fb0 con 0x7f8cf00b2fc0 2023-11-24T06:09:04.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.710+0000 7f2654a84700 1 Processor -- start 2023-11-24T06:09:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.711+0000 7f2654a84700 1 -- start start 2023-11-24T06:09:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.711+0000 7f8ceffff700 1 --2- 172.21.15.97:0/3745069786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00b2fc0 0x7f8cf00d3a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009d90 tx=0x7f8ce800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aae5f7731427e89 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.711+0000 7f8ceeffd700 1 -- 172.21.15.97:0/3745069786 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce800e070 con 0x7f8cf00b2fc0 2023-11-24T06:09:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.711+0000 7f8ceeffd700 1 -- 172.21.15.97:0/3745069786 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce800bb20 con 0x7f8cf00b2fc0 2023-11-24T06:09:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.711+0000 7f8ceeffd700 1 -- 172.21.15.97:0/3745069786 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce8012430 con 0x7f8cf00b2fc0 2023-11-24T06:09:04.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f2654a84700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638005410 0x7f2650156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f2654a84700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26501562d0 0x7f2650141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f2654a84700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2650141310 0x7f26500ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f2654a84700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26500cd530 con 0x7f2638005410 2023-11-24T06:09:04.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f2654a84700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26501425b0 con 0x7f2650141310 2023-11-24T06:09:04.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f8cf625b700 1 -- 172.21.15.97:0/3745069786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00b2fc0 msgr2=0x7f8cf00d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f8cf625b700 1 --2- 172.21.15.97:0/3745069786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00b2fc0 0x7f8cf00d3a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009d90 tx=0x7f8ce800b330 comp rx=0 tx=0).stop 2023-11-24T06:09:04.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f2654a84700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2650156750 con 0x7f26501562d0 2023-11-24T06:09:04.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638005410 0x7f2650156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638005410 0x7f2650156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45350/0 (socket says 172.21.15.97:45350) 2023-11-24T06:09:04.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 -- 172.21.15.97:0/3187145399 learned_addr learned my addr 172.21.15.97:0/3187145399 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f8cf625b700 1 -- 172.21.15.97:0/3745069786 shutdown_connections 2023-11-24T06:09:04.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f8cf625b700 1 --2- 172.21.15.97:0/3745069786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00b2fc0 0x7f8cf00d3a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f8cf625b700 1 --2- 172.21.15.97:0/3745069786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00d1570 0x7f8cf00d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f8cf625b700 1 -- 172.21.15.97:0/3745069786 >> 172.21.15.97:0/3745069786 conn(0x7f8cf006b410 msgr2=0x7f8cf006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264ed9d700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2650141310 0x7f26500ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264dd9b700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26501562d0 0x7f2650141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f8cf625b700 1 -- 172.21.15.97:0/3745069786 shutdown_connections 2023-11-24T06:09:04.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f8cf625b700 1 -- 172.21.15.97:0/3745069786 wait complete. 2023-11-24T06:09:04.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 -- 172.21.15.97:0/3187145399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26501562d0 msgr2=0x7f2650141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26501562d0 0x7f2650141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 -- 172.21.15.97:0/3187145399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2650141310 msgr2=0x7f26500ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2650141310 0x7f26500ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 -- 172.21.15.97:0/3187145399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500cd7d0 con 0x7f2638005410 2023-11-24T06:09:04.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.712+0000 7f264e59c700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638005410 0x7f2650156990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f264400c730 tx=0x7f264400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.713+0000 7f17bf7fe700 1 -- 172.21.15.97:0/1529323051 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4004230 con 0x7f17c00b3b50 2023-11-24T06:09:04.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.713+0000 7f904d7fa700 1 -- 172.21.15.97:0/1569735783 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9044054260 con 0x7f9050169d80 2023-11-24T06:09:04.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.714+0000 7f8cf625b700 1 Processor -- start 2023-11-24T06:09:04.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.714+0000 7f8cf625b700 1 -- start start 2023-11-24T06:09:04.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f263effd700 1 -- 172.21.15.97:0/3187145399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644004c80 con 0x7f2638005410 2023-11-24T06:09:04.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f263effd700 1 -- 172.21.15.97:0/3187145399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2644004de0 con 0x7f2638005410 2023-11-24T06:09:04.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f263effd700 1 -- 172.21.15.97:0/3187145399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644007310 con 0x7f2638005410 2023-11-24T06:09:04.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f472effd700 1 -- 172.21.15.97:0/1036762899 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f473808f060 con 0x7f4740160030 2023-11-24T06:09:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8cf625b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00b2fc0 0x7f8cf01f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8cf625b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00d1570 0x7f8cf0152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8cf625b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cf015dfc0 0x7f8cf0160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8cf625b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf015e940 con 0x7f8cf015dfc0 2023-11-24T06:09:04.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8cf625b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf015f720 con 0x7f8cf00d1570 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8cf625b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf01f4910 con 0x7f8cf00b2fc0 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8ce6ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00d1570 0x7f8cf0152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8ce6ffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00d1570 0x7f8cf0152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39516/0 (socket says 172.21.15.97:39516) 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8ce6ffd700 1 -- 172.21.15.97:0/439594342 learned_addr learned my addr 172.21.15.97:0/439594342 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8cef7fe700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00b2fc0 0x7f8cf01f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8ce6ffd700 1 -- 172.21.15.97:0/439594342 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00b2fc0 msgr2=0x7f8cf01f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8ce6ffd700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00b2fc0 0x7f8cf01f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8ce6ffd700 1 -- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cf015dfc0 msgr2=0x7f8cf0160250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.715+0000 7f8ce6ffd700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cf015dfc0 0x7f8cf0160250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f8ce6ffd700 1 -- 172.21.15.97:0/439594342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8009a70 con 0x7f8cf00d1570 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f8ce6ffd700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00d1570 0x7f8cf0152220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000e9b0 tx=0x7f8ce000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f8cecff9700 1 -- 172.21.15.97:0/439594342 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0009710 con 0x7f8cf00d1570 2023-11-24T06:09:04.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f8cecff9700 1 -- 172.21.15.97:0/439594342 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce000c850 con 0x7f8cf00d1570 2023-11-24T06:09:04.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf015eb30 con 0x7f8cf00d1570 2023-11-24T06:09:04.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f8cecff9700 1 -- 172.21.15.97:0/439594342 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000cbc0 con 0x7f8cf00d1570 2023-11-24T06:09:04.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf015f0d0 con 0x7f8cf00d1570 2023-11-24T06:09:04.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26500cdab0 con 0x7f2638005410 2023-11-24T06:09:04.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.717+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf015f8d0 con 0x7f8cf00d1570 2023-11-24T06:09:04.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.716+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26500ce070 con 0x7f2638005410 2023-11-24T06:09:04.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.717+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26500ce3e0 con 0x7f2638005410 2023-11-24T06:09:04.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.717+0000 7f49cffff700 1 -- 172.21.15.97:0/3554369106 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc011830 con 0x7f49e815bd40 2023-11-24T06:09:04.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e8156330 con 0x7f49e815bd40 2023-11-24T06:09:04.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f17bf7fe700 1 -- 172.21.15.97:0/1529323051 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b4004390 con 0x7f17c00b3b50 2023-11-24T06:09:04.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f17bf7fe700 1 -- 172.21.15.97:0/1529323051 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40036a0 con 0x7f17c00b3b50 2023-11-24T06:09:04.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f263effd700 1 -- 172.21.15.97:0/3187145399 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2644007470 con 0x7f2638005410 2023-11-24T06:09:04.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.719+0000 7f263effd700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f263006aed0 0x7f263006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.719+0000 7f263effd700 1 -- 172.21.15.97:0/3187145399 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2644089330 con 0x7f2638005410 2023-11-24T06:09:04.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f8cecff9700 1 -- 172.21.15.97:0/439594342 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ce000cd20 con 0x7f8cf00d1570 2023-11-24T06:09:04.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f8cecff9700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cd006aed0 0x7f8cd006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.719+0000 7f8cecff9700 1 -- 172.21.15.97:0/439594342 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ce00895a0 con 0x7f8cf00d1570 2023-11-24T06:09:04.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.719+0000 7f8cef7fe700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cd006aed0 0x7f8cd006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e8156b30 con 0x7f49e815bd40 2023-11-24T06:09:04.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f49cffff700 1 -- 172.21.15.97:0/3554369106 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc011990 con 0x7f49e815bd40 2023-11-24T06:09:04.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.718+0000 7f49cffff700 1 -- 172.21.15.97:0/3554369106 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc011d20 con 0x7f49e815bd40 2023-11-24T06:09:04.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.719+0000 7f8cef7fe700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cd006aed0 0x7f8cd006d390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8005910 tx=0x7f8cd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.720+0000 7f49cffff700 1 -- 172.21.15.97:0/3554369106 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49dc01a400 con 0x7f49e815bd40 2023-11-24T06:09:04.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.720+0000 7f49cffff700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f49d406aed0 0x7f49d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.720+0000 7f49e77fe700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f49d406aed0 0x7f49d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.720+0000 7f49cffff700 1 -- 172.21.15.97:0/3554369106 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49dc0895f0 con 0x7f49e815bd40 2023-11-24T06:09:04.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.720+0000 7f17c6fc5700 1 -- 172.21.15.97:0/1529323051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c00b3b50 msgr2=0x7f17c00b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.720+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/1529323051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c00b3b50 0x7f17c00b3f30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f17b40098f0 tx=0x7f17b4009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:04.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f49e77fe700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f49d406aed0 0x7f49d406d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009c10 tx=0x7f49d8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f264dd9b700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f263006aed0 0x7f263006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f17c6fc5700 1 -- 172.21.15.97:0/1529323051 shutdown_connections 2023-11-24T06:09:04.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/1529323051 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c00b3b50 0x7f17c00b3f30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/1529323051 >> v2:172.21.15.97:3300/0 conn(0x7f17c006d4d0 0x7f17c006eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f17c6fc5700 1 -- 172.21.15.97:0/1529323051 >> 172.21.15.97:0/1529323051 conn(0x7f17c00cfa30 msgr2=0x7f17c00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:04.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f264dd9b700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f263006aed0 0x7f263006d390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2650153150 tx=0x7f2648009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c80052e0 con 0x7f49e815bd40 2023-11-24T06:09:04.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.721+0000 7f615d7fa700 1 -- 172.21.15.97:0/3000632919 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6154053640 con 0x7f61600b5cc0 2023-11-24T06:09:04.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.724+0000 7f17c6fc5700 1 -- 172.21.15.97:0/1529323051 shutdown_connections 2023-11-24T06:09:04.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.724+0000 7f17c6fc5700 1 -- 172.21.15.97:0/1529323051 wait complete. 2023-11-24T06:09:04.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.725+0000 7f8cecff9700 1 -- 172.21.15.97:0/439594342 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8cf015f8d0 con 0x7f8cf00d1570 2023-11-24T06:09:04.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.725+0000 7f17c6fc5700 1 Processor -- start 2023-11-24T06:09:04.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.726+0000 7f263effd700 1 -- 172.21.15.97:0/3187145399 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26440542d0 con 0x7f2638005410 2023-11-24T06:09:04.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.726+0000 7f17c6fc5700 1 -- start start 2023-11-24T06:09:04.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c6fc5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c006d4d0 0x7f17c0155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c6fc5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c00b3b50 0x7f17c01fb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c6fc5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c0163a80 0x7f17c01fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c6fc5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c01562f0 con 0x7f17c006d4d0 2023-11-24T06:09:04.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c6fc5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0154d10 con 0x7f17c00b3b50 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c6fc5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c0154ff0 con 0x7f17c0163a80 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c4d61700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c006d4d0 0x7f17c0155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c4d61700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c006d4d0 0x7f17c0155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45366/0 (socket says 172.21.15.97:45366) 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17c4d61700 1 -- 172.21.15.97:0/766794263 learned_addr learned my addr 172.21.15.97:0/766794263 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.727+0000 7f17bffff700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c00b3b50 0x7f17c01fb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.728+0000 7f17c4d61700 1 -- 172.21.15.97:0/766794263 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c0163a80 msgr2=0x7f17c01fb700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.728+0000 7f17c4d61700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c0163a80 0x7f17c01fb700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.728+0000 7f17c4d61700 1 -- 172.21.15.97:0/766794263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c00b3b50 msgr2=0x7f17c01fb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.728+0000 7f17c4d61700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c00b3b50 0x7f17c01fb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.728+0000 7f17c4d61700 1 -- 172.21.15.97:0/766794263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b4009550 con 0x7f17c006d4d0 2023-11-24T06:09:04.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.728+0000 7f17c4d61700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c006d4d0 0x7f17c0155db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f17c006d420 tx=0x7f17b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.732+0000 7f49cffff700 1 -- 172.21.15.97:0/3554369106 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49dc054640 con 0x7f49e815bd40 2023-11-24T06:09:04.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.729+0000 7f17bd7fa700 1 -- 172.21.15.97:0/766794263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4003cc0 con 0x7f17c006d4d0 2023-11-24T06:09:04.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.729+0000 7f17bd7fa700 1 -- 172.21.15.97:0/766794263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b401f070 con 0x7f17c006d4d0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.729+0000 7f17bd7fa700 1 -- 172.21.15.97:0/766794263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b400b880 con 0x7f17c006d4d0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.729+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c01552d0 con 0x7f17c006d4d0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.729+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c0155890 con 0x7f17c006d4d0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.732+0000 7f17bd7fa700 1 -- 172.21.15.97:0/766794263 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17b4021030 con 0x7f17c006d4d0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.733+0000 7f17bd7fa700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f17ac06af20 0x7f17ac06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.733+0000 7f17bd7fa700 1 -- 172.21.15.97:0/766794263 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17b408ac40 con 0x7f17c006d4d0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.733+0000 7f17bffff700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f17ac06af20 0x7f17ac06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.734+0000 7f17bffff700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f17ac06af20 0x7f17ac06d3e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17b800a400 tx=0x7f17b800d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.743+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a00052e0 con 0x7f17c006d4d0 2023-11-24T06:09:04.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:04.753+0000 7f17bd7fa700 1 -- 172.21.15.97:0/766794263 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17b4055d60 con 0x7f17c006d4d0 2023-11-24T06:09:05.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.115+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3a84067f60 con 0x7f3a840adf00 2023-11-24T06:09:05.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.115+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9030005860 con 0x7f9050169d80 2023-11-24T06:09:05.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.116+0000 7f3a80ff9700 1 -- 172.21.15.97:0/1421590204 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a74057850 con 0x7f3a840adf00 2023-11-24T06:09:05.122 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2023-11-24T06:09:05.122 INFO:teuthology.orchestra.run.smithi097.stdout:55834574853 2023-11-24T06:09:05.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.116+0000 7f904d7fa700 1 -- 172.21.15.97:0/1569735783 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9044057880 con 0x7f9050169d80 2023-11-24T06:09:05.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3a6406aed0 msgr2=0x7f3a6406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3a6406aed0 0x7f3a6406d390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00b9b0 tx=0x7f3a6c008040 comp rx=0 tx=0).stop 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a840adf00 msgr2=0x7f3a840cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a840adf00 0x7f3a840cd600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3a74007fd0 tx=0x7f3a7400d390 comp rx=0 tx=0).stop 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 shutdown_connections 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3a6406aed0 0x7f3a6406d390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3a841446a0 0x7f3a841fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a8406c540 0x7f3a8415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 --2- 172.21.15.97:0/1421590204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a840adf00 0x7f3a840cd600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 >> 172.21.15.97:0/1421590204 conn(0x7f3a84071b90 msgr2=0x7f3a84071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:05.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 shutdown_connections 2023-11-24T06:09:05.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f3a8a04f700 1 -- 172.21.15.97:0/1421590204 wait complete. 2023-11-24T06:09:05.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f903c06aed0 msgr2=0x7f903c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f90571a5700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f903c06aed0 0x7f903c06d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9040000c00 tx=0x7f9040014000 comp rx=0 tx=0).stop 2023-11-24T06:09:05.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9050169d80 msgr2=0x7f90500aaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.120+0000 7f90571a5700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9050169d80 0x7f90500aaef0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f904400a700 tx=0x7f904400aa10 comp rx=0 tx=0).stop 2023-11-24T06:09:05.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.122+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 shutdown_connections 2023-11-24T06:09:05.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.122+0000 7f90571a5700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f905008a530 0x7f905008a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.122+0000 7f90571a5700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90500b4090 0x7f90500a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.122+0000 7f90571a5700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f903c06aed0 0x7f903c06d390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.122+0000 7f90571a5700 1 --2- 172.21.15.97:0/1569735783 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9050169d80 0x7f90500aaef0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.122+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 >> 172.21.15.97:0/1569735783 conn(0x7f90500cfa30 msgr2=0x7f9050070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:05.126 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-11-24T06:09:05.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.121+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8cf0067f60 con 0x7f8cf00d1570 2023-11-24T06:09:05.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.121+0000 7f8cecff9700 1 -- 172.21.15.97:0/439594342 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8ce00546e0 con 0x7f8cf00d1570 2023-11-24T06:09:05.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.123+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f61600b07c0 con 0x7f61600b5cc0 2023-11-24T06:09:05.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.124+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 shutdown_connections 2023-11-24T06:09:05.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.124+0000 7f90571a5700 1 -- 172.21.15.97:0/1569735783 wait complete. 2023-11-24T06:09:05.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.124+0000 7f615d7fa700 1 -- 172.21.15.97:0/3000632919 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6154056c60 con 0x7f61600b5cc0 2023-11-24T06:09:05.128 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2023-11-24T06:09:05.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.123+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4740061aa0 con 0x7f4740160030 2023-11-24T06:09:05.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.125+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f26500751e0 con 0x7f2638005410 2023-11-24T06:09:05.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cd006aed0 msgr2=0x7f8cd006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cd006aed0 0x7f8cd006d390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8005910 tx=0x7f8cd8009040 comp rx=0 tx=0).stop 2023-11-24T06:09:05.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00d1570 msgr2=0x7f8cf0152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00d1570 0x7f8cf0152220 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000e9b0 tx=0x7f8ce000ecc0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 shutdown_connections 2023-11-24T06:09:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cf015dfc0 0x7f8cf0160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cd006aed0 0x7f8cd006d390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cf00b2fc0 0x7f8cf01f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 --2- 172.21.15.97:0/439594342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cf00d1570 0x7f8cf0152220 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 >> 172.21.15.97:0/439594342 conn(0x7f8cf006b410 msgr2=0x7f8cf006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:05.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 shutdown_connections 2023-11-24T06:09:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.126+0000 7f8cf625b700 1 -- 172.21.15.97:0/439594342 wait complete. 2023-11-24T06:09:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.128+0000 7f472effd700 1 -- 172.21.15.97:0/1036762899 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4738017070 con 0x7f4740160030 2023-11-24T06:09:05.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.128+0000 7f263effd700 1 -- 172.21.15.97:0/3187145399 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f26440578f0 con 0x7f2638005410 2023-11-24T06:09:05.131 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2023-11-24T06:09:05.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.129+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f614406af00 msgr2=0x7f614406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.129+0000 7f6166b10700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f614406af00 0x7f614406d3c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f614c009bc0 tx=0x7f614c00f000 comp rx=0 tx=0).stop 2023-11-24T06:09:05.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.129+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600b5cc0 msgr2=0x7f616015b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.129+0000 7f6166b10700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600b5cc0 0x7f616015b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f615400c780 tx=0x7f615400ca90 comp rx=0 tx=0).stop 2023-11-24T06:09:05.136 INFO:teuthology.orchestra.run.smithi097.stdout:55834574854 2023-11-24T06:09:05.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.131+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 shutdown_connections 2023-11-24T06:09:05.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.131+0000 7f6166b10700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61600d05c0 0x7f6160149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.131+0000 7f6166b10700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61600746a0 0x7f616014b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.131+0000 7f6166b10700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f614406af00 0x7f614406d3c0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f614c009bc0 tx=0x7f614c00f000 comp rx=0 tx=0).stop 2023-11-24T06:09:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.131+0000 7f6166b10700 1 --2- 172.21.15.97:0/3000632919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f61600b5cc0 0x7f616015b310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.131+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 >> 172.21.15.97:0/3000632919 conn(0x7f61600b3130 msgr2=0x7f6160164390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.132+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f263006aed0 msgr2=0x7f263006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.132+0000 7f2654a84700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f263006aed0 0x7f263006d390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2650153150 tx=0x7f2648009040 comp rx=0 tx=0).stop 2023-11-24T06:09:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.132+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638005410 msgr2=0x7f2650156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.132+0000 7f2654a84700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638005410 0x7f2650156990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f264400c730 tx=0x7f264400ca40 comp rx=0 tx=0).stop 2023-11-24T06:09:05.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.132+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f472006aed0 msgr2=0x7f472006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.132+0000 7f4744a2a700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f472006aed0 0x7f472006d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4740152ec0 tx=0x7f4730008040 comp rx=0 tx=0).stop 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.132+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4740160030 msgr2=0x7f47400caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.132+0000 7f4744a2a700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4740160030 0x7f47400caf90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4738014000 tx=0x7f4738004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 shutdown_connections 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f2654a84700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2650141310 0x7f26500ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f2654a84700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2638005410 0x7f2650156990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f2654a84700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f263006aed0 0x7f263006d390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f2654a84700 1 --2- 172.21.15.97:0/3187145399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26501562d0 0x7f2650141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 >> 172.21.15.97:0/3187145399 conn(0x7f26500cfa30 msgr2=0x7f2650072710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:05.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 shutdown_connections 2023-11-24T06:09:05.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f4744a2a700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4740160030 0x7f47400caf90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f4744a2a700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f472006aed0 0x7f472006d390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f4744a2a700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f474006b0d0 0x7f47400c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f4744a2a700 1 --2- 172.21.15.97:0/1036762899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47400c4c40 0x7f47400caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.133+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 >> 172.21.15.97:0/1036762899 conn(0x7f4740073130 msgr2=0x7f47400b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:05.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.134+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 shutdown_connections 2023-11-24T06:09:05.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.134+0000 7f2654a84700 1 -- 172.21.15.97:0/3187145399 wait complete. 2023-11-24T06:09:05.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.135+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f49c8005590 con 0x7f49e815bd40 2023-11-24T06:09:05.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.136+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 shutdown_connections 2023-11-24T06:09:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.136+0000 7f49cffff700 1 -- 172.21.15.97:0/3554369106 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49dc015030 con 0x7f49e815bd40 2023-11-24T06:09:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.136+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 shutdown_connections 2023-11-24T06:09:05.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.136+0000 7f6166b10700 1 -- 172.21.15.97:0/3000632919 wait complete. 2023-11-24T06:09:05.141 INFO:teuthology.orchestra.run.smithi097.stdout:55834574853 2023-11-24T06:09:05.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.140+0000 7f4744a2a700 1 -- 172.21.15.97:0/1036762899 wait complete. 2023-11-24T06:09:05.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.6 2023-11-24T06:09:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f49d406aed0 msgr2=0x7f49d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f49d406aed0 0x7f49d406d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009c10 tx=0x7f49d8008030 comp rx=0 tx=0).stop 2023-11-24T06:09:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e815bd40 msgr2=0x7f49e80b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e815bd40 0x7f49e80b4170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00b500 tx=0x7f49dc00b810 comp rx=0 tx=0).stop 2023-11-24T06:09:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 shutdown_connections 2023-11-24T06:09:05.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49e815c660 0x7f49e80b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49e80afb20 0x7f49e81f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f49d406aed0 0x7f49d406d390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 --2- 172.21.15.97:0/3554369106 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f49e815bd40 0x7f49e80b4170 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 >> 172.21.15.97:0/3554369106 conn(0x7f49e80743b0 msgr2=0x7f49e8071340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 shutdown_connections 2023-11-24T06:09:05.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.148+0000 7f49ee090700 1 -- 172.21.15.97:0/3554369106 wait complete. 2023-11-24T06:09:05.154 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.0 2023-11-24T06:09:05.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.4 2023-11-24T06:09:05.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.153+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f17a0005590 con 0x7f17c006d4d0 2023-11-24T06:09:05.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.153+0000 7f17bd7fa700 1 -- 172.21.15.97:0/766794263 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17b4059380 con 0x7f17c006d4d0 2023-11-24T06:09:05.157 INFO:teuthology.orchestra.run.smithi097.stdout:55834574853 2023-11-24T06:09:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.155+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f17ac06af20 msgr2=0x7f17ac06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.155+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f17ac06af20 0x7f17ac06d3e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17b800a400 tx=0x7f17b800d040 comp rx=0 tx=0).stop 2023-11-24T06:09:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.155+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c006d4d0 msgr2=0x7f17c0155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.155+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c006d4d0 0x7f17c0155db0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f17c006d420 tx=0x7f17b4004750 comp rx=0 tx=0).stop 2023-11-24T06:09:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.156+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 shutdown_connections 2023-11-24T06:09:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.156+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f17ac06af20 0x7f17ac06d3e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.156+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17c0163a80 0x7f17c01fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.156+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c006d4d0 0x7f17c0155db0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f17c006d420 tx=0x7f17b4004750 comp rx=0 tx=0).stop 2023-11-24T06:09:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.156+0000 7f17c6fc5700 1 --2- 172.21.15.97:0/766794263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c00b3b50 0x7f17c01fb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:05.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.156+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 >> 172.21.15.97:0/766794263 conn(0x7f17c00cfa30 msgr2=0x7f17c0071040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:05.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.7 2023-11-24T06:09:05.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.159+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 shutdown_connections 2023-11-24T06:09:05.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:05.159+0000 7f17c6fc5700 1 -- 172.21.15.97:0/766794263 wait complete. 2023-11-24T06:09:05.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.5 2023-11-24T06:09:05.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.3 2023-11-24T06:09:05.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.1 2023-11-24T06:09:05.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.2 2023-11-24T06:09:06.145 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:09:06.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:09:06.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:09:06.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:09:06.164 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:09:06.166 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:09:06.170 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:09:06.175 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.354+0000 7f799a686700 1 Processor -- start 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.355+0000 7f799a686700 1 -- start start 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.356+0000 7f799a686700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f799415d6a0 0x7f79940b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.356+0000 7f799a686700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7994071be0 0x7f7994140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.356+0000 7f799a686700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79940720e0 con 0x7f79940b4090 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.356+0000 7f799a686700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7994072d60 con 0x7f799415d6a0 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.356+0000 7f799a686700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799406de80 con 0x7f7994071be0 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.357+0000 7f7993fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f799415d6a0 0x7f79940b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.357+0000 7f7993fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f799415d6a0 0x7f79940b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45374/0 (socket says 172.21.15.97:45374) 2023-11-24T06:09:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.357+0000 7f7993fff700 1 -- 172.21.15.97:0/3871503451 learned_addr learned my addr 172.21.15.97:0/3871503451 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.357+0000 7f7998c23700 1 --2- 172.21.15.97:0/3871503451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7994071be0 0x7f7994140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.358+0000 7f7993fff700 1 -- 172.21.15.97:0/3871503451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7994071be0 msgr2=0x7f7994140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.358+0000 7f7993fff700 1 --2- 172.21.15.97:0/3871503451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7994071be0 0x7f7994140fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.358+0000 7f7993fff700 1 -- 172.21.15.97:0/3871503451 >> v1:172.21.15.183:6789/0 conn(0x7f79940b4090 legacy=0x7f79940714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.358+0000 7f7993fff700 1 -- 172.21.15.97:0/3871503451 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79940723c0 con 0x7f799415d6a0 2023-11-24T06:09:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.358+0000 7f7993fff700 1 --2- 172.21.15.97:0/3871503451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799415d6a0 0x7f79940b3b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f798c00d3a0 tx=0x7f798c00d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e26342ec3c1905a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.359+0000 7f7992ffd700 1 -- 172.21.15.97:0/3871503451 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c010070 con 0x7f799415d6a0 2023-11-24T06:09:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.359+0000 7f7992ffd700 1 -- 172.21.15.97:0/3871503451 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c00dc10 con 0x7f799415d6a0 2023-11-24T06:09:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.359+0000 7f7992ffd700 1 -- 172.21.15.97:0/3871503451 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c007b30 con 0x7f799415d6a0 2023-11-24T06:09:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.360+0000 7f799a686700 1 -- 172.21.15.97:0/3871503451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799415d6a0 msgr2=0x7f79940b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.360+0000 7f799a686700 1 --2- 172.21.15.97:0/3871503451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799415d6a0 0x7f79940b3b50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f798c00d3a0 tx=0x7f798c00d6b0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.360+0000 7f799a686700 1 -- 172.21.15.97:0/3871503451 shutdown_connections 2023-11-24T06:09:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.360+0000 7f799a686700 1 --2- 172.21.15.97:0/3871503451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7994071be0 0x7f7994140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.360+0000 7f799a686700 1 --2- 172.21.15.97:0/3871503451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f799415d6a0 0x7f79940b3b50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.360+0000 7f799a686700 1 -- 172.21.15.97:0/3871503451 >> 172.21.15.97:0/3871503451 conn(0x7f79940cfa30 msgr2=0x7f79940cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.361+0000 7f799a686700 1 -- 172.21.15.97:0/3871503451 shutdown_connections 2023-11-24T06:09:06.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.361+0000 7f799a686700 1 -- 172.21.15.97:0/3871503451 wait complete. 2023-11-24T06:09:06.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.362+0000 7f799a686700 1 Processor -- start 2023-11-24T06:09:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.363+0000 7f799a686700 1 -- start start 2023-11-24T06:09:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.364+0000 7f799a686700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994071be0 0x7f79940aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.364+0000 7f799a686700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79940b4090 0x7f799414f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.365+0000 7f799a686700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f799415d6a0 0x7f799414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.365+0000 7f799a686700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7994150090 con 0x7f7994071be0 2023-11-24T06:09:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.365+0000 7f799a686700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7994150370 con 0x7f799415d6a0 2023-11-24T06:09:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.365+0000 7f799a686700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7994150650 con 0x7f79940b4090 2023-11-24T06:09:06.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.366+0000 7f7993fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994071be0 0x7f79940aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.366+0000 7f7993fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994071be0 0x7f79940aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45382/0 (socket says 172.21.15.97:45382) 2023-11-24T06:09:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.366+0000 7f7993fff700 1 -- 172.21.15.97:0/2921672626 learned_addr learned my addr 172.21.15.97:0/2921672626 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.366+0000 7f79937fe700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79940b4090 0x7f799414f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.366+0000 7f7998c23700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f799415d6a0 0x7f799414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.366+0000 7f7993fff700 1 -- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79940b4090 msgr2=0x7f799414f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.366+0000 7f7993fff700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79940b4090 0x7f799414f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f7993fff700 1 -- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f799415d6a0 msgr2=0x7f799414f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f7993fff700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f799415d6a0 0x7f799414f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f7993fff700 1 -- 172.21.15.97:0/2921672626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c00d050 con 0x7f7994071be0 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f7998c23700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f799415d6a0 0x7f799414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f7993fff700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994071be0 0x7f79940aff50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f798c018040 tx=0x7f798c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f7990ff9700 1 -- 172.21.15.97:0/2921672626 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c004510 con 0x7f7994071be0 2023-11-24T06:09:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f7990ff9700 1 -- 172.21.15.97:0/2921672626 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c004670 con 0x7f7994071be0 2023-11-24T06:09:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f7990ff9700 1 -- 172.21.15.97:0/2921672626 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c008030 con 0x7f7994071be0 2023-11-24T06:09:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7994150930 con 0x7f7994071be0 2023-11-24T06:09:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.367+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799414fda0 con 0x7f7994071be0 2023-11-24T06:09:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.372+0000 7f7990ff9700 1 -- 172.21.15.97:0/2921672626 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f798c008190 con 0x7f7994071be0 2023-11-24T06:09:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.372+0000 7f7990ff9700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f798006af20 0x7f798006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.372+0000 7f7990ff9700 1 -- 172.21.15.97:0/2921672626 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f798c08b120 con 0x7f7994071be0 2023-11-24T06:09:06.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.374+0000 7f79937fe700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f798006af20 0x7f798006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.374+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79740052e0 con 0x7f7994071be0 2023-11-24T06:09:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.383+0000 7f7990ff9700 1 -- 172.21.15.97:0/2921672626 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f798c056460 con 0x7f7994071be0 2023-11-24T06:09:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.384+0000 7f79937fe700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f798006af20 0x7f798006d3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f79941e4f20 tx=0x7f7984009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.403+0000 7f863c526700 1 Processor -- start 2023-11-24T06:09:06.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.404+0000 7f863c526700 1 -- start start 2023-11-24T06:09:06.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.403+0000 7fa919765700 1 Processor -- start 2023-11-24T06:09:06.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7f863c526700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8634072bb0 0x7f863406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7f863c526700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86340b4260 0x7f8634070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7f863c526700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863406ce30 con 0x7f863416abf0 2023-11-24T06:09:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7f863c526700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863406d110 con 0x7f86340b4260 2023-11-24T06:09:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7f863c526700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863406d3f0 con 0x7f8634072bb0 2023-11-24T06:09:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7f863a2c2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8634072bb0 0x7f863406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7f863a2c2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8634072bb0 0x7f863406efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39548/0 (socket says 172.21.15.97:39548) 2023-11-24T06:09:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7f863a2c2700 1 -- 172.21.15.97:0/1505987898 learned_addr learned my addr 172.21.15.97:0/1505987898 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.406+0000 7f863aac3700 1 --2- 172.21.15.97:0/1505987898 >> v2:172.21.15.97:3300/0 conn(0x7f86340b4260 0x7f8634070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.406+0000 7fa919765700 1 -- start start 2023-11-24T06:09:06.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fa919765700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa9140b49b0 0x7fa91406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fa919765700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa91415de50 0x7fa91415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fa919765700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9140cc5c0 con 0x7fa914151500 2023-11-24T06:09:06.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fa919765700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa914071d70 con 0x7fa91415de50 2023-11-24T06:09:06.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fa919765700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91415e820 con 0x7fa9140b49b0 2023-11-24T06:09:06.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7f863a2c2700 1 -- 172.21.15.97:0/1505987898 >> v2:172.21.15.97:3300/0 conn(0x7f86340b4260 msgr2=0x7f8634070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7f863a2c2700 1 --2- 172.21.15.97:0/1505987898 >> v2:172.21.15.97:3300/0 conn(0x7f86340b4260 0x7f8634070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7f863a2c2700 1 -- 172.21.15.97:0/1505987898 >> v1:172.21.15.97:6789/0 conn(0x7f863416abf0 legacy=0x7f86340b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7f863a2c2700 1 -- 172.21.15.97:0/1505987898 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863406d6d0 con 0x7f8634072bb0 2023-11-24T06:09:06.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7f863a2c2700 1 --2- 172.21.15.97:0/1505987898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634072bb0 0x7f863406efc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f863406c600 tx=0x7f862c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4818d69d8d39985 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fa912ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa914151500 0x7fa9140cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37852/0 (socket says 172.21.15.97:37852) 2023-11-24T06:09:06.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fa912ffd700 1 -- 172.21.15.97:0/3919680600 learned_addr learned my addr 172.21.15.97:0/3919680600 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:06.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fa9137fe700 1 --2- 172.21.15.97:0/3919680600 >> v2:172.21.15.97:3300/0 conn(0x7fa91415de50 0x7fa91415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fa9127fc700 1 --2- 172.21.15.97:0/3919680600 >> v2:172.21.15.183:3300/0 conn(0x7fa9140b49b0 0x7fa91406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7fece4db7700 1 Processor -- start 2023-11-24T06:09:06.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.405+0000 7fece4db7700 1 -- start start 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.406+0000 7fece4db7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fece016ae60 0x7fece006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fece4db7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fece00b4260 0x7fece0070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fece4db7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece008bdb0 con 0x7fece0072bb0 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fece4db7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece006ce30 con 0x7fece00b4260 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fece4db7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece006d110 con 0x7fece016ae60 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fecddd9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fece0072bb0 0x7fece00b3b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37846/0 (socket says 172.21.15.97:37846) 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fecddd9b700 1 -- 172.21.15.97:0/2857563925 learned_addr learned my addr 172.21.15.97:0/2857563925 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958244986 0 0) 0x7fece008bdb0 con 0x7fece0072bb0 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc80035b0 con 0x7fece0072bb0 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fecded9d700 1 --2- 172.21.15.97:0/2857563925 >> v2:172.21.15.97:3300/0 conn(0x7fece00b4260 0x7fece0070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.407+0000 7fecde59c700 1 --2- 172.21.15.97:0/2857563925 >> v2:172.21.15.183:3300/0 conn(0x7fece016ae60 0x7fece006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7f86392c0700 1 -- 172.21.15.97:0/1505987898 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c00e070 con 0x7f8634072bb0 2023-11-24T06:09:06.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7f86392c0700 1 -- 172.21.15.97:0/1505987898 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c00b910 con 0x7f8634072bb0 2023-11-24T06:09:06.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1298853697 0 0) 0x7fecc80035b0 con 0x7fece0072bb0 2023-11-24T06:09:06.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 >> v2:172.21.15.183:3300/0 conn(0x7fece016ae60 msgr2=0x7fece006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 --2- 172.21.15.97:0/2857563925 >> v2:172.21.15.183:3300/0 conn(0x7fece016ae60 0x7fece006efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 >> v2:172.21.15.97:3300/0 conn(0x7fece00b4260 msgr2=0x7fece0070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 --2- 172.21.15.97:0/2857563925 >> v2:172.21.15.97:3300/0 conn(0x7fece00b4260 0x7fece0070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece006d3f0 con 0x7fece0072bb0 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fecd80019f0 con 0x7fece0072bb0 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 >> v1:172.21.15.97:6789/0 conn(0x7fece0072bb0 legacy=0x7fece00b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8003c20 0x7fecc8006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc80066c0 0x7fecc8008b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80091c0 0x7fecc800b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece006ce30 con 0x7fecc80066c0 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece006d110 con 0x7fecc8003c20 2023-11-24T06:09:06.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc800bfe0 con 0x7fecc80091c0 2023-11-24T06:09:06.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fa9127fc700 1 -- 172.21.15.97:0/3919680600 >> v2:172.21.15.97:3300/0 conn(0x7fa91415de50 msgr2=0x7fa91415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fa9127fc700 1 --2- 172.21.15.97:0/3919680600 >> v2:172.21.15.97:3300/0 conn(0x7fa91415de50 0x7fa91415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fa9127fc700 1 -- 172.21.15.97:0/3919680600 >> v1:172.21.15.97:6789/0 conn(0x7fa914151500 legacy=0x7fa9140cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fa9127fc700 1 -- 172.21.15.97:0/3919680600 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140717b0 con 0x7fa9140b49b0 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fa9127fc700 1 --2- 172.21.15.97:0/3919680600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9140b49b0 0x7fa91406d350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9140b2b80 tx=0x7fa90c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a9cabcd9d34d2f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fa911ffb700 1 -- 172.21.15.97:0/3919680600 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00cb10 con 0x7fa9140b49b0 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fa911ffb700 1 -- 172.21.15.97:0/3919680600 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90c00cc70 con 0x7fa9140b49b0 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fa911ffb700 1 -- 172.21.15.97:0/3919680600 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00a2e0 con 0x7fa9140b49b0 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7f86392c0700 1 -- 172.21.15.97:0/1505987898 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c00bca0 con 0x7f8634072bb0 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7f863c526700 1 -- 172.21.15.97:0/1505987898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634072bb0 msgr2=0x7f863406efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7f863c526700 1 --2- 172.21.15.97:0/1505987898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634072bb0 0x7f863406efc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f863406c600 tx=0x7f862c00b330 comp rx=0 tx=0).stop 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7f863c526700 1 -- 172.21.15.97:0/1505987898 shutdown_connections 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7f863c526700 1 --2- 172.21.15.97:0/1505987898 >> v2:172.21.15.97:3300/0 conn(0x7f86340b4260 0x7f8634070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7f863c526700 1 --2- 172.21.15.97:0/1505987898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8634072bb0 0x7f863406efc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7f863c526700 1 -- 172.21.15.97:0/1505987898 >> 172.21.15.97:0/1505987898 conn(0x7f86340cfa30 msgr2=0x7f86340cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fa919765700 1 -- 172.21.15.97:0/3919680600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9140b49b0 msgr2=0x7fa91406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fa919765700 1 --2- 172.21.15.97:0/3919680600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9140b49b0 0x7fa91406d350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa9140b2b80 tx=0x7fa90c00c330 comp rx=0 tx=0).stop 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.408+0000 7fecded9d700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80091c0 0x7fecc800b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecddd9b700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc80066c0 0x7fecc8008b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecddd9b700 1 -- 172.21.15.97:0/2857563925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80091c0 msgr2=0x7fecc800b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecddd9b700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80091c0 0x7fecc800b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecddd9b700 1 -- 172.21.15.97:0/2857563925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8003c20 msgr2=0x7fecc8006050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecddd9b700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8003c20 0x7fecc8006050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecddd9b700 1 -- 172.21.15.97:0/2857563925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc80035b0 con 0x7fecc80066c0 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecded9d700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fecc80091c0 0x7fecc800b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecded9d700 1 -- 172.21.15.97:0/2857563925 reap_dead start 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecddd9b700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc80066c0 0x7fecc8008b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fecd800a780 tx=0x7fecc8003860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328be85eb45a3df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.409+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8015c20 con 0x7fecc80066c0 2023-11-24T06:09:06.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd8015da0 con 0x7fecc80066c0 2023-11-24T06:09:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fecdd59a700 1 -- 172.21.15.97:0/2857563925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd801f670 con 0x7fecc80066c0 2023-11-24T06:09:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fa919765700 1 -- 172.21.15.97:0/3919680600 shutdown_connections 2023-11-24T06:09:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fa919765700 1 --2- 172.21.15.97:0/3919680600 >> v2:172.21.15.97:3300/0 conn(0x7fa91415de50 0x7fa91415e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fa919765700 1 --2- 172.21.15.97:0/3919680600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9140b49b0 0x7fa91406d350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fa919765700 1 -- 172.21.15.97:0/3919680600 >> 172.21.15.97:0/3919680600 conn(0x7fa9140b1740 msgr2=0x7fa9140b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fece4db7700 1 -- 172.21.15.97:0/2857563925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc80066c0 msgr2=0x7fecc8008b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7fece4db7700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc80066c0 0x7fecc8008b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fecd800a780 tx=0x7fecc8003860 comp rx=0 tx=0).stop 2023-11-24T06:09:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.411+0000 7faad8260700 1 Processor -- start 2023-11-24T06:09:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.411+0000 7fa919765700 1 -- 172.21.15.97:0/3919680600 shutdown_connections 2023-11-24T06:09:06.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7f863c526700 1 -- 172.21.15.97:0/1505987898 shutdown_connections 2023-11-24T06:09:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.410+0000 7f863c526700 1 -- 172.21.15.97:0/1505987898 wait complete. 2023-11-24T06:09:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7f863c526700 1 Processor -- start 2023-11-24T06:09:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7f863c526700 1 -- start start 2023-11-24T06:09:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.411+0000 7fa919765700 1 -- 172.21.15.97:0/3919680600 wait complete. 2023-11-24T06:09:06.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7fa919765700 1 Processor -- start 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.411+0000 7faad8260700 1 -- start start 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7faad8260700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faad015d6a0 0x7faad00b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7faad8260700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faad0071be0 0x7faad0140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7faad8260700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad00720e0 con 0x7faad00b4090 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7faad8260700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad0072d60 con 0x7faad015d6a0 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7faad8260700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad006de80 con 0x7faad0071be0 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7fece4db7700 1 -- 172.21.15.97:0/2857563925 shutdown_connections 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7fece4db7700 1 --2- 172.21.15.97:0/2857563925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc80066c0 0x7fecc8008b50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7fece4db7700 1 -- 172.21.15.97:0/2857563925 >> 172.21.15.97:0/2857563925 conn(0x7fece00cfa30 msgr2=0x7fece00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f863c526700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86340b4260 0x7f8634163b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f863c526700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f863416abf0 0x7f863408ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f863c526700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8634178880 0x7f8634178d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f863c526700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86341640c0 con 0x7f863416abf0 2023-11-24T06:09:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f863c526700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8634164a30 con 0x7f86340b4260 2023-11-24T06:09:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f863c526700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8634164d10 con 0x7f8634178880 2023-11-24T06:09:06.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f8639ac1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f863416abf0 0x7f863408ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f8639ac1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f863416abf0 0x7f863408ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45418/0 (socket says 172.21.15.97:45418) 2023-11-24T06:09:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f8639ac1700 1 -- 172.21.15.97:0/3485999201 learned_addr learned my addr 172.21.15.97:0/3485999201 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7faad5ffc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faad015d6a0 0x7faad00b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7faad5ffc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faad015d6a0 0x7faad00b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45412/0 (socket says 172.21.15.97:45412) 2023-11-24T06:09:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7faad5ffc700 1 -- 172.21.15.97:0/3203681540 learned_addr learned my addr 172.21.15.97:0/3203681540 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7faad5ffc700 1 -- 172.21.15.97:0/3203681540 >> v2:172.21.15.183:3300/0 conn(0x7faad0071be0 msgr2=0x7faad0140fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:09:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7faad5ffc700 1 --2- 172.21.15.97:0/3203681540 >> v2:172.21.15.183:3300/0 conn(0x7faad0071be0 0x7faad0140fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7faad5ffc700 1 -- 172.21.15.97:0/3203681540 >> v1:172.21.15.97:6789/0 conn(0x7faad00b4090 legacy=0x7faad00714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7faad5ffc700 1 -- 172.21.15.97:0/3203681540 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad00723c0 con 0x7faad015d6a0 2023-11-24T06:09:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f863a2c2700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86340b4260 0x7f8634163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.412+0000 7fa919765700 1 -- start start 2023-11-24T06:09:06.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa919765700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9140b49b0 0x7fa914156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa919765700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa914151500 0x7fa91414eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa919765700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa91415de50 0x7fa91415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa919765700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91414f030 con 0x7fa9140b49b0 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa919765700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa914157800 con 0x7fa914151500 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa919765700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140cc980 con 0x7fa91415de50 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa9127fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa914151500 0x7fa91414eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa9127fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa914151500 0x7fa91414eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39586/0 (socket says 172.21.15.97:39586) 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7fa9127fc700 1 -- 172.21.15.97:0/2953614448 learned_addr learned my addr 172.21.15.97:0/2953614448 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.413+0000 7f863aac3700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8634178880 0x7f8634178d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa9137fe700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa91415de50 0x7fa91415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa912ffd700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9140b49b0 0x7fa914156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7faad5ffc700 1 --2- 172.21.15.97:0/3203681540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad015d6a0 0x7faad00b3b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faad0070a50 tx=0x7faac800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9231ac91ae41f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7f8639ac1700 1 -- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8634178880 msgr2=0x7f8634178d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7f8639ac1700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8634178880 0x7f8634178d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7f8639ac1700 1 -- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86340b4260 msgr2=0x7f8634163b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7f8639ac1700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86340b4260 0x7f8634163b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7f8639ac1700 1 -- 172.21.15.97:0/3485999201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c009b30 con 0x7f863416abf0 2023-11-24T06:09:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa912ffd700 1 -- 172.21.15.97:0/2953614448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa91415de50 msgr2=0x7fa91415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa912ffd700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa91415de50 0x7fa91415cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa912ffd700 1 -- 172.21.15.97:0/2953614448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa914151500 msgr2=0x7fa91414eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa912ffd700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa914151500 0x7fa91414eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa912ffd700 1 -- 172.21.15.97:0/2953614448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c00ab80 con 0x7fa9140b49b0 2023-11-24T06:09:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa912ffd700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9140b49b0 0x7fa914156930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c400 tx=0x7fa90800c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa8fb7fe700 1 -- 172.21.15.97:0/2953614448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908005d70 con 0x7fa9140b49b0 2023-11-24T06:09:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7f8639ac1700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f863416abf0 0x7f863408ae40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f863000c450 tx=0x7f863000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa8fb7fe700 1 -- 172.21.15.97:0/2953614448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800ce80 con 0x7fa9140b49b0 2023-11-24T06:09:06.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa8fb7fe700 1 -- 172.21.15.97:0/2953614448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080072f0 con 0x7fa9140b49b0 2023-11-24T06:09:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.415+0000 7f862affd700 1 -- 172.21.15.97:0/3485999201 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300052d0 con 0x7f863416abf0 2023-11-24T06:09:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.415+0000 7f862affd700 1 -- 172.21.15.97:0/3485999201 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630005430 con 0x7f863416abf0 2023-11-24T06:09:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.415+0000 7f862affd700 1 -- 172.21.15.97:0/3485999201 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300038c0 con 0x7f863416abf0 2023-11-24T06:09:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.415+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8634177da0 con 0x7f863416abf0 2023-11-24T06:09:06.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.415+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8634178360 con 0x7f863416abf0 2023-11-24T06:09:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.414+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa914156f60 con 0x7fa9140b49b0 2023-11-24T06:09:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.415+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa914157520 con 0x7fa9140b49b0 2023-11-24T06:09:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.416+0000 7faad4ffa700 1 -- 172.21.15.97:0/3203681540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8009110 con 0x7faad015d6a0 2023-11-24T06:09:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.416+0000 7faad4ffa700 1 -- 172.21.15.97:0/3203681540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac8009270 con 0x7faad015d6a0 2023-11-24T06:09:06.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.416+0000 7faad4ffa700 1 -- 172.21.15.97:0/3203681540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8003c40 con 0x7faad015d6a0 2023-11-24T06:09:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.416+0000 7fece4db7700 1 -- 172.21.15.97:0/2857563925 shutdown_connections 2023-11-24T06:09:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7f863aac3700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8634178880 0x7f8634178d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7fa8fb7fe700 1 -- 172.21.15.97:0/2953614448 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa908007450 con 0x7fa9140b49b0 2023-11-24T06:09:06.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7faad8260700 1 -- 172.21.15.97:0/3203681540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad015d6a0 msgr2=0x7faad00b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7faad8260700 1 --2- 172.21.15.97:0/3203681540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad015d6a0 0x7faad00b3b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faad0070a50 tx=0x7faac800f740 comp rx=0 tx=0).stop 2023-11-24T06:09:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7fece4db7700 1 -- 172.21.15.97:0/2857563925 wait complete. 2023-11-24T06:09:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f40052e0 con 0x7fa9140b49b0 2023-11-24T06:09:06.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7f862affd700 1 -- 172.21.15.97:0/3485999201 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8630003a20 con 0x7f863416abf0 2023-11-24T06:09:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7fa8fb7fe700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa90006aed0 0x7fa90006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.417+0000 7fa8fb7fe700 1 -- 172.21.15.97:0/2953614448 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa908088950 con 0x7fa9140b49b0 2023-11-24T06:09:06.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.418+0000 7f862affd700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f861806afd0 0x7f861806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.418+0000 7fa9127fc700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa90006aed0 0x7fa90006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.418+0000 7f862affd700 1 -- 172.21.15.97:0/3485999201 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86300889b0 con 0x7f863416abf0 2023-11-24T06:09:06.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.418+0000 7f863a2c2700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f861806afd0 0x7f861806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.418+0000 7fa9127fc700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa90006aed0 0x7fa90006d390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9141fb620 tx=0x7fa90c01d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7faad8260700 1 -- 172.21.15.97:0/3203681540 shutdown_connections 2023-11-24T06:09:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7faad8260700 1 --2- 172.21.15.97:0/3203681540 >> v2:172.21.15.183:3300/0 conn(0x7faad0071be0 0x7faad0140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7faad8260700 1 --2- 172.21.15.97:0/3203681540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad015d6a0 0x7faad00b3b50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7faad8260700 1 -- 172.21.15.97:0/3203681540 >> 172.21.15.97:0/3203681540 conn(0x7faad00cfa30 msgr2=0x7faad00cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7faad8260700 1 -- 172.21.15.97:0/3203681540 shutdown_connections 2023-11-24T06:09:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7fece4db7700 1 Processor -- start 2023-11-24T06:09:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.420+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861c0052e0 con 0x7f863416abf0 2023-11-24T06:09:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7fece4db7700 1 -- start start 2023-11-24T06:09:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.420+0000 7fece4db7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc80066c0 0x7fece008b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.420+0000 7fece4db7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fece0150ec0 0x7fece0151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.420+0000 7fece4db7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fece00ce080 0x7fece00ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.420+0000 7fece4db7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece0156dc0 con 0x7fece00ce080 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.420+0000 7fece4db7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece008ba60 con 0x7fecc80066c0 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.420+0000 7fece4db7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece00ccfe0 con 0x7fece0150ec0 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecddd9b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fece0150ec0 0x7fece0151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecde59c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc80066c0 0x7fece008b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecddd9b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fece0150ec0 0x7fece0151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60704/0 (socket says 172.21.15.97:60704) 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecddd9b700 1 -- 172.21.15.97:0/4098362879 learned_addr learned my addr 172.21.15.97:0/4098362879 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7faad8260700 1 -- 172.21.15.97:0/3203681540 wait complete. 2023-11-24T06:09:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7faad8260700 1 Processor -- start 2023-11-24T06:09:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecded9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fece00ce080 0x7fece00ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecde59c700 1 -- 172.21.15.97:0/4098362879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fece0150ec0 msgr2=0x7fece0151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecde59c700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fece0150ec0 0x7fece0151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecde59c700 1 -- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fece00ce080 msgr2=0x7fece00ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecde59c700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fece00ce080 0x7fece00ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7fecde59c700 1 -- 172.21.15.97:0/4098362879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00cd2c0 con 0x7fecc80066c0 2023-11-24T06:09:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7faad8260700 1 -- start start 2023-11-24T06:09:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad8260700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faad0071be0 0x7faad0159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad8260700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad00b4090 0x7faad0158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad8260700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faad015d6a0 0x7faad00ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad8260700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00cf6b0 con 0x7faad00b4090 2023-11-24T06:09:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad8260700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00ceaa0 con 0x7faad0071be0 2023-11-24T06:09:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad8260700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad00ced80 con 0x7faad015d6a0 2023-11-24T06:09:06.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad5ffc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faad0071be0 0x7faad0159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad5ffc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faad0071be0 0x7faad0159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39606/0 (socket says 172.21.15.97:39606) 2023-11-24T06:09:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad5ffc700 1 -- 172.21.15.97:0/11259482 learned_addr learned my addr 172.21.15.97:0/11259482 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7faad67fd700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faad015d6a0 0x7faad00ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7f37ce352700 1 Processor -- start 2023-11-24T06:09:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.419+0000 7f37ce352700 1 -- start start 2023-11-24T06:09:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37ce352700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f37c80b3b50 0x7f37c80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37ce352700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37c806d4d0 0x7f37c806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37ce352700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c806ced0 con 0x7f37c8163a80 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37ce352700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c806da70 con 0x7f37c806d4d0 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37ce352700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c80b4470 con 0x7f37c80b3b50 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37cc8ef700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f37c8163a80 0x7f37c8163ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37894/0 (socket says 172.21.15.97:37894) 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37cc8ef700 1 -- 172.21.15.97:0/3660503355 learned_addr learned my addr 172.21.15.97:0/3660503355 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37c7fff700 1 --2- 172.21.15.97:0/3660503355 >> v2:172.21.15.183:3300/0 conn(0x7f37c80b3b50 0x7f37c80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37c77fe700 1 --2- 172.21.15.97:0/3660503355 >> v2:172.21.15.97:3300/0 conn(0x7f37c806d4d0 0x7f37c806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37c7fff700 1 -- 172.21.15.97:0/3660503355 >> v2:172.21.15.97:3300/0 conn(0x7f37c806d4d0 msgr2=0x7f37c806eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37c7fff700 1 --2- 172.21.15.97:0/3660503355 >> v2:172.21.15.97:3300/0 conn(0x7f37c806d4d0 0x7f37c806eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37c7fff700 1 -- 172.21.15.97:0/3660503355 >> v1:172.21.15.97:6789/0 conn(0x7f37c8163a80 legacy=0x7f37c8163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.421+0000 7f37c7fff700 1 -- 172.21.15.97:0/3660503355 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c81634e0 con 0x7f37c80b3b50 2023-11-24T06:09:06.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7f37c7fff700 1 --2- 172.21.15.97:0/3660503355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80b3b50 0x7f37c80b3f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f37c80750c0 tx=0x7f37c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=369c3da03cb3ad36 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7f37c6ffd700 1 -- 172.21.15.97:0/3660503355 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c000e070 con 0x7f37c80b3b50 2023-11-24T06:09:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7f37c6ffd700 1 -- 172.21.15.97:0/3660503355 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c000bb20 con 0x7f37c80b3b50 2023-11-24T06:09:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.422+0000 7f37c6ffd700 1 -- 172.21.15.97:0/3660503355 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c0012430 con 0x7f37c80b3b50 2023-11-24T06:09:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad5ffc700 1 -- 172.21.15.97:0/11259482 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faad015d6a0 msgr2=0x7faad00ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad5ffc700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faad015d6a0 0x7faad00ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad57fb700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad00b4090 0x7faad0158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad5ffc700 1 -- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad00b4090 msgr2=0x7faad0158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad5ffc700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad00b4090 0x7faad0158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad5ffc700 1 -- 172.21.15.97:0/11259482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac8009b30 con 0x7faad0071be0 2023-11-24T06:09:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad57fb700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad00b4090 0x7faad0158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad5ffc700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faad0071be0 0x7faad0159570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faac800f780 tx=0x7faac8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7f37ce352700 1 -- 172.21.15.97:0/3660503355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80b3b50 msgr2=0x7f37c80b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7f37ce352700 1 --2- 172.21.15.97:0/3660503355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80b3b50 0x7f37c80b3f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f37c80750c0 tx=0x7f37c000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7f37ce352700 1 -- 172.21.15.97:0/3660503355 shutdown_connections 2023-11-24T06:09:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7f37ce352700 1 --2- 172.21.15.97:0/3660503355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c80b3b50 0x7f37c80b3f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7f37ce352700 1 --2- 172.21.15.97:0/3660503355 >> v2:172.21.15.97:3300/0 conn(0x7f37c806d4d0 0x7f37c806eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7f37ce352700 1 -- 172.21.15.97:0/3660503355 >> 172.21.15.97:0/3660503355 conn(0x7f37c80cfa30 msgr2=0x7f37c80cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7f37ce352700 1 -- 172.21.15.97:0/3660503355 shutdown_connections 2023-11-24T06:09:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7f37ce352700 1 -- 172.21.15.97:0/3660503355 wait complete. 2023-11-24T06:09:06.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faac67fc700 1 -- 172.21.15.97:0/11259482 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac80048a0 con 0x7faad0071be0 2023-11-24T06:09:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faac67fc700 1 -- 172.21.15.97:0/11259482 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac80090f0 con 0x7faad0071be0 2023-11-24T06:09:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faac67fc700 1 -- 172.21.15.97:0/11259482 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8022680 con 0x7faad0071be0 2023-11-24T06:09:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad00cf060 con 0x7faad0071be0 2023-11-24T06:09:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.423+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad0158bd0 con 0x7faad0071be0 2023-11-24T06:09:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.424+0000 7f863a2c2700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f861806afd0 0x7f861806d490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f862c000f80 tx=0x7f862c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.425+0000 7fecde59c700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc80066c0 0x7fece008b520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fece00505a0 tx=0x7fecd400eae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.425+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab80052e0 con 0x7faad0071be0 2023-11-24T06:09:06.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.425+0000 7f37ce352700 1 Processor -- start 2023-11-24T06:09:06.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.425+0000 7fecceffd700 1 -- 172.21.15.97:0/4098362879 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4013680 con 0x7fecc80066c0 2023-11-24T06:09:06.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.425+0000 7fecceffd700 1 -- 172.21.15.97:0/4098362879 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd40137e0 con 0x7fecc80066c0 2023-11-24T06:09:06.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.425+0000 7fecceffd700 1 -- 172.21.15.97:0/4098362879 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4013b90 con 0x7fecc80066c0 2023-11-24T06:09:06.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.425+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece00cd5a0 con 0x7fecc80066c0 2023-11-24T06:09:06.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.425+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece00cdb60 con 0x7fecc80066c0 2023-11-24T06:09:06.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.426+0000 7faac67fc700 1 -- 172.21.15.97:0/11259482 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faac802b3f0 con 0x7faad0071be0 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.426+0000 7f37ce352700 1 -- start start 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37ce352700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c806d4d0 0x7f37c814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37ce352700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b3b50 0x7f37c80ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37ce352700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8163a80 0x7f37c814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37ce352700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c81535f0 con 0x7f37c806d4d0 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37ce352700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80ccf70 con 0x7f37c8163a80 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37ce352700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c814b740 con 0x7f37c80b3b50 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37c7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c806d4d0 0x7f37c814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7faac67fc700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faab406afa0 0x7faab406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7faac67fc700 1 -- 172.21.15.97:0/11259482 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faac808a3b0 con 0x7faad0071be0 2023-11-24T06:09:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37c7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c806d4d0 0x7f37c814b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45452/0 (socket says 172.21.15.97:45452) 2023-11-24T06:09:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37c7fff700 1 -- 172.21.15.97:0/2083752961 learned_addr learned my addr 172.21.15.97:0/2083752961 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37cc8ef700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8163a80 0x7f37c814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.427+0000 7f37c77fe700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b3b50 0x7f37c80ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7fecceffd700 1 -- 172.21.15.97:0/4098362879 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecd4013cf0 con 0x7fecc80066c0 2023-11-24T06:09:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7f37c7fff700 1 -- 172.21.15.97:0/2083752961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b3b50 msgr2=0x7f37c80ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7f37c7fff700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b3b50 0x7f37c80ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7f37c7fff700 1 -- 172.21.15.97:0/2083752961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8163a80 msgr2=0x7f37c814cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7f37c7fff700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8163a80 0x7f37c814cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7f37c7fff700 1 -- 172.21.15.97:0/2083752961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c0009a70 con 0x7f37c806d4d0 2023-11-24T06:09:06.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7f37c7fff700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c806d4d0 0x7f37c814b200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f37c000b370 tx=0x7f37c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7faad57fb700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faab406afa0 0x7faab406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7fa8fb7fe700 1 -- 172.21.15.97:0/2953614448 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9080539f0 con 0x7fa9140b49b0 2023-11-24T06:09:06.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7fecceffd700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fecc006afa0 0x7fecc006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece00cdd80 con 0x7fecc80066c0 2023-11-24T06:09:06.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7fecceffd700 1 -- 172.21.15.97:0/4098362879 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecd4089650 con 0x7fecc80066c0 2023-11-24T06:09:06.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7fecddd9b700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fecc006afa0 0x7fecc006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7f108f350700 1 Processor -- start 2023-11-24T06:09:06.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.428+0000 7f1252bbc700 1 Processor -- start 2023-11-24T06:09:06.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.429+0000 7faad57fb700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faab406afa0 0x7faab406d460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faad0159ab0 tx=0x7faacc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.429+0000 7fecddd9b700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fecc006afa0 0x7fecc006d460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fecd8001330 tx=0x7fecd8006000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.429+0000 7f108f350700 1 -- start start 2023-11-24T06:09:06.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f108f350700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f10880b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f108f350700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10880af6d0 0x7f1088071da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f108f350700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1088071910 con 0x7f1088162220 2023-11-24T06:09:06.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f108f350700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10880afb40 con 0x7f10880af6d0 2023-11-24T06:09:06.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f108f350700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1088070fe0 con 0x7f1088166660 2023-11-24T06:09:06.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.429+0000 7f1252bbc700 1 -- start start 2023-11-24T06:09:06.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f1252bbc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0cf440 0x7f124c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f1252bbc700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c072110 con 0x7f124c1665a0 2023-11-24T06:09:06.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f1252bbc700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c06ee00 con 0x7f124c070fd0 2023-11-24T06:09:06.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f1252bbc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c071500 con 0x7f124c0cf440 2023-11-24T06:09:06.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f1250958700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0cf440 0x7f124c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f124bfff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f124c070fd0 0x7f124c165e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:47242/0 (socket says 172.21.15.97:47242) 2023-11-24T06:09:06.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f124bfff700 1 -- 172.21.15.97:0/1895497109 learned_addr learned my addr 172.21.15.97:0/1895497109 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:06.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f108d0ec700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1088162220 0x7f1088165f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37914/0 (socket says 172.21.15.97:37914) 2023-11-24T06:09:06.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f108d0ec700 1 -- 172.21.15.97:0/768282636 learned_addr learned my addr 172.21.15.97:0/768282636 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:06.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.430+0000 7f108c8eb700 1 --2- 172.21.15.97:0/768282636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f10880b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f108d8ed700 1 --2- 172.21.15.97:0/768282636 >> v2:172.21.15.183:3300/0 conn(0x7f10880af6d0 0x7f1088071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f1250958700 1 -- 172.21.15.97:0/1895497109 >> v1:172.21.15.183:6789/0 conn(0x7f124c070fd0 legacy=0x7f124c165e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f1250958700 1 -- 172.21.15.97:0/1895497109 >> v1:172.21.15.97:6789/0 conn(0x7f124c1665a0 legacy=0x7f124c0b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f1250958700 1 -- 172.21.15.97:0/1895497109 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0717e0 con 0x7f124c0cf440 2023-11-24T06:09:06.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f108c8eb700 1 -- 172.21.15.97:0/768282636 >> v2:172.21.15.183:3300/0 conn(0x7f10880af6d0 msgr2=0x7f1088071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:06.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f108c8eb700 1 --2- 172.21.15.97:0/768282636 >> v2:172.21.15.183:3300/0 conn(0x7f10880af6d0 0x7f1088071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f108c8eb700 1 -- 172.21.15.97:0/768282636 >> v1:172.21.15.97:6789/0 conn(0x7f1088162220 legacy=0x7f1088165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f108c8eb700 1 -- 172.21.15.97:0/768282636 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10880712c0 con 0x7f1088166660 2023-11-24T06:09:06.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f1250958700 1 --2- 172.21.15.97:0/1895497109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0cf440 0x7f124c06e8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f123c009870 tx=0x7f123c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767aaedf86f4ace1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f108c8eb700 1 --2- 172.21.15.97:0/768282636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f10880b5190 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1078009840 tx=0x7f1078009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5faac2340a576f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f124b7fe700 1 -- 172.21.15.97:0/1895497109 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c011830 con 0x7f124c0cf440 2023-11-24T06:09:06.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f124b7fe700 1 -- 172.21.15.97:0/1895497109 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c011990 con 0x7f124c0cf440 2023-11-24T06:09:06.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f124b7fe700 1 -- 172.21.15.97:0/1895497109 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c011d20 con 0x7f124c0cf440 2023-11-24T06:09:06.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f107ffff700 1 -- 172.21.15.97:0/768282636 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078011830 con 0x7f1088166660 2023-11-24T06:09:06.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f107ffff700 1 -- 172.21.15.97:0/768282636 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1078011990 con 0x7f1088166660 2023-11-24T06:09:06.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7f107ffff700 1 -- 172.21.15.97:0/768282636 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078011d20 con 0x7f1088166660 2023-11-24T06:09:06.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.431+0000 7faac67fc700 1 -- 172.21.15.97:0/11259482 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faac8055590 con 0x7faad0071be0 2023-11-24T06:09:06.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f108f350700 1 -- 172.21.15.97:0/768282636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 msgr2=0x7f10880b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f108f350700 1 --2- 172.21.15.97:0/768282636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f10880b5190 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1078009840 tx=0x7f1078009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f1252bbc700 1 -- 172.21.15.97:0/1895497109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0cf440 msgr2=0x7f124c06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f1252bbc700 1 --2- 172.21.15.97:0/1895497109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0cf440 0x7f124c06e8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f123c009870 tx=0x7f123c009b80 comp rx=0 tx=0).stop 2023-11-24T06:09:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f1252bbc700 1 -- 172.21.15.97:0/1895497109 shutdown_connections 2023-11-24T06:09:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f1252bbc700 1 --2- 172.21.15.97:0/1895497109 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c0cf440 0x7f124c06e8c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f1252bbc700 1 -- 172.21.15.97:0/1895497109 >> 172.21.15.97:0/1895497109 conn(0x7f124c0b1740 msgr2=0x7f124c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f1252bbc700 1 -- 172.21.15.97:0/1895497109 shutdown_connections 2023-11-24T06:09:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.432+0000 7f1252bbc700 1 -- 172.21.15.97:0/1895497109 wait complete. 2023-11-24T06:09:06.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.433+0000 7f37c4ff9700 1 -- 172.21.15.97:0/2083752961 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c0004800 con 0x7f37c806d4d0 2023-11-24T06:09:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.433+0000 7f108f350700 1 -- 172.21.15.97:0/768282636 shutdown_connections 2023-11-24T06:09:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.433+0000 7f108f350700 1 --2- 172.21.15.97:0/768282636 >> v2:172.21.15.183:3300/0 conn(0x7f10880af6d0 0x7f1088071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.433+0000 7f108f350700 1 --2- 172.21.15.97:0/768282636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f10880b5190 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.433+0000 7f108f350700 1 -- 172.21.15.97:0/768282636 >> 172.21.15.97:0/768282636 conn(0x7f108806cec0 msgr2=0x7f108806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.433+0000 7f37c4ff9700 1 -- 172.21.15.97:0/2083752961 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c0020070 con 0x7f37c806d4d0 2023-11-24T06:09:06.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.433+0000 7f37c4ff9700 1 -- 172.21.15.97:0/2083752961 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c00276c0 con 0x7f37c806d4d0 2023-11-24T06:09:06.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.434+0000 7f108f350700 1 -- 172.21.15.97:0/768282636 shutdown_connections 2023-11-24T06:09:06.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.434+0000 7f862affd700 1 -- 172.21.15.97:0/3485999201 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8630053a60 con 0x7f863416abf0 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.433+0000 7f1252bbc700 1 Processor -- start 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.434+0000 7f1252bbc700 1 -- start start 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1252bbc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c070fd0 0x7f124c151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1252bbc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0cf440 0x7f124c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1252bbc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c1665a0 0x7f124c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1252bbc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c15d9e0 con 0x7f124c0cf440 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1252bbc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c15b0f0 con 0x7f124c070fd0 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1252bbc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c15d0b0 con 0x7f124c1665a0 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1250958700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c070fd0 0x7f124c151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1250958700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c070fd0 0x7f124c151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39634/0 (socket says 172.21.15.97:39634) 2023-11-24T06:09:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1250958700 1 -- 172.21.15.97:0/450210916 learned_addr learned my addr 172.21.15.97:0/450210916 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1251159700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c1665a0 0x7f124c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.434+0000 7f108f350700 1 -- 172.21.15.97:0/768282636 wait complete. 2023-11-24T06:09:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f108f350700 1 Processor -- start 2023-11-24T06:09:06.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f108f350700 1 -- start start 2023-11-24T06:09:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f124bfff700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0cf440 0x7f124c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1251159700 1 -- 172.21.15.97:0/450210916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c070fd0 msgr2=0x7f124c151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1251159700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c070fd0 0x7f124c151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1251159700 1 -- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0cf440 msgr2=0x7f124c1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1251159700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0cf440 0x7f124c1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.435+0000 7f1251159700 1 -- 172.21.15.97:0/450210916 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123c0094a0 con 0x7f124c1665a0 2023-11-24T06:09:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.436+0000 7f1251159700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c1665a0 0x7f124c15de70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1234000f60 tx=0x7f123400c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.436+0000 7fecceffd700 1 -- 172.21.15.97:0/4098362879 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecd4021080 con 0x7fecc80066c0 2023-11-24T06:09:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.436+0000 7f108f350700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10880af6d0 0x7f10881531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.436+0000 7f108f350700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088162220 0x7f10881f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.436+0000 7f108f350700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f1088160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.436+0000 7f108f350700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10881649b0 con 0x7f1088162220 2023-11-24T06:09:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.436+0000 7f108f350700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108815ff00 con 0x7f10880af6d0 2023-11-24T06:09:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.436+0000 7f108f350700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10881f8040 con 0x7f1088166660 2023-11-24T06:09:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088162220 0x7f10881f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088162220 0x7f10881f9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45482/0 (socket says 172.21.15.97:45482) 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 -- 172.21.15.97:0/1004860678 learned_addr learned my addr 172.21.15.97:0/1004860678 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108d0ec700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10880af6d0 0x7f10881531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108d8ed700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f1088160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 -- 172.21.15.97:0/1004860678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 msgr2=0x7f1088160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f1088160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 -- 172.21.15.97:0/1004860678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10880af6d0 msgr2=0x7f10881531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10880af6d0 0x7f10881531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 -- 172.21.15.97:0/1004860678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10780094a0 con 0x7f1088162220 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108c8eb700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088162220 0x7f10881f9200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1078009b90 tx=0x7f1078004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f107dffb700 1 -- 172.21.15.97:0/1004860678 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10780036a0 con 0x7f1088162220 2023-11-24T06:09:06.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f107dffb700 1 -- 172.21.15.97:0/1004860678 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107801f070 con 0x7f1088162220 2023-11-24T06:09:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f107dffb700 1 -- 172.21.15.97:0/1004860678 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1078003a20 con 0x7f1088162220 2023-11-24T06:09:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.437+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1088153700 con 0x7f1088162220 2023-11-24T06:09:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.438+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1088164110 con 0x7f1088162220 2023-11-24T06:09:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.439+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1088067f60 con 0x7f1088162220 2023-11-24T06:09:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.440+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c8153030 con 0x7f37c806d4d0 2023-11-24T06:09:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.440+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c80ccc00 con 0x7f37c806d4d0 2023-11-24T06:09:06.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.441+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c80751e0 con 0x7f37c806d4d0 2023-11-24T06:09:06.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.443+0000 7f107dffb700 1 -- 172.21.15.97:0/1004860678 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1078003b80 con 0x7f1088162220 2023-11-24T06:09:06.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.443+0000 7f107dffb700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106c06aed0 0x7f106c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.443+0000 7f107dffb700 1 -- 172.21.15.97:0/1004860678 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1078021070 con 0x7f1088162220 2023-11-24T06:09:06.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.443+0000 7f108d0ec700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106c06aed0 0x7f106c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.444+0000 7f108d0ec700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106c06aed0 0x7f106c06d390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1084005910 tx=0x7f1084009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.439+0000 7f12497fa700 1 -- 172.21.15.97:0/450210916 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12340052d0 con 0x7f124c1665a0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.439+0000 7f12497fa700 1 -- 172.21.15.97:0/450210916 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1234005450 con 0x7f124c1665a0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.439+0000 7f12497fa700 1 -- 172.21.15.97:0/450210916 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12340038c0 con 0x7f124c1665a0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.439+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c15d390 con 0x7f124c1665a0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.439+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c153f30 con 0x7f124c1665a0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.440+0000 7f12497fa700 1 -- 172.21.15.97:0/450210916 <== mon.2 v2:172.21.15.183:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1234003a20 con 0x7f124c1665a0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.441+0000 7f12497fa700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f122c06af20 0x7f122c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.441+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12300052e0 con 0x7f124c1665a0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.441+0000 7f12497fa700 1 -- 172.21.15.97:0/450210916 <== mon.2 v2:172.21.15.183:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1234088890 con 0x7f124c1665a0 2023-11-24T06:09:06.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.443+0000 7f1250958700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f122c06af20 0x7f122c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.445+0000 7f1250958700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f122c06af20 0x7f122c06d3e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f123c009bc0 tx=0x7f123c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.446+0000 7f12497fa700 1 -- 172.21.15.97:0/450210916 <== mon.2 v2:172.21.15.183:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1234053920 con 0x7f124c1665a0 2023-11-24T06:09:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.451+0000 7f107dffb700 1 -- 172.21.15.97:0/1004860678 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1078056130 con 0x7f1088162220 2023-11-24T06:09:06.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.463+0000 7f37c4ff9700 1 -- 172.21.15.97:0/2083752961 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37c0031430 con 0x7f37c806d4d0 2023-11-24T06:09:06.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.463+0000 7f37c4ff9700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37b406aed0 0x7f37b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:06.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.463+0000 7f37c4ff9700 1 -- 172.21.15.97:0/2083752961 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37c0022070 con 0x7f37c806d4d0 2023-11-24T06:09:06.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.463+0000 7f37c4ff9700 1 -- 172.21.15.97:0/2083752961 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37c008bae0 con 0x7f37c806d4d0 2023-11-24T06:09:06.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.464+0000 7f37c77fe700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37b406aed0 0x7f37b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:06.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.464+0000 7f37c77fe700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37b406aed0 0x7f37b406d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37c8162280 tx=0x7f37bc009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.788+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7974005860 con 0x7f7994071be0 2023-11-24T06:09:06.796 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-11-24T06:09:06.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.790+0000 7f7990ff9700 1 -- 172.21.15.97:0/2921672626 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f798c059a80 con 0x7f7994071be0 2023-11-24T06:09:06.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.799+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f798006af20 msgr2=0x7f798006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.799+0000 7f799a686700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f798006af20 0x7f798006d3e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f79941e4f20 tx=0x7f7984009040 comp rx=0 tx=0).stop 2023-11-24T06:09:06.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.799+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994071be0 msgr2=0x7f79940aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.799+0000 7f799a686700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994071be0 0x7f79940aff50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f798c018040 tx=0x7f798c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.801+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 shutdown_connections 2023-11-24T06:09:06.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.801+0000 7f799a686700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f799415d6a0 0x7f799414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.801+0000 7f799a686700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994071be0 0x7f79940aff50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.801+0000 7f799a686700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f798006af20 0x7f798006d3e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.801+0000 7f799a686700 1 --2- 172.21.15.97:0/2921672626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79940b4090 0x7f799414f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.801+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 >> 172.21.15.97:0/2921672626 conn(0x7f79940cfa30 msgr2=0x7f79941434d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.801+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 shutdown_connections 2023-11-24T06:09:06.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.801+0000 7f799a686700 1 -- 172.21.15.97:0/2921672626 wait complete. 2023-11-24T06:09:06.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-11-24T06:09:06.822 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.836+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fece00751e0 con 0x7fecc80066c0 2023-11-24T06:09:06.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.838+0000 7fecceffd700 1 -- 172.21.15.97:0/4098362879 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fecd4017020 con 0x7fecc80066c0 2023-11-24T06:09:06.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.838+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7faab8005590 con 0x7faad0071be0 2023-11-24T06:09:06.841 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-11-24T06:09:06.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.839+0000 7faac67fc700 1 -- 172.21.15.97:0/11259482 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faac8058bb0 con 0x7faad0071be0 2023-11-24T06:09:06.842 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-11-24T06:09:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.840+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa8f4005590 con 0x7fa9140b49b0 2023-11-24T06:09:06.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.841+0000 7fa8fb7fe700 1 -- 172.21.15.97:0/2953614448 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa908017090 con 0x7fa9140b49b0 2023-11-24T06:09:06.844 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-11-24T06:09:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.842+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faab406afa0 msgr2=0x7faab406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.842+0000 7faad8260700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faab406afa0 0x7faab406d460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faad0159ab0 tx=0x7faacc009040 comp rx=0 tx=0).stop 2023-11-24T06:09:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.842+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faad0071be0 msgr2=0x7faad0159570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.842+0000 7faad8260700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faad0071be0 0x7faad0159570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faac800f780 tx=0x7faac8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 shutdown_connections 2023-11-24T06:09:06.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7faad8260700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faab406afa0 0x7faab406d460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7faad8260700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faad015d6a0 0x7faad00ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7faad8260700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faad0071be0 0x7faad0159570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7faad8260700 1 --2- 172.21.15.97:0/11259482 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faad00b4090 0x7faad0158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 >> 172.21.15.97:0/11259482 conn(0x7faad00cfa30 msgr2=0x7faad0140270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fecc006afa0 msgr2=0x7fecc006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fecc006afa0 0x7fecc006d460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fecd8001330 tx=0x7fecd8006000 comp rx=0 tx=0).stop 2023-11-24T06:09:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc80066c0 msgr2=0x7fece008b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc80066c0 0x7fece008b520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fece00505a0 tx=0x7fecd400eae0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 shutdown_connections 2023-11-24T06:09:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1230005590 con 0x7f124c1665a0 2023-11-24T06:09:06.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7faad8260700 1 -- 172.21.15.97:0/11259482 wait complete. 2023-11-24T06:09:06.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa90006aed0 msgr2=0x7fa90006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fa919765700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa90006aed0 0x7fa90006d390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9141fb620 tx=0x7fa90c01d040 comp rx=0 tx=0).stop 2023-11-24T06:09:06.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9140b49b0 msgr2=0x7fa914156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fa919765700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9140b49b0 0x7fa914156930 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c400 tx=0x7fa90800c710 comp rx=0 tx=0).stop 2023-11-24T06:09:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 shutdown_connections 2023-11-24T06:09:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fece00ce080 0x7fece00ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc80066c0 0x7fece008b520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fecc006afa0 0x7fecc006d460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 --2- 172.21.15.97:0/4098362879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fece0150ec0 0x7fece0151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.844+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 >> 172.21.15.97:0/4098362879 conn(0x7fece00cfa30 msgr2=0x7fece016d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 shutdown_connections 2023-11-24T06:09:06.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fece4db7700 1 -- 172.21.15.97:0/4098362879 wait complete. 2023-11-24T06:09:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 shutdown_connections 2023-11-24T06:09:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fa919765700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa90006aed0 0x7fa90006d390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fa919765700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa91415de50 0x7fa91415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fa919765700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9140b49b0 0x7fa914156930 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fa919765700 1 --2- 172.21.15.97:0/2953614448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa914151500 0x7fa91414eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 >> 172.21.15.97:0/2953614448 conn(0x7fa9140b1740 msgr2=0x7fa91415c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 shutdown_connections 2023-11-24T06:09:06.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7f12497fa700 1 -- 172.21.15.97:0/450210916 <== mon.2 v2:172.21.15.183:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1234017020 con 0x7f124c1665a0 2023-11-24T06:09:06.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f37c8157c20 con 0x7f37c806d4d0 2023-11-24T06:09:06.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7fa919765700 1 -- 172.21.15.97:0/2953614448 wait complete. 2023-11-24T06:09:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.845+0000 7f37c4ff9700 1 -- 172.21.15.97:0/2083752961 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f37c002c090 con 0x7f37c806d4d0 2023-11-24T06:09:06.854 INFO:teuthology.orchestra.run.smithi097.stdout:55834574857 2023-11-24T06:09:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.846+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f861c005590 con 0x7f863416abf0 2023-11-24T06:09:06.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.846+0000 7f862affd700 1 -- 172.21.15.97:0/3485999201 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8630017030 con 0x7f863416abf0 2023-11-24T06:09:06.855 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-11-24T06:09:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.848+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37b406aed0 msgr2=0x7f37b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.848+0000 7f37ce352700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37b406aed0 0x7f37b406d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37c8162280 tx=0x7f37bc009210 comp rx=0 tx=0).stop 2023-11-24T06:09:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.848+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c806d4d0 msgr2=0x7f37c814b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.848+0000 7f37ce352700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c806d4d0 0x7f37c814b200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f37c000b370 tx=0x7f37c0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.856 INFO:teuthology.orchestra.run.smithi097.stdout:55834574855 2023-11-24T06:09:06.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f122c06af20 msgr2=0x7f122c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f122c06af20 0x7f122c06d3e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f123c009bc0 tx=0x7f123c008030 comp rx=0 tx=0).stop 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c1665a0 msgr2=0x7f124c15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c1665a0 0x7f124c15de70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1234000f60 tx=0x7f123400c720 comp rx=0 tx=0).stop 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 shutdown_connections 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f122c06af20 0x7f122c06d3e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f124c1665a0 0x7f124c15de70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f124c070fd0 0x7f124c151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 --2- 172.21.15.97:0/450210916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f124c0cf440 0x7f124c1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.852+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 >> 172.21.15.97:0/450210916 conn(0x7f124c0b1740 msgr2=0x7f124c0b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.853+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 shutdown_connections 2023-11-24T06:09:06.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f861806afd0 msgr2=0x7f861806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f861806afd0 0x7f861806d490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f862c000f80 tx=0x7f862c009000 comp rx=0 tx=0).stop 2023-11-24T06:09:06.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f863416abf0 msgr2=0x7f863408ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f863416abf0 0x7f863408ae40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f863000c450 tx=0x7f863000c760 comp rx=0 tx=0).stop 2023-11-24T06:09:06.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 shutdown_connections 2023-11-24T06:09:06.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f861806afd0 0x7f861806d490 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f862c000f80 tx=0x7f862c009000 comp rx=0 tx=0).stop 2023-11-24T06:09:06.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8634178880 0x7f8634178d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86340b4260 0x7f8634163b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 --2- 172.21.15.97:0/3485999201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f863416abf0 0x7f863408ae40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.854+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 >> 172.21.15.97:0/3485999201 conn(0x7f86340cfa30 msgr2=0x7f86341652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.856+0000 7f1252bbc700 1 -- 172.21.15.97:0/450210916 wait complete. 2023-11-24T06:09:06.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.857+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 shutdown_connections 2023-11-24T06:09:06.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.857+0000 7f37ce352700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8163a80 0x7f37c814cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.857+0000 7f37ce352700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c806d4d0 0x7f37c814b200 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.857+0000 7f37ce352700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37b406aed0 0x7f37b406d390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.857+0000 7f37ce352700 1 --2- 172.21.15.97:0/2083752961 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37c80b3b50 0x7f37c80ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.857+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 >> 172.21.15.97:0/2083752961 conn(0x7f37c80cfa30 msgr2=0x7f37c80af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.858+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 shutdown_connections 2023-11-24T06:09:06.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.858+0000 7f863c526700 1 -- 172.21.15.97:0/3485999201 wait complete. 2023-11-24T06:09:06.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.858+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f10880cc470 con 0x7f1088162220 2023-11-24T06:09:06.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.858+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 shutdown_connections 2023-11-24T06:09:06.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.858+0000 7f37ce352700 1 -- 172.21.15.97:0/2083752961 wait complete. 2023-11-24T06:09:06.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.858+0000 7f107dffb700 1 -- 172.21.15.97:0/1004860678 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1078015020 con 0x7f1088162220 2023-11-24T06:09:06.862 INFO:teuthology.orchestra.run.smithi097.stdout:55834574856 2023-11-24T06:09:06.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.862+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106c06aed0 msgr2=0x7f106c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.862+0000 7f108f350700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106c06aed0 0x7f106c06d390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1084005910 tx=0x7f1084009040 comp rx=0 tx=0).stop 2023-11-24T06:09:06.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.862+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088162220 msgr2=0x7f10881f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:06.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.862+0000 7f108f350700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088162220 0x7f10881f9200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1078009b90 tx=0x7f1078004750 comp rx=0 tx=0).stop 2023-11-24T06:09:06.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-11-24T06:09:06.866 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.865+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 shutdown_connections 2023-11-24T06:09:06.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.865+0000 7f108f350700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106c06aed0 0x7f106c06d390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.865+0000 7f108f350700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1088166660 0x7f1088160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.865+0000 7f108f350700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10880af6d0 0x7f10881531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.865+0000 7f108f350700 1 --2- 172.21.15.97:0/1004860678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088162220 0x7f10881f9200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:06.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.865+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 >> 172.21.15.97:0/1004860678 conn(0x7f108806cec0 msgr2=0x7f10880b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:06.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.865+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 shutdown_connections 2023-11-24T06:09:06.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:06.865+0000 7f108f350700 1 -- 172.21.15.97:0/1004860678 wait complete. 2023-11-24T06:09:06.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-11-24T06:09:06.873 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-11-24T06:09:06.873 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-11-24T06:09:06.877 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-11-24T06:09:06.883 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-11-24T06:09:06.887 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-11-24T06:09:06.888 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:06.888 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:09:06.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:09:07.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.043+0000 7f969ed07700 1 Processor -- start 2023-11-24T06:09:07.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.043+0000 7f969ed07700 1 -- start start 2023-11-24T06:09:07.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969ed07700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96980af4d0 0x7f96980b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969ed07700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980ae430 con 0x7f969815e080 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969ed07700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980ae710 con 0x7f969806c710 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969ed07700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96980af0b0 con 0x7f96980af4d0 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969d2a4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f96980af4d0 0x7f96980b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969caa3700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f969815e080 0x7f969806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37944/0 (socket says 172.21.15.97:37944) 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969caa3700 1 -- 172.21.15.97:0/798658401 learned_addr learned my addr 172.21.15.97:0/798658401 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969d2a4700 1 -- 172.21.15.97:0/798658401 >> v1:172.21.15.183:6789/0 conn(0x7f969806c710 legacy=0x7f969806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969d2a4700 1 -- 172.21.15.97:0/798658401 >> v1:172.21.15.97:6789/0 conn(0x7f969815e080 legacy=0x7f969806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.044+0000 7f969d2a4700 1 -- 172.21.15.97:0/798658401 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980ae9f0 con 0x7f96980af4d0 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f969d2a4700 1 --2- 172.21.15.97:0/798658401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96980af4d0 0x7f96980b5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9690009d90 tx=0x7f969000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef0f1497d1f1f9b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f96977fe700 1 -- 172.21.15.97:0/798658401 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000e070 con 0x7f96980af4d0 2023-11-24T06:09:07.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f96977fe700 1 -- 172.21.15.97:0/798658401 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969000bb90 con 0x7f96980af4d0 2023-11-24T06:09:07.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f96977fe700 1 -- 172.21.15.97:0/798658401 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690003a20 con 0x7f96980af4d0 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f969ed07700 1 -- 172.21.15.97:0/798658401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96980af4d0 msgr2=0x7f96980b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f969ed07700 1 --2- 172.21.15.97:0/798658401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96980af4d0 0x7f96980b5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9690009d90 tx=0x7f969000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f969ed07700 1 -- 172.21.15.97:0/798658401 shutdown_connections 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f969ed07700 1 --2- 172.21.15.97:0/798658401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96980af4d0 0x7f96980b5050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f969ed07700 1 -- 172.21.15.97:0/798658401 >> 172.21.15.97:0/798658401 conn(0x7f9698071b90 msgr2=0x7f9698071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f969ed07700 1 -- 172.21.15.97:0/798658401 shutdown_connections 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.045+0000 7f969ed07700 1 -- 172.21.15.97:0/798658401 wait complete. 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.047+0000 7f969ed07700 1 Processor -- start 2023-11-24T06:09:07.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.047+0000 7f969ed07700 1 -- start start 2023-11-24T06:09:07.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969ed07700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969806c710 0x7f9698153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969ed07700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96980af4d0 0x7f9698160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969ed07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f969815e080 0x7f9698152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969ed07700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9698153030 con 0x7f969815e080 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969ed07700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980cd620 con 0x7f96980af4d0 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969ed07700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980cd900 con 0x7f969806c710 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f969815e080 0x7f9698152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f969815e080 0x7f9698152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45500/0 (socket says 172.21.15.97:45500) 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 -- 172.21.15.97:0/2696919327 learned_addr learned my addr 172.21.15.97:0/2696919327 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f9697fff700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96980af4d0 0x7f9698160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969caa3700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969806c710 0x7f9698153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 -- 172.21.15.97:0/2696919327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969806c710 msgr2=0x7f9698153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969806c710 0x7f9698153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 -- 172.21.15.97:0/2696919327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96980af4d0 msgr2=0x7f9698160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96980af4d0 0x7f9698160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 -- 172.21.15.97:0/2696919327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009a70 con 0x7f969815e080 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.048+0000 7f969d2a4700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f969815e080 0x7f9698152af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f969809f040 tx=0x7f9690004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.049+0000 7f96957fa700 1 -- 172.21.15.97:0/2696919327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690012630 con 0x7f969815e080 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.049+0000 7f96957fa700 1 -- 172.21.15.97:0/2696919327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96900127b0 con 0x7f969815e080 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.049+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96980cdbe0 con 0x7f969815e080 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.049+0000 7f96957fa700 1 -- 172.21.15.97:0/2696919327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690012b60 con 0x7f969815e080 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.049+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96980ce1a0 con 0x7f969815e080 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.050+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96980753f0 con 0x7f969815e080 2023-11-24T06:09:07.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.050+0000 7f96957fa700 1 -- 172.21.15.97:0/2696919327 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9690012cc0 con 0x7f969815e080 2023-11-24T06:09:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.050+0000 7f96957fa700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f967c06aed0 0x7f967c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.050+0000 7f96957fa700 1 -- 172.21.15.97:0/2696919327 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f969008be90 con 0x7f969815e080 2023-11-24T06:09:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.050+0000 7f969caa3700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f967c06aed0 0x7f967c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.051+0000 7f969caa3700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f967c06aed0 0x7f967c06d390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9698156c30 tx=0x7f968c007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.055+0000 7f96957fa700 1 -- 172.21.15.97:0/2696919327 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f969005ab10 con 0x7f969815e080 2023-11-24T06:09:07.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.241+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96980b05a0 con 0x7f967c06aed0 2023-11-24T06:09:07.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.248+0000 7f96957fa700 1 -- 172.21.15.97:0/2696919327 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264494 (secure 0 0 0) 0x7f968c004b90 con 0x7f967c06aed0 2023-11-24T06:09:07.254 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:07.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f967c06aed0 msgr2=0x7f967c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f967c06aed0 0x7f967c06d390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9698156c30 tx=0x7f968c007f10 comp rx=0 tx=0).stop 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f969815e080 msgr2=0x7f9698152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f969815e080 0x7f9698152af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f969809f040 tx=0x7f9690004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 shutdown_connections 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f969815e080 0x7f9698152af0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f967c06aed0 0x7f967c06d390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f969806c710 0x7f9698153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 --2- 172.21.15.97:0/2696919327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f96980af4d0 0x7f9698160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.251+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 >> 172.21.15.97:0/2696919327 conn(0x7f9698071b90 msgr2=0x7f96980b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.252+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 shutdown_connections 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.252+0000 7f969ed07700 1 -- 172.21.15.97:0/2696919327 wait complete. 2023-11-24T06:09:07.257 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:09:07.266 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:09:06.326863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219040,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755680,"statfs":{"total":773094113280,"available":772869816320,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004174"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830864+0000","last_change":"2023-11-24T06:08:58.542671+0000","last_active":"2023-11-24T06:08:59.830864+0000","last_peered":"2023-11-24T06:08:59.830864+0000","last_clean":"2023-11-24T06:08:59.830864+0000","last_became_active":"2023-11-24T06:08:58.541937+0000","last_became_peered":"2023-11-24T06:08:58.541937+0000","last_unstale":"2023-11-24T06:08:59.830864+0000","last_undegraded":"2023-11-24T06:08:59.830864+0000","last_fullsized":"2023-11-24T06:08:59.830864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:30.040948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516727+0000","last_change":"2023-11-24T06:08:58.522085+0000","last_active":"2023-11-24T06:08:59.516727+0000","last_peered":"2023-11-24T06:08:59.516727+0000","last_clean":"2023-11-24T06:08:59.516727+0000","last_became_active":"2023-11-24T06:08:58.521116+0000","last_became_peered":"2023-11-24T06:08:58.521116+0000","last_unstale":"2023-11-24T06:08:59.516727+0000","last_undegraded":"2023-11-24T06:08:59.516727+0000","last_fullsized":"2023-11-24T06:08:59.516727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:10.129141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830976+0000","last_change":"2023-11-24T06:08:58.515201+0000","last_active":"2023-11-24T06:08:59.830976+0000","last_peered":"2023-11-24T06:08:59.830976+0000","last_clean":"2023-11-24T06:08:59.830976+0000","last_became_active":"2023-11-24T06:08:58.514911+0000","last_became_peered":"2023-11-24T06:08:58.514911+0000","last_unstale":"2023-11-24T06:08:59.830976+0000","last_undegraded":"2023-11-24T06:08:59.830976+0000","last_fullsized":"2023-11-24T06:08:59.830976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518651+0000","last_change":"2023-11-24T06:08:58.520988+0000","last_active":"2023-11-24T06:08:59.518651+0000","last_peered":"2023-11-24T06:08:59.518651+0000","last_clean":"2023-11-24T06:08:59.518651+0000","last_became_active":"2023-11-24T06:08:58.520029+0000","last_became_peered":"2023-11-24T06:08:58.520029+0000","last_unstale":"2023-11-24T06:08:59.518651+0000","last_undegraded":"2023-11-24T06:08:59.518651+0000","last_fullsized":"2023-11-24T06:08:59.518651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:13.221261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830522+0000","last_change":"2023-11-24T06:08:58.535989+0000","last_active":"2023-11-24T06:08:59.830522+0000","last_peered":"2023-11-24T06:08:59.830522+0000","last_clean":"2023-11-24T06:08:59.830522+0000","last_became_active":"2023-11-24T06:08:58.535697+0000","last_became_peered":"2023-11-24T06:08:58.535697+0000","last_unstale":"2023-11-24T06:08:59.830522+0000","last_undegraded":"2023-11-24T06:08:59.830522+0000","last_fullsized":"2023-11-24T06:08:59.830522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:27:52.505428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517494+0000","last_change":"2023-11-24T06:08:58.523237+0000","last_active":"2023-11-24T06:08:59.517494+0000","last_peered":"2023-11-24T06:08:59.517494+0000","last_clean":"2023-11-24T06:08:59.517494+0000","last_became_active":"2023-11-24T06:08:58.522368+0000","last_became_peered":"2023-11-24T06:08:58.522368+0000","last_unstale":"2023-11-24T06:08:59.517494+0000","last_undegraded":"2023-11-24T06:08:59.517494+0000","last_fullsized":"2023-11-24T06:08:59.517494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:32.177428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829818+0000","last_change":"2023-11-24T06:08:58.533834+0000","last_active":"2023-11-24T06:08:59.829818+0000","last_peered":"2023-11-24T06:08:59.829818+0000","last_clean":"2023-11-24T06:08:59.829818+0000","last_became_active":"2023-11-24T06:08:58.533539+0000","last_became_peered":"2023-11-24T06:08:58.533539+0000","last_unstale":"2023-11-24T06:08:59.829818+0000","last_undegraded":"2023-11-24T06:08:59.829818+0000","last_fullsized":"2023-11-24T06:08:59.829818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:10.129141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829824+0000","last_change":"2023-11-24T06:08:58.531528+0000","last_active":"2023-11-24T06:08:59.829824+0000","last_peered":"2023-11-24T06:08:59.829824+0000","last_clean":"2023-11-24T06:08:59.829824+0000","last_became_active":"2023-11-24T06:08:58.531144+0000","last_became_peered":"2023-11-24T06:08:58.531144+0000","last_unstale":"2023-11-24T06:08:59.829824+0000","last_undegraded":"2023-11-24T06:08:59.829824+0000","last_fullsized":"2023-11-24T06:08:59.829824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:01.910093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830820+0000","last_change":"2023-11-24T06:08:58.542860+0000","last_active":"2023-11-24T06:08:59.830820+0000","last_peered":"2023-11-24T06:08:59.830820+0000","last_clean":"2023-11-24T06:08:59.830820+0000","last_became_active":"2023-11-24T06:08:58.542347+0000","last_became_peered":"2023-11-24T06:08:58.542347+0000","last_unstale":"2023-11-24T06:08:59.830820+0000","last_undegraded":"2023-11-24T06:08:59.830820+0000","last_fullsized":"2023-11-24T06:08:59.830820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:01.789347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830098+0000","last_change":"2023-11-24T06:08:58.529308+0000","last_active":"2023-11-24T06:08:59.830098+0000","last_peered":"2023-11-24T06:08:59.830098+0000","last_clean":"2023-11-24T06:08:59.830098+0000","last_became_active":"2023-11-24T06:08:58.528986+0000","last_became_peered":"2023-11-24T06:08:58.528986+0000","last_unstale":"2023-11-24T06:08:59.830098+0000","last_undegraded":"2023-11-24T06:08:59.830098+0000","last_fullsized":"2023-11-24T06:08:59.830098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829848+0000","last_change":"2023-11-24T06:08:58.530560+0000","last_active":"2023-11-24T06:08:59.829848+0000","last_peered":"2023-11-24T06:08:59.829848+0000","last_clean":"2023-11-24T06:08:59.829848+0000","last_became_active":"2023-11-24T06:08:58.530299+0000","last_became_peered":"2023-11-24T06:08:58.530299+0000","last_unstale":"2023-11-24T06:08:59.829848+0000","last_undegraded":"2023-11-24T06:08:59.829848+0000","last_fullsized":"2023-11-24T06:08:59.829848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517877+0000","last_change":"2023-11-24T06:08:58.529565+0000","last_active":"2023-11-24T06:08:59.517877+0000","last_peered":"2023-11-24T06:08:59.517877+0000","last_clean":"2023-11-24T06:08:59.517877+0000","last_became_active":"2023-11-24T06:08:58.529170+0000","last_became_peered":"2023-11-24T06:08:58.529170+0000","last_unstale":"2023-11-24T06:08:59.517877+0000","last_undegraded":"2023-11-24T06:08:59.517877+0000","last_fullsized":"2023-11-24T06:08:59.517877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829628+0000","last_change":"2023-11-24T06:08:58.521255+0000","last_active":"2023-11-24T06:08:59.829628+0000","last_peered":"2023-11-24T06:08:59.829628+0000","last_clean":"2023-11-24T06:08:59.829628+0000","last_became_active":"2023-11-24T06:08:58.520950+0000","last_became_peered":"2023-11-24T06:08:58.520950+0000","last_unstale":"2023-11-24T06:08:59.829628+0000","last_undegraded":"2023-11-24T06:08:59.829628+0000","last_fullsized":"2023-11-24T06:08:59.829628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:01.809823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829844+0000","last_change":"2023-11-24T06:08:58.525412+0000","last_active":"2023-11-24T06:08:59.829844+0000","last_peered":"2023-11-24T06:08:59.829844+0000","last_clean":"2023-11-24T06:08:59.829844+0000","last_became_active":"2023-11-24T06:08:58.525080+0000","last_became_peered":"2023-11-24T06:08:58.525080+0000","last_unstale":"2023-11-24T06:08:59.829844+0000","last_undegraded":"2023-11-24T06:08:59.829844+0000","last_fullsized":"2023-11-24T06:08:59.829844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:40.002717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516190+0000","last_change":"2023-11-24T06:08:58.529101+0000","last_active":"2023-11-24T06:08:59.516190+0000","last_peered":"2023-11-24T06:08:59.516190+0000","last_clean":"2023-11-24T06:08:59.516190+0000","last_became_active":"2023-11-24T06:08:58.528814+0000","last_became_peered":"2023-11-24T06:08:58.528814+0000","last_unstale":"2023-11-24T06:08:59.516190+0000","last_undegraded":"2023-11-24T06:08:59.516190+0000","last_fullsized":"2023-11-24T06:08:59.516190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516855+0000","last_change":"2023-11-24T06:08:58.523287+0000","last_active":"2023-11-24T06:08:59.516855+0000","last_peered":"2023-11-24T06:08:59.516855+0000","last_clean":"2023-11-24T06:08:59.516855+0000","last_became_active":"2023-11-24T06:08:58.523033+0000","last_became_peered":"2023-11-24T06:08:58.523033+0000","last_unstale":"2023-11-24T06:08:59.516855+0000","last_undegraded":"2023-11-24T06:08:59.516855+0000","last_fullsized":"2023-11-24T06:08:59.516855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:10.129141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517860+0000","last_change":"2023-11-24T06:08:58.543046+0000","last_active":"2023-11-24T06:08:59.517860+0000","last_peered":"2023-11-24T06:08:59.517860+0000","last_clean":"2023-11-24T06:08:59.517860+0000","last_became_active":"2023-11-24T06:08:58.542703+0000","last_became_peered":"2023-11-24T06:08:58.542703+0000","last_unstale":"2023-11-24T06:08:59.517860+0000","last_undegraded":"2023-11-24T06:08:59.517860+0000","last_fullsized":"2023-11-24T06:08:59.517860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:09.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829953+0000","last_change":"2023-11-24T06:08:58.521888+0000","last_active":"2023-11-24T06:08:59.829953+0000","last_peered":"2023-11-24T06:08:59.829953+0000","last_clean":"2023-11-24T06:08:59.829953+0000","last_became_active":"2023-11-24T06:08:58.521599+0000","last_became_peered":"2023-11-24T06:08:58.521599+0000","last_unstale":"2023-11-24T06:08:59.829953+0000","last_undegraded":"2023-11-24T06:08:59.829953+0000","last_fullsized":"2023-11-24T06:08:59.829953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830479+0000","last_change":"2023-11-24T06:08:58.536029+0000","last_active":"2023-11-24T06:08:59.830479+0000","last_peered":"2023-11-24T06:08:59.830479+0000","last_clean":"2023-11-24T06:08:59.830479+0000","last_became_active":"2023-11-24T06:08:58.535076+0000","last_became_peered":"2023-11-24T06:08:58.535076+0000","last_unstale":"2023-11-24T06:08:59.830479+0000","last_undegraded":"2023-11-24T06:08:59.830479+0000","last_fullsized":"2023-11-24T06:08:59.830479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:09.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.344516+0000","last_change":"2023-11-24T06:08:57.529523+0000","last_active":"2023-11-24T06:08:59.344516+0000","last_peered":"2023-11-24T06:08:59.344516+0000","last_clean":"2023-11-24T06:08:59.344516+0000","last_became_active":"2023-11-24T06:08:57.529207+0000","last_became_peered":"2023-11-24T06:08:57.529207+0000","last_unstale":"2023-11-24T06:08:59.344516+0000","last_undegraded":"2023-11-24T06:08:59.344516+0000","last_fullsized":"2023-11-24T06:08:59.344516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:52.525711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830189+0000","last_change":"2023-11-24T06:08:58.523242+0000","last_active":"2023-11-24T06:08:59.830189+0000","last_peered":"2023-11-24T06:08:59.830189+0000","last_clean":"2023-11-24T06:08:59.830189+0000","last_became_active":"2023-11-24T06:08:58.522951+0000","last_became_peered":"2023-11-24T06:08:58.522951+0000","last_unstale":"2023-11-24T06:08:59.830189+0000","last_undegraded":"2023-11-24T06:08:59.830189+0000","last_fullsized":"2023-11-24T06:08:59.830189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343204+0000","last_change":"2023-11-24T06:08:57.523196+0000","last_active":"2023-11-24T06:08:59.343204+0000","last_peered":"2023-11-24T06:08:59.343204+0000","last_clean":"2023-11-24T06:08:59.343204+0000","last_became_active":"2023-11-24T06:08:57.522871+0000","last_became_peered":"2023-11-24T06:08:57.522871+0000","last_unstale":"2023-11-24T06:08:59.343204+0000","last_undegraded":"2023-11-24T06:08:59.343204+0000","last_fullsized":"2023-11-24T06:08:59.343204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:22.630593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.831052+0000","last_change":"2023-11-24T06:08:58.530832+0000","last_active":"2023-11-24T06:08:59.831052+0000","last_peered":"2023-11-24T06:08:59.831052+0000","last_clean":"2023-11-24T06:08:59.831052+0000","last_became_active":"2023-11-24T06:08:58.530383+0000","last_became_peered":"2023-11-24T06:08:58.530383+0000","last_unstale":"2023-11-24T06:08:59.831052+0000","last_undegraded":"2023-11-24T06:08:59.831052+0000","last_fullsized":"2023-11-24T06:08:59.831052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339081+0000","last_change":"2023-11-24T06:08:57.528776+0000","last_active":"2023-11-24T06:08:59.339081+0000","last_peered":"2023-11-24T06:08:59.339081+0000","last_clean":"2023-11-24T06:08:59.339081+0000","last_became_active":"2023-11-24T06:08:57.528480+0000","last_became_peered":"2023-11-24T06:08:57.528480+0000","last_unstale":"2023-11-24T06:08:59.339081+0000","last_undegraded":"2023-11-24T06:08:59.339081+0000","last_fullsized":"2023-11-24T06:08:59.339081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518377+0000","last_change":"2023-11-24T06:08:58.524300+0000","last_active":"2023-11-24T06:08:59.518377+0000","last_peered":"2023-11-24T06:08:59.518377+0000","last_clean":"2023-11-24T06:08:59.518377+0000","last_became_active":"2023-11-24T06:08:58.522378+0000","last_became_peered":"2023-11-24T06:08:58.522378+0000","last_unstale":"2023-11-24T06:08:59.518377+0000","last_undegraded":"2023-11-24T06:08:59.518377+0000","last_fullsized":"2023-11-24T06:08:59.518377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:27:52.505428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339013+0000","last_change":"2023-11-24T06:08:57.526272+0000","last_active":"2023-11-24T06:08:59.339013+0000","last_peered":"2023-11-24T06:08:59.339013+0000","last_clean":"2023-11-24T06:08:59.339013+0000","last_became_active":"2023-11-24T06:08:57.525998+0000","last_became_peered":"2023-11-24T06:08:57.525998+0000","last_unstale":"2023-11-24T06:08:59.339013+0000","last_undegraded":"2023-11-24T06:08:59.339013+0000","last_fullsized":"2023-11-24T06:08:59.339013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518271+0000","last_change":"2023-11-24T06:08:58.523547+0000","last_active":"2023-11-24T06:08:59.518271+0000","last_peered":"2023-11-24T06:08:59.518271+0000","last_clean":"2023-11-24T06:08:59.518271+0000","last_became_active":"2023-11-24T06:08:58.522784+0000","last_became_peered":"2023-11-24T06:08:58.522784+0000","last_unstale":"2023-11-24T06:08:59.518271+0000","last_undegraded":"2023-11-24T06:08:59.518271+0000","last_fullsized":"2023-11-24T06:08:59.518271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:33.338886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343116+0000","last_change":"2023-11-24T06:08:57.528824+0000","last_active":"2023-11-24T06:08:59.343116+0000","last_peered":"2023-11-24T06:08:59.343116+0000","last_clean":"2023-11-24T06:08:59.343116+0000","last_became_active":"2023-11-24T06:08:57.528517+0000","last_became_peered":"2023-11-24T06:08:57.528517+0000","last_unstale":"2023-11-24T06:08:59.343116+0000","last_undegraded":"2023-11-24T06:08:59.343116+0000","last_fullsized":"2023-11-24T06:08:59.343116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517286+0000","last_change":"2023-11-24T06:08:58.523181+0000","last_active":"2023-11-24T06:08:59.517286+0000","last_peered":"2023-11-24T06:08:59.517286+0000","last_clean":"2023-11-24T06:08:59.517286+0000","last_became_active":"2023-11-24T06:08:58.522917+0000","last_became_peered":"2023-11-24T06:08:58.522917+0000","last_unstale":"2023-11-24T06:08:59.517286+0000","last_undegraded":"2023-11-24T06:08:59.517286+0000","last_fullsized":"2023-11-24T06:08:59.517286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337315+0000","last_change":"2023-11-24T06:08:57.524473+0000","last_active":"2023-11-24T06:08:59.337315+0000","last_peered":"2023-11-24T06:08:59.337315+0000","last_clean":"2023-11-24T06:08:59.337315+0000","last_became_active":"2023-11-24T06:08:57.523947+0000","last_became_peered":"2023-11-24T06:08:57.523947+0000","last_unstale":"2023-11-24T06:08:59.337315+0000","last_undegraded":"2023-11-24T06:08:59.337315+0000","last_fullsized":"2023-11-24T06:08:59.337315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830977+0000","last_change":"2023-11-24T06:08:58.526170+0000","last_active":"2023-11-24T06:08:59.830977+0000","last_peered":"2023-11-24T06:08:59.830977+0000","last_clean":"2023-11-24T06:08:59.830977+0000","last_became_active":"2023-11-24T06:08:58.525874+0000","last_became_peered":"2023-11-24T06:08:58.525874+0000","last_unstale":"2023-11-24T06:08:59.830977+0000","last_undegraded":"2023-11-24T06:08:59.830977+0000","last_fullsized":"2023-11-24T06:08:59.830977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:32.599128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343057+0000","last_change":"2023-11-24T06:08:57.520895+0000","last_active":"2023-11-24T06:08:59.343057+0000","last_peered":"2023-11-24T06:08:59.343057+0000","last_clean":"2023-11-24T06:08:59.343057+0000","last_became_active":"2023-11-24T06:08:57.520367+0000","last_became_peered":"2023-11-24T06:08:57.520367+0000","last_unstale":"2023-11-24T06:08:59.343057+0000","last_undegraded":"2023-11-24T06:08:59.343057+0000","last_fullsized":"2023-11-24T06:08:59.343057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:55.763227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830105+0000","last_change":"2023-11-24T06:08:58.519434+0000","last_active":"2023-11-24T06:08:59.830105+0000","last_peered":"2023-11-24T06:08:59.830105+0000","last_clean":"2023-11-24T06:08:59.830105+0000","last_became_active":"2023-11-24T06:08:58.519148+0000","last_became_peered":"2023-11-24T06:08:58.519148+0000","last_unstale":"2023-11-24T06:08:59.830105+0000","last_undegraded":"2023-11-24T06:08:59.830105+0000","last_fullsized":"2023-11-24T06:08:59.830105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339366+0000","last_change":"2023-11-24T06:08:57.525692+0000","last_active":"2023-11-24T06:08:59.339366+0000","last_peered":"2023-11-24T06:08:59.339366+0000","last_clean":"2023-11-24T06:08:59.339366+0000","last_became_active":"2023-11-24T06:08:57.525451+0000","last_became_peered":"2023-11-24T06:08:57.525451+0000","last_unstale":"2023-11-24T06:08:59.339366+0000","last_undegraded":"2023-11-24T06:08:59.339366+0000","last_fullsized":"2023-11-24T06:08:59.339366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516297+0000","last_change":"2023-11-24T06:08:58.521741+0000","last_active":"2023-11-24T06:08:59.516297+0000","last_peered":"2023-11-24T06:08:59.516297+0000","last_clean":"2023-11-24T06:08:59.516297+0000","last_became_active":"2023-11-24T06:08:58.521275+0000","last_became_peered":"2023-11-24T06:08:58.521275+0000","last_unstale":"2023-11-24T06:08:59.516297+0000","last_undegraded":"2023-11-24T06:08:59.516297+0000","last_fullsized":"2023-11-24T06:08:59.516297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337411+0000","last_change":"2023-11-24T06:08:57.526466+0000","last_active":"2023-11-24T06:08:59.337411+0000","last_peered":"2023-11-24T06:08:59.337411+0000","last_clean":"2023-11-24T06:08:59.337411+0000","last_became_active":"2023-11-24T06:08:57.526174+0000","last_became_peered":"2023-11-24T06:08:57.526174+0000","last_unstale":"2023-11-24T06:08:59.337411+0000","last_undegraded":"2023-11-24T06:08:59.337411+0000","last_fullsized":"2023-11-24T06:08:59.337411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517356+0000","last_change":"2023-11-24T06:08:58.541089+0000","last_active":"2023-11-24T06:08:59.517356+0000","last_peered":"2023-11-24T06:08:59.517356+0000","last_clean":"2023-11-24T06:08:59.517356+0000","last_became_active":"2023-11-24T06:08:58.540792+0000","last_became_peered":"2023-11-24T06:08:58.540792+0000","last_unstale":"2023-11-24T06:08:59.517356+0000","last_undegraded":"2023-11-24T06:08:59.517356+0000","last_fullsized":"2023-11-24T06:08:59.517356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:51.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339541+0000","last_change":"2023-11-24T06:08:57.525181+0000","last_active":"2023-11-24T06:08:59.339541+0000","last_peered":"2023-11-24T06:08:59.339541+0000","last_clean":"2023-11-24T06:08:59.339541+0000","last_became_active":"2023-11-24T06:08:57.524906+0000","last_became_peered":"2023-11-24T06:08:57.524906+0000","last_unstale":"2023-11-24T06:08:59.339541+0000","last_undegraded":"2023-11-24T06:08:59.339541+0000","last_fullsized":"2023-11-24T06:08:59.339541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.831159+0000","last_change":"2023-11-24T06:08:58.530664+0000","last_active":"2023-11-24T06:08:59.831159+0000","last_peered":"2023-11-24T06:08:59.831159+0000","last_clean":"2023-11-24T06:08:59.831159+0000","last_became_active":"2023-11-24T06:08:58.530068+0000","last_became_peered":"2023-11-24T06:08:58.530068+0000","last_unstale":"2023-11-24T06:08:59.831159+0000","last_undegraded":"2023-11-24T06:08:59.831159+0000","last_fullsized":"2023-11-24T06:08:59.831159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340262+0000","last_change":"2023-11-24T06:08:57.525584+0000","last_active":"2023-11-24T06:08:59.340262+0000","last_peered":"2023-11-24T06:08:59.340262+0000","last_clean":"2023-11-24T06:08:59.340262+0000","last_became_active":"2023-11-24T06:08:57.525340+0000","last_became_peered":"2023-11-24T06:08:57.525340+0000","last_unstale":"2023-11-24T06:08:59.340262+0000","last_undegraded":"2023-11-24T06:08:59.340262+0000","last_fullsized":"2023-11-24T06:08:59.340262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517718+0000","last_change":"2023-11-24T06:08:58.537817+0000","last_active":"2023-11-24T06:08:59.517718+0000","last_peered":"2023-11-24T06:08:59.517718+0000","last_clean":"2023-11-24T06:08:59.517718+0000","last_became_active":"2023-11-24T06:08:58.537604+0000","last_became_peered":"2023-11-24T06:08:58.537604+0000","last_unstale":"2023-11-24T06:08:59.517718+0000","last_undegraded":"2023-11-24T06:08:59.517718+0000","last_fullsized":"2023-11-24T06:08:59.517718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337329+0000","last_change":"2023-11-24T06:08:57.520656+0000","last_active":"2023-11-24T06:08:59.337329+0000","last_peered":"2023-11-24T06:08:59.337329+0000","last_clean":"2023-11-24T06:08:59.337329+0000","last_became_active":"2023-11-24T06:08:57.520398+0000","last_became_peered":"2023-11-24T06:08:57.520398+0000","last_unstale":"2023-11-24T06:08:59.337329+0000","last_undegraded":"2023-11-24T06:08:59.337329+0000","last_fullsized":"2023-11-24T06:08:59.337329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830513+0000","last_change":"2023-11-24T06:08:58.533919+0000","last_active":"2023-11-24T06:08:59.830513+0000","last_peered":"2023-11-24T06:08:59.830513+0000","last_clean":"2023-11-24T06:08:59.830513+0000","last_became_active":"2023-11-24T06:08:58.533565+0000","last_became_peered":"2023-11-24T06:08:58.533565+0000","last_unstale":"2023-11-24T06:08:59.830513+0000","last_undegraded":"2023-11-24T06:08:59.830513+0000","last_fullsized":"2023-11-24T06:08:59.830513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:33.338886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.344421+0000","last_change":"2023-11-24T06:08:57.523298+0000","last_active":"2023-11-24T06:08:59.344421+0000","last_peered":"2023-11-24T06:08:59.344421+0000","last_clean":"2023-11-24T06:08:59.344421+0000","last_became_active":"2023-11-24T06:08:57.523000+0000","last_became_peered":"2023-11-24T06:08:57.523000+0000","last_unstale":"2023-11-24T06:08:59.344421+0000","last_undegraded":"2023-11-24T06:08:59.344421+0000","last_fullsized":"2023-11-24T06:08:59.344421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829711+0000","last_change":"2023-11-24T06:08:58.539907+0000","last_active":"2023-11-24T06:08:59.829711+0000","last_peered":"2023-11-24T06:08:59.829711+0000","last_clean":"2023-11-24T06:08:59.829711+0000","last_became_active":"2023-11-24T06:08:58.539663+0000","last_became_peered":"2023-11-24T06:08:58.539663+0000","last_unstale":"2023-11-24T06:08:59.829711+0000","last_undegraded":"2023-11-24T06:08:59.829711+0000","last_fullsized":"2023-11-24T06:08:59.829711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:26.004865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339449+0000","last_change":"2023-11-24T06:08:57.523074+0000","last_active":"2023-11-24T06:08:59.339449+0000","last_peered":"2023-11-24T06:08:59.339449+0000","last_clean":"2023-11-24T06:08:59.339449+0000","last_became_active":"2023-11-24T06:08:57.522527+0000","last_became_peered":"2023-11-24T06:08:57.522527+0000","last_unstale":"2023-11-24T06:08:59.339449+0000","last_undegraded":"2023-11-24T06:08:59.339449+0000","last_fullsized":"2023-11-24T06:08:59.339449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:31.591882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830543+0000","last_change":"2023-11-24T06:08:58.538966+0000","last_active":"2023-11-24T06:08:59.830543+0000","last_peered":"2023-11-24T06:08:59.830543+0000","last_clean":"2023-11-24T06:08:59.830543+0000","last_became_active":"2023-11-24T06:08:58.538575+0000","last_became_peered":"2023-11-24T06:08:58.538575+0000","last_unstale":"2023-11-24T06:08:59.830543+0000","last_undegraded":"2023-11-24T06:08:59.830543+0000","last_fullsized":"2023-11-24T06:08:59.830543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:45.776694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337286+0000","last_change":"2023-11-24T06:08:57.524582+0000","last_active":"2023-11-24T06:08:59.337286+0000","last_peered":"2023-11-24T06:08:59.337286+0000","last_clean":"2023-11-24T06:08:59.337286+0000","last_became_active":"2023-11-24T06:08:57.524351+0000","last_became_peered":"2023-11-24T06:08:57.524351+0000","last_unstale":"2023-11-24T06:08:59.337286+0000","last_undegraded":"2023-11-24T06:08:59.337286+0000","last_fullsized":"2023-11-24T06:08:59.337286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:22.630593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829397+0000","last_change":"2023-11-24T06:08:58.528140+0000","last_active":"2023-11-24T06:08:59.829397+0000","last_peered":"2023-11-24T06:08:59.829397+0000","last_clean":"2023-11-24T06:08:59.829397+0000","last_became_active":"2023-11-24T06:08:58.527685+0000","last_became_peered":"2023-11-24T06:08:58.527685+0000","last_unstale":"2023-11-24T06:08:59.829397+0000","last_undegraded":"2023-11-24T06:08:59.829397+0000","last_fullsized":"2023-11-24T06:08:59.829397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:01.910093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343108+0000","last_change":"2023-11-24T06:08:57.519672+0000","last_active":"2023-11-24T06:08:59.343108+0000","last_peered":"2023-11-24T06:08:59.343108+0000","last_clean":"2023-11-24T06:08:59.343108+0000","last_became_active":"2023-11-24T06:08:57.519379+0000","last_became_peered":"2023-11-24T06:08:57.519379+0000","last_unstale":"2023-11-24T06:08:59.343108+0000","last_undegraded":"2023-11-24T06:08:59.343108+0000","last_fullsized":"2023-11-24T06:08:59.343108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:43.426568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830278+0000","last_change":"2023-11-24T06:08:58.531525+0000","last_active":"2023-11-24T06:08:59.830278+0000","last_peered":"2023-11-24T06:08:59.830278+0000","last_clean":"2023-11-24T06:08:59.830278+0000","last_became_active":"2023-11-24T06:08:58.529801+0000","last_became_peered":"2023-11-24T06:08:58.529801+0000","last_unstale":"2023-11-24T06:08:59.830278+0000","last_undegraded":"2023-11-24T06:08:59.830278+0000","last_fullsized":"2023-11-24T06:08:59.830278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:32.177428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.336436+0000","last_change":"2023-11-24T06:08:57.523021+0000","last_active":"2023-11-24T06:08:59.336436+0000","last_peered":"2023-11-24T06:08:59.336436+0000","last_clean":"2023-11-24T06:08:59.336436+0000","last_became_active":"2023-11-24T06:08:57.522474+0000","last_became_peered":"2023-11-24T06:08:57.522474+0000","last_unstale":"2023-11-24T06:08:59.336436+0000","last_undegraded":"2023-11-24T06:08:59.336436+0000","last_fullsized":"2023-11-24T06:08:59.336436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:43.891317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830650+0000","last_change":"2023-11-24T06:08:58.535871+0000","last_active":"2023-11-24T06:08:59.830650+0000","last_peered":"2023-11-24T06:08:59.830650+0000","last_clean":"2023-11-24T06:08:59.830650+0000","last_became_active":"2023-11-24T06:08:58.535563+0000","last_became_peered":"2023-11-24T06:08:58.535563+0000","last_unstale":"2023-11-24T06:08:59.830650+0000","last_undegraded":"2023-11-24T06:08:59.830650+0000","last_fullsized":"2023-11-24T06:08:59.830650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:55.981203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340214+0000","last_change":"2023-11-24T06:08:57.519605+0000","last_active":"2023-11-24T06:08:59.340214+0000","last_peered":"2023-11-24T06:08:59.340214+0000","last_clean":"2023-11-24T06:08:59.340214+0000","last_became_active":"2023-11-24T06:08:57.519308+0000","last_became_peered":"2023-11-24T06:08:57.519308+0000","last_unstale":"2023-11-24T06:08:59.340214+0000","last_undegraded":"2023-11-24T06:08:59.340214+0000","last_fullsized":"2023-11-24T06:08:59.340214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518512+0000","last_change":"2023-11-24T06:08:58.523994+0000","last_active":"2023-11-24T06:08:59.518512+0000","last_peered":"2023-11-24T06:08:59.518512+0000","last_clean":"2023-11-24T06:08:59.518512+0000","last_became_active":"2023-11-24T06:08:58.523750+0000","last_became_peered":"2023-11-24T06:08:58.523750+0000","last_unstale":"2023-11-24T06:08:59.518512+0000","last_undegraded":"2023-11-24T06:08:59.518512+0000","last_fullsized":"2023-11-24T06:08:59.518512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:45.776694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339437+0000","last_change":"2023-11-24T06:08:57.525492+0000","last_active":"2023-11-24T06:08:59.339437+0000","last_peered":"2023-11-24T06:08:59.339437+0000","last_clean":"2023-11-24T06:08:59.339437+0000","last_became_active":"2023-11-24T06:08:57.525231+0000","last_became_peered":"2023-11-24T06:08:57.525231+0000","last_unstale":"2023-11-24T06:08:59.339437+0000","last_undegraded":"2023-11-24T06:08:59.339437+0000","last_fullsized":"2023-11-24T06:08:59.339437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:35.934471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518156+0000","last_change":"2023-11-24T06:08:58.525233+0000","last_active":"2023-11-24T06:08:59.518156+0000","last_peered":"2023-11-24T06:08:59.518156+0000","last_clean":"2023-11-24T06:08:59.518156+0000","last_became_active":"2023-11-24T06:08:58.524919+0000","last_became_peered":"2023-11-24T06:08:58.524919+0000","last_unstale":"2023-11-24T06:08:59.518156+0000","last_undegraded":"2023-11-24T06:08:59.518156+0000","last_fullsized":"2023-11-24T06:08:59.518156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516155+0000","last_change":"2023-11-24T06:08:58.521720+0000","last_active":"2023-11-24T06:08:59.516155+0000","last_peered":"2023-11-24T06:08:59.516155+0000","last_clean":"2023-11-24T06:08:59.516155+0000","last_became_active":"2023-11-24T06:08:58.521358+0000","last_became_peered":"2023-11-24T06:08:58.521358+0000","last_unstale":"2023-11-24T06:08:59.516155+0000","last_undegraded":"2023-11-24T06:08:59.516155+0000","last_fullsized":"2023-11-24T06:08:59.516155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343132+0000","last_change":"2023-11-24T06:08:57.521087+0000","last_active":"2023-11-24T06:08:59.343132+0000","last_peered":"2023-11-24T06:08:59.343132+0000","last_clean":"2023-11-24T06:08:59.343132+0000","last_became_active":"2023-11-24T06:08:57.520758+0000","last_became_peered":"2023-11-24T06:08:57.520758+0000","last_unstale":"2023-11-24T06:08:59.343132+0000","last_undegraded":"2023-11-24T06:08:59.343132+0000","last_fullsized":"2023-11-24T06:08:59.343132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:28.366035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829947+0000","last_change":"2023-11-24T06:08:58.531268+0000","last_active":"2023-11-24T06:08:59.829947+0000","last_peered":"2023-11-24T06:08:59.829947+0000","last_clean":"2023-11-24T06:08:59.829947+0000","last_became_active":"2023-11-24T06:08:58.530982+0000","last_became_peered":"2023-11-24T06:08:58.530982+0000","last_unstale":"2023-11-24T06:08:59.829947+0000","last_undegraded":"2023-11-24T06:08:59.829947+0000","last_fullsized":"2023-11-24T06:08:59.829947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:32.599128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339463+0000","last_change":"2023-11-24T06:08:57.519233+0000","last_active":"2023-11-24T06:08:59.339463+0000","last_peered":"2023-11-24T06:08:59.339463+0000","last_clean":"2023-11-24T06:08:59.339463+0000","last_became_active":"2023-11-24T06:08:57.518413+0000","last_became_peered":"2023-11-24T06:08:57.518413+0000","last_unstale":"2023-11-24T06:08:59.339463+0000","last_undegraded":"2023-11-24T06:08:59.339463+0000","last_fullsized":"2023-11-24T06:08:59.339463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.516224+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:08:58.516224+0000","last_peered":"2023-11-24T06:08:58.516224+0000","last_clean":"2023-11-24T06:08:58.516224+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:08:58.516224+0000","last_undegraded":"2023-11-24T06:08:58.516224+0000","last_fullsized":"2023-11-24T06:08:58.516224+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829578+0000","last_change":"2023-11-24T06:08:58.528408+0000","last_active":"2023-11-24T06:08:59.829578+0000","last_peered":"2023-11-24T06:08:59.829578+0000","last_clean":"2023-11-24T06:08:59.829578+0000","last_became_active":"2023-11-24T06:08:58.528125+0000","last_became_peered":"2023-11-24T06:08:58.528125+0000","last_unstale":"2023-11-24T06:08:59.829578+0000","last_undegraded":"2023-11-24T06:08:59.829578+0000","last_fullsized":"2023-11-24T06:08:59.829578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:18.279785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.338944+0000","last_change":"2023-11-24T06:08:57.527351+0000","last_active":"2023-11-24T06:08:59.338944+0000","last_peered":"2023-11-24T06:08:59.338944+0000","last_clean":"2023-11-24T06:08:59.338944+0000","last_became_active":"2023-11-24T06:08:57.527085+0000","last_became_peered":"2023-11-24T06:08:57.527085+0000","last_unstale":"2023-11-24T06:08:59.338944+0000","last_undegraded":"2023-11-24T06:08:59.338944+0000","last_fullsized":"2023-11-24T06:08:59.338944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339066+0000","last_change":"2023-11-24T06:08:57.528763+0000","last_active":"2023-11-24T06:08:59.339066+0000","last_peered":"2023-11-24T06:08:59.339066+0000","last_clean":"2023-11-24T06:08:59.339066+0000","last_became_active":"2023-11-24T06:08:57.528469+0000","last_became_peered":"2023-11-24T06:08:57.528469+0000","last_unstale":"2023-11-24T06:08:59.339066+0000","last_undegraded":"2023-11-24T06:08:59.339066+0000","last_fullsized":"2023-11-24T06:08:59.339066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516266+0000","last_change":"2023-11-24T06:08:58.522186+0000","last_active":"2023-11-24T06:08:59.516266+0000","last_peered":"2023-11-24T06:08:59.516266+0000","last_clean":"2023-11-24T06:08:59.516266+0000","last_became_active":"2023-11-24T06:08:58.521844+0000","last_became_peered":"2023-11-24T06:08:58.521844+0000","last_unstale":"2023-11-24T06:08:59.516266+0000","last_undegraded":"2023-11-24T06:08:59.516266+0000","last_fullsized":"2023-11-24T06:08:59.516266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516554+0000","last_change":"2023-11-24T06:08:57.519266+0000","last_active":"2023-11-24T06:08:59.516554+0000","last_peered":"2023-11-24T06:08:59.516554+0000","last_clean":"2023-11-24T06:08:59.516554+0000","last_became_active":"2023-11-24T06:08:57.518952+0000","last_became_peered":"2023-11-24T06:08:57.518952+0000","last_unstale":"2023-11-24T06:08:59.516554+0000","last_undegraded":"2023-11-24T06:08:59.516554+0000","last_fullsized":"2023-11-24T06:08:59.516554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830207+0000","last_change":"2023-11-24T06:08:58.539562+0000","last_active":"2023-11-24T06:08:59.830207+0000","last_peered":"2023-11-24T06:08:59.830207+0000","last_clean":"2023-11-24T06:08:59.830207+0000","last_became_active":"2023-11-24T06:08:58.539268+0000","last_became_peered":"2023-11-24T06:08:58.539268+0000","last_unstale":"2023-11-24T06:08:59.830207+0000","last_undegraded":"2023-11-24T06:08:59.830207+0000","last_fullsized":"2023-11-24T06:08:59.830207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:10.129141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337398+0000","last_change":"2023-11-24T06:08:57.525492+0000","last_active":"2023-11-24T06:08:59.337398+0000","last_peered":"2023-11-24T06:08:59.337398+0000","last_clean":"2023-11-24T06:08:59.337398+0000","last_became_active":"2023-11-24T06:08:57.525231+0000","last_became_peered":"2023-11-24T06:08:57.525231+0000","last_unstale":"2023-11-24T06:08:59.337398+0000","last_undegraded":"2023-11-24T06:08:59.337398+0000","last_fullsized":"2023-11-24T06:08:59.337398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.831038+0000","last_change":"2023-11-24T06:08:58.522482+0000","last_active":"2023-11-24T06:08:59.831038+0000","last_peered":"2023-11-24T06:08:59.831038+0000","last_clean":"2023-11-24T06:08:59.831038+0000","last_became_active":"2023-11-24T06:08:58.522237+0000","last_became_peered":"2023-11-24T06:08:58.522237+0000","last_unstale":"2023-11-24T06:08:59.831038+0000","last_undegraded":"2023-11-24T06:08:59.831038+0000","last_fullsized":"2023-11-24T06:08:59.831038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.338911+0000","last_change":"2023-11-24T06:08:57.525325+0000","last_active":"2023-11-24T06:08:59.338911+0000","last_peered":"2023-11-24T06:08:59.338911+0000","last_clean":"2023-11-24T06:08:59.338911+0000","last_became_active":"2023-11-24T06:08:57.525051+0000","last_became_peered":"2023-11-24T06:08:57.525051+0000","last_unstale":"2023-11-24T06:08:59.338911+0000","last_undegraded":"2023-11-24T06:08:59.338911+0000","last_fullsized":"2023-11-24T06:08:59.338911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:16.569994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517913+0000","last_change":"2023-11-24T06:08:58.534115+0000","last_active":"2023-11-24T06:08:59.517913+0000","last_peered":"2023-11-24T06:08:59.517913+0000","last_clean":"2023-11-24T06:08:59.517913+0000","last_became_active":"2023-11-24T06:08:58.525128+0000","last_became_peered":"2023-11-24T06:08:58.525128+0000","last_unstale":"2023-11-24T06:08:59.517913+0000","last_undegraded":"2023-11-24T06:08:59.517913+0000","last_fullsized":"2023-11-24T06:08:59.517913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340132+0000","last_change":"2023-11-24T06:08:57.530707+0000","last_active":"2023-11-24T06:08:59.340132+0000","last_peered":"2023-11-24T06:08:59.340132+0000","last_clean":"2023-11-24T06:08:59.340132+0000","last_became_active":"2023-11-24T06:08:57.529531+0000","last_became_peered":"2023-11-24T06:08:57.529531+0000","last_unstale":"2023-11-24T06:08:59.340132+0000","last_undegraded":"2023-11-24T06:08:59.340132+0000","last_fullsized":"2023-11-24T06:08:59.340132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829973+0000","last_change":"2023-11-24T06:08:58.523161+0000","last_active":"2023-11-24T06:08:59.829973+0000","last_peered":"2023-11-24T06:08:59.829973+0000","last_clean":"2023-11-24T06:08:59.829973+0000","last_became_active":"2023-11-24T06:08:58.522753+0000","last_became_peered":"2023-11-24T06:08:58.522753+0000","last_unstale":"2023-11-24T06:08:59.829973+0000","last_undegraded":"2023-11-24T06:08:59.829973+0000","last_fullsized":"2023-11-24T06:08:59.829973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340171+0000","last_change":"2023-11-24T06:08:57.521686+0000","last_active":"2023-11-24T06:08:59.340171+0000","last_peered":"2023-11-24T06:08:59.340171+0000","last_clean":"2023-11-24T06:08:59.340171+0000","last_became_active":"2023-11-24T06:08:57.520431+0000","last_became_peered":"2023-11-24T06:08:57.520431+0000","last_unstale":"2023-11-24T06:08:59.340171+0000","last_undegraded":"2023-11-24T06:08:59.340171+0000","last_fullsized":"2023-11-24T06:08:59.340171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:31.591882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830207+0000","last_change":"2023-11-24T06:08:58.519601+0000","last_active":"2023-11-24T06:08:59.830207+0000","last_peered":"2023-11-24T06:08:59.830207+0000","last_clean":"2023-11-24T06:08:59.830207+0000","last_became_active":"2023-11-24T06:08:58.519340+0000","last_became_peered":"2023-11-24T06:08:58.519340+0000","last_unstale":"2023-11-24T06:08:59.830207+0000","last_undegraded":"2023-11-24T06:08:59.830207+0000","last_fullsized":"2023-11-24T06:08:59.830207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:51.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339001+0000","last_change":"2023-11-24T06:08:57.529494+0000","last_active":"2023-11-24T06:08:59.339001+0000","last_peered":"2023-11-24T06:08:59.339001+0000","last_clean":"2023-11-24T06:08:59.339001+0000","last_became_active":"2023-11-24T06:08:57.529231+0000","last_became_peered":"2023-11-24T06:08:57.529231+0000","last_unstale":"2023-11-24T06:08:59.339001+0000","last_undegraded":"2023-11-24T06:08:59.339001+0000","last_fullsized":"2023-11-24T06:08:59.339001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:06.359234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829452+0000","last_change":"2023-11-24T06:08:58.529994+0000","last_active":"2023-11-24T06:08:59.829452+0000","last_peered":"2023-11-24T06:08:59.829452+0000","last_clean":"2023-11-24T06:08:59.829452+0000","last_became_active":"2023-11-24T06:08:58.529528+0000","last_became_peered":"2023-11-24T06:08:58.529528+0000","last_unstale":"2023-11-24T06:08:59.829452+0000","last_undegraded":"2023-11-24T06:08:59.829452+0000","last_fullsized":"2023-11-24T06:08:59.829452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:32.599128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343170+0000","last_change":"2023-11-24T06:08:57.525323+0000","last_active":"2023-11-24T06:08:59.343170+0000","last_peered":"2023-11-24T06:08:59.343170+0000","last_clean":"2023-11-24T06:08:59.343170+0000","last_became_active":"2023-11-24T06:08:57.525087+0000","last_became_peered":"2023-11-24T06:08:57.525087+0000","last_unstale":"2023-11-24T06:08:59.343170+0000","last_undegraded":"2023-11-24T06:08:59.343170+0000","last_fullsized":"2023-11-24T06:08:59.343170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:43.891317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830689+0000","last_change":"2023-11-24T06:08:58.539679+0000","last_active":"2023-11-24T06:08:59.830689+0000","last_peered":"2023-11-24T06:08:59.830689+0000","last_clean":"2023-11-24T06:08:59.830689+0000","last_became_active":"2023-11-24T06:08:58.539279+0000","last_became_peered":"2023-11-24T06:08:58.539279+0000","last_unstale":"2023-11-24T06:08:59.830689+0000","last_undegraded":"2023-11-24T06:08:59.830689+0000","last_fullsized":"2023-11-24T06:08:59.830689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:13.221261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340249+0000","last_change":"2023-11-24T06:08:57.523738+0000","last_active":"2023-11-24T06:08:59.340249+0000","last_peered":"2023-11-24T06:08:59.340249+0000","last_clean":"2023-11-24T06:08:59.340249+0000","last_became_active":"2023-11-24T06:08:57.523396+0000","last_became_peered":"2023-11-24T06:08:57.523396+0000","last_unstale":"2023-11-24T06:08:59.340249+0000","last_undegraded":"2023-11-24T06:08:59.340249+0000","last_fullsized":"2023-11-24T06:08:59.340249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518526+0000","last_change":"2023-11-24T06:08:58.520593+0000","last_active":"2023-11-24T06:08:59.518526+0000","last_peered":"2023-11-24T06:08:59.518526+0000","last_clean":"2023-11-24T06:08:59.518526+0000","last_became_active":"2023-11-24T06:08:58.520297+0000","last_became_peered":"2023-11-24T06:08:58.520297+0000","last_unstale":"2023-11-24T06:08:59.518526+0000","last_undegraded":"2023-11-24T06:08:59.518526+0000","last_fullsized":"2023-11-24T06:08:59.518526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:55.981203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343206+0000","last_change":"2023-11-24T06:08:57.521315+0000","last_active":"2023-11-24T06:08:59.343206+0000","last_peered":"2023-11-24T06:08:59.343206+0000","last_clean":"2023-11-24T06:08:59.343206+0000","last_became_active":"2023-11-24T06:08:57.520718+0000","last_became_peered":"2023-11-24T06:08:57.520718+0000","last_unstale":"2023-11-24T06:08:59.343206+0000","last_undegraded":"2023-11-24T06:08:59.343206+0000","last_fullsized":"2023-11-24T06:08:59.343206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:16.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830728+0000","last_change":"2023-11-24T06:08:58.542621+0000","last_active":"2023-11-24T06:08:59.830728+0000","last_peered":"2023-11-24T06:08:59.830728+0000","last_clean":"2023-11-24T06:08:59.830728+0000","last_became_active":"2023-11-24T06:08:58.541632+0000","last_became_peered":"2023-11-24T06:08:58.541632+0000","last_unstale":"2023-11-24T06:08:59.830728+0000","last_undegraded":"2023-11-24T06:08:59.830728+0000","last_fullsized":"2023-11-24T06:08:59.830728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:40.188112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829349+0000","last_change":"2023-11-24T06:08:58.527955+0000","last_active":"2023-11-24T06:08:59.829349+0000","last_peered":"2023-11-24T06:08:59.829349+0000","last_clean":"2023-11-24T06:08:59.829349+0000","last_became_active":"2023-11-24T06:08:58.527336+0000","last_became_peered":"2023-11-24T06:08:58.527336+0000","last_unstale":"2023-11-24T06:08:59.829349+0000","last_undegraded":"2023-11-24T06:08:59.829349+0000","last_fullsized":"2023-11-24T06:08:59.829349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:01.809823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830151+0000","last_change":"2023-11-24T06:08:58.530318+0000","last_active":"2023-11-24T06:08:59.830151+0000","last_peered":"2023-11-24T06:08:59.830151+0000","last_clean":"2023-11-24T06:08:59.830151+0000","last_became_active":"2023-11-24T06:08:58.530015+0000","last_became_peered":"2023-11-24T06:08:58.530015+0000","last_unstale":"2023-11-24T06:08:59.830151+0000","last_undegraded":"2023-11-24T06:08:59.830151+0000","last_fullsized":"2023-11-24T06:08:59.830151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517268+0000","last_change":"2023-11-24T06:08:58.522901+0000","last_active":"2023-11-24T06:08:59.517268+0000","last_peered":"2023-11-24T06:08:59.517268+0000","last_clean":"2023-11-24T06:08:59.517268+0000","last_became_active":"2023-11-24T06:08:58.522620+0000","last_became_peered":"2023-11-24T06:08:58.522620+0000","last_unstale":"2023-11-24T06:08:59.517268+0000","last_undegraded":"2023-11-24T06:08:59.517268+0000","last_fullsized":"2023-11-24T06:08:59.517268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830900+0000","last_change":"2023-11-24T06:08:58.519217+0000","last_active":"2023-11-24T06:08:59.830900+0000","last_peered":"2023-11-24T06:08:59.830900+0000","last_clean":"2023-11-24T06:08:59.830900+0000","last_became_active":"2023-11-24T06:08:58.518577+0000","last_became_peered":"2023-11-24T06:08:58.518577+0000","last_unstale":"2023-11-24T06:08:59.830900+0000","last_undegraded":"2023-11-24T06:08:59.830900+0000","last_fullsized":"2023-11-24T06:08:59.830900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:01.910093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830247+0000","last_change":"2023-11-24T06:08:58.523627+0000","last_active":"2023-11-24T06:08:59.830247+0000","last_peered":"2023-11-24T06:08:59.830247+0000","last_clean":"2023-11-24T06:08:59.830247+0000","last_became_active":"2023-11-24T06:08:58.522543+0000","last_became_peered":"2023-11-24T06:08:58.522543+0000","last_unstale":"2023-11-24T06:08:59.830247+0000","last_undegraded":"2023-11-24T06:08:59.830247+0000","last_fullsized":"2023-11-24T06:08:59.830247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:09.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516517+0000","last_change":"2023-11-24T06:08:58.521811+0000","last_active":"2023-11-24T06:08:59.516517+0000","last_peered":"2023-11-24T06:08:59.516517+0000","last_clean":"2023-11-24T06:08:59.516517+0000","last_became_active":"2023-11-24T06:08:58.521129+0000","last_became_peered":"2023-11-24T06:08:58.521129+0000","last_unstale":"2023-11-24T06:08:59.516517+0000","last_undegraded":"2023-11-24T06:08:59.516517+0000","last_fullsized":"2023-11-24T06:08:59.516517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:51.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829914+0000","last_change":"2023-11-24T06:08:58.539747+0000","last_active":"2023-11-24T06:08:59.829914+0000","last_peered":"2023-11-24T06:08:59.829914+0000","last_clean":"2023-11-24T06:08:59.829914+0000","last_became_active":"2023-11-24T06:08:58.539449+0000","last_became_peered":"2023-11-24T06:08:58.539449+0000","last_unstale":"2023-11-24T06:08:59.829914+0000","last_undegraded":"2023-11-24T06:08:59.829914+0000","last_fullsized":"2023-11-24T06:08:59.829914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830096+0000","last_change":"2023-11-24T06:08:58.523632+0000","last_active":"2023-11-24T06:08:59.830096+0000","last_peered":"2023-11-24T06:08:59.830096+0000","last_clean":"2023-11-24T06:08:59.830096+0000","last_became_active":"2023-11-24T06:08:58.523365+0000","last_became_peered":"2023-11-24T06:08:58.523365+0000","last_unstale":"2023-11-24T06:08:59.830096+0000","last_undegraded":"2023-11-24T06:08:59.830096+0000","last_fullsized":"2023-11-24T06:08:59.830096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517020+0000","last_change":"2023-11-24T06:08:58.527956+0000","last_active":"2023-11-24T06:08:59.517020+0000","last_peered":"2023-11-24T06:08:59.517020+0000","last_clean":"2023-11-24T06:08:59.517020+0000","last_became_active":"2023-11-24T06:08:58.527700+0000","last_became_peered":"2023-11-24T06:08:58.527700+0000","last_unstale":"2023-11-24T06:08:59.517020+0000","last_undegraded":"2023-11-24T06:08:59.517020+0000","last_fullsized":"2023-11-24T06:08:59.517020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830320+0000","last_change":"2023-11-24T06:08:58.529061+0000","last_active":"2023-11-24T06:08:59.830320+0000","last_peered":"2023-11-24T06:08:59.830320+0000","last_clean":"2023-11-24T06:08:59.830320+0000","last_became_active":"2023-11-24T06:08:58.528753+0000","last_became_peered":"2023-11-24T06:08:58.528753+0000","last_unstale":"2023-11-24T06:08:59.830320+0000","last_undegraded":"2023-11-24T06:08:59.830320+0000","last_fullsized":"2023-11-24T06:08:59.830320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:32.177428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517097+0000","last_change":"2023-11-24T06:08:58.541017+0000","last_active":"2023-11-24T06:08:59.517097+0000","last_peered":"2023-11-24T06:08:59.517097+0000","last_clean":"2023-11-24T06:08:59.517097+0000","last_became_active":"2023-11-24T06:08:58.540746+0000","last_became_peered":"2023-11-24T06:08:58.540746+0000","last_unstale":"2023-11-24T06:08:59.517097+0000","last_undegraded":"2023-11-24T06:08:59.517097+0000","last_fullsized":"2023-11-24T06:08:59.517097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830345+0000","last_change":"2023-11-24T06:08:58.540211+0000","last_active":"2023-11-24T06:08:59.830345+0000","last_peered":"2023-11-24T06:08:59.830345+0000","last_clean":"2023-11-24T06:08:59.830345+0000","last_became_active":"2023-11-24T06:08:58.539978+0000","last_became_peered":"2023-11-24T06:08:58.539978+0000","last_unstale":"2023-11-24T06:08:59.830345+0000","last_undegraded":"2023-11-24T06:08:59.830345+0000","last_fullsized":"2023-11-24T06:08:59.830345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:01.809823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829744+0000","last_change":"2023-11-24T06:08:58.529024+0000","last_active":"2023-11-24T06:08:59.829744+0000","last_peered":"2023-11-24T06:08:59.829744+0000","last_clean":"2023-11-24T06:08:59.829744+0000","last_became_active":"2023-11-24T06:08:58.528704+0000","last_became_peered":"2023-11-24T06:08:58.528704+0000","last_unstale":"2023-11-24T06:08:59.829744+0000","last_undegraded":"2023-11-24T06:08:59.829744+0000","last_fullsized":"2023-11-24T06:08:59.829744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:09:07.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:09:07.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.425+0000 7f1bb9928700 1 Processor -- start 2023-11-24T06:09:07.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.425+0000 7f1bb9928700 1 -- start start 2023-11-24T06:09:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.425+0000 7f1bb9928700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb4162920 0x7f1bb415e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.425+0000 7f1bb9928700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1bb40cfbd0 0x7f1bb415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.425+0000 7f1bb9928700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb4161bf0 con 0x7f1bb415bd30 2023-11-24T06:09:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.425+0000 7f1bb9928700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb415f450 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.425+0000 7f1bb9928700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb4161e40 con 0x7f1bb4162920 2023-11-24T06:09:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1bb40cfbd0 0x7f1bb415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1bb40cfbd0 0x7f1bb415ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45508/0 (socket says 172.21.15.97:45508) 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb37fe700 1 -- 172.21.15.97:0/626118080 learned_addr learned my addr 172.21.15.97:0/626118080 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb27fc700 1 --2- 172.21.15.97:0/626118080 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb4162920 0x7f1bb415e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb37fe700 1 -- 172.21.15.97:0/626118080 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb4162920 msgr2=0x7f1bb415e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb37fe700 1 --2- 172.21.15.97:0/626118080 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb4162920 0x7f1bb415e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb37fe700 1 -- 172.21.15.97:0/626118080 >> v1:172.21.15.183:6789/0 conn(0x7f1bb415bd30 legacy=0x7f1bb40cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb37fe700 1 -- 172.21.15.97:0/626118080 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb4162120 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb37fe700 1 --2- 172.21.15.97:0/626118080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415ec20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009e50 tx=0x7f1bac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4421dd9b066993b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb1ffb700 1 -- 172.21.15.97:0/626118080 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00e070 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb1ffb700 1 -- 172.21.15.97:0/626118080 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac00b9c0 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.426+0000 7f1bb1ffb700 1 -- 172.21.15.97:0/626118080 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00bd70 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.427+0000 7f1bb9928700 1 -- 172.21.15.97:0/626118080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 msgr2=0x7f1bb415ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.427+0000 7f1bb9928700 1 --2- 172.21.15.97:0/626118080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415ec20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009e50 tx=0x7f1bac00b330 comp rx=0 tx=0).stop 2023-11-24T06:09:07.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.427+0000 7f1bb9928700 1 -- 172.21.15.97:0/626118080 shutdown_connections 2023-11-24T06:09:07.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.427+0000 7f1bb9928700 1 --2- 172.21.15.97:0/626118080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415ec20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.427+0000 7f1bb9928700 1 --2- 172.21.15.97:0/626118080 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb4162920 0x7f1bb415e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.427+0000 7f1bb9928700 1 -- 172.21.15.97:0/626118080 >> 172.21.15.97:0/626118080 conn(0x7f1bb4072710 msgr2=0x7f1bb4072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:07.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.427+0000 7f1bb9928700 1 -- 172.21.15.97:0/626118080 shutdown_connections 2023-11-24T06:09:07.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.427+0000 7f1bb9928700 1 -- 172.21.15.97:0/626118080 wait complete. 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.428+0000 7f1bb9928700 1 Processor -- start 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.428+0000 7f1bb9928700 1 -- start start 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb9928700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb9928700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb415bd30 0x7f1bb40c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb9928700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb4162920 0x7f1bb40b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb9928700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40c7ca0 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb9928700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40c7f80 con 0x7f1bb4162920 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb9928700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40c8260 con 0x7f1bb415bd30 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45510/0 (socket says 172.21.15.97:45510) 2023-11-24T06:09:07.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 -- 172.21.15.97:0/1242981022 learned_addr learned my addr 172.21.15.97:0/1242981022 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb27fc700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb415bd30 0x7f1bb40c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb37fe700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb4162920 0x7f1bb40b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 -- 172.21.15.97:0/1242981022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb415bd30 msgr2=0x7f1bb40c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb415bd30 0x7f1bb40c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 -- 172.21.15.97:0/1242981022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb4162920 msgr2=0x7f1bb40b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb4162920 0x7f1bb40b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 -- 172.21.15.97:0/1242981022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac009b30 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.429+0000 7f1bb2ffd700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415b0e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400c4e0 tx=0x7f1ba400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.430+0000 7f1b9bfff700 1 -- 172.21.15.97:0/1242981022 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40052d0 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.430+0000 7f1b9bfff700 1 -- 172.21.15.97:0/1242981022 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4005430 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.430+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb40c8540 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.430+0000 7f1b9bfff700 1 -- 172.21.15.97:0/1242981022 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40038c0 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.430+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb415b620 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.430+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb40ccb20 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.431+0000 7f1b9bfff700 1 -- 172.21.15.97:0/1242981022 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ba4003a20 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.431+0000 7f1b9bfff700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ba006aed0 0x7f1ba006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.431+0000 7f1bb27fc700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ba006aed0 0x7f1ba006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.431+0000 7f1b9bfff700 1 -- 172.21.15.97:0/1242981022 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ba4088980 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.432+0000 7f1bb27fc700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ba006aed0 0x7f1ba006d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40cd9e0 tx=0x7f1ba8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.436+0000 7f1b9bfff700 1 -- 172.21.15.97:0/1242981022 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ba4053a30 con 0x7f1bb40cfbd0 2023-11-24T06:09:07.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.625+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bb415f160 con 0x7f1ba006aed0 2023-11-24T06:09:07.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.633+0000 7f1b9bfff700 1 -- 172.21.15.97:0/1242981022 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264494 (secure 0 0 0) 0x7f1ba80076d0 con 0x7f1ba006aed0 2023-11-24T06:09:07.639 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ba006aed0 msgr2=0x7f1ba006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ba006aed0 0x7f1ba006d390 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1bb40cd9e0 tx=0x7f1ba8008040 comp rx=0 tx=0).stop 2023-11-24T06:09:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 msgr2=0x7f1bb415b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415b0e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400c4e0 tx=0x7f1ba400c7f0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 shutdown_connections 2023-11-24T06:09:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb4162920 0x7f1bb40b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1bb40cfbd0 0x7f1bb415b0e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ba006aed0 0x7f1ba006d390 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 --2- 172.21.15.97:0/1242981022 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb415bd30 0x7f1bb40c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 >> 172.21.15.97:0/1242981022 conn(0x7f1bb4072710 msgr2=0x7f1bb406d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 shutdown_connections 2023-11-24T06:09:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.636+0000 7f1bb9928700 1 -- 172.21.15.97:0/1242981022 wait complete. 2023-11-24T06:09:07.642 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:09:07.651 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:09:06.326863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219040,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755680,"statfs":{"total":773094113280,"available":772869816320,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004174"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830864+0000","last_change":"2023-11-24T06:08:58.542671+0000","last_active":"2023-11-24T06:08:59.830864+0000","last_peered":"2023-11-24T06:08:59.830864+0000","last_clean":"2023-11-24T06:08:59.830864+0000","last_became_active":"2023-11-24T06:08:58.541937+0000","last_became_peered":"2023-11-24T06:08:58.541937+0000","last_unstale":"2023-11-24T06:08:59.830864+0000","last_undegraded":"2023-11-24T06:08:59.830864+0000","last_fullsized":"2023-11-24T06:08:59.830864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:44:30.040948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516727+0000","last_change":"2023-11-24T06:08:58.522085+0000","last_active":"2023-11-24T06:08:59.516727+0000","last_peered":"2023-11-24T06:08:59.516727+0000","last_clean":"2023-11-24T06:08:59.516727+0000","last_became_active":"2023-11-24T06:08:58.521116+0000","last_became_peered":"2023-11-24T06:08:58.521116+0000","last_unstale":"2023-11-24T06:08:59.516727+0000","last_undegraded":"2023-11-24T06:08:59.516727+0000","last_fullsized":"2023-11-24T06:08:59.516727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:10.129141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830976+0000","last_change":"2023-11-24T06:08:58.515201+0000","last_active":"2023-11-24T06:08:59.830976+0000","last_peered":"2023-11-24T06:08:59.830976+0000","last_clean":"2023-11-24T06:08:59.830976+0000","last_became_active":"2023-11-24T06:08:58.514911+0000","last_became_peered":"2023-11-24T06:08:58.514911+0000","last_unstale":"2023-11-24T06:08:59.830976+0000","last_undegraded":"2023-11-24T06:08:59.830976+0000","last_fullsized":"2023-11-24T06:08:59.830976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518651+0000","last_change":"2023-11-24T06:08:58.520988+0000","last_active":"2023-11-24T06:08:59.518651+0000","last_peered":"2023-11-24T06:08:59.518651+0000","last_clean":"2023-11-24T06:08:59.518651+0000","last_became_active":"2023-11-24T06:08:58.520029+0000","last_became_peered":"2023-11-24T06:08:58.520029+0000","last_unstale":"2023-11-24T06:08:59.518651+0000","last_undegraded":"2023-11-24T06:08:59.518651+0000","last_fullsized":"2023-11-24T06:08:59.518651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:13.221261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830522+0000","last_change":"2023-11-24T06:08:58.535989+0000","last_active":"2023-11-24T06:08:59.830522+0000","last_peered":"2023-11-24T06:08:59.830522+0000","last_clean":"2023-11-24T06:08:59.830522+0000","last_became_active":"2023-11-24T06:08:58.535697+0000","last_became_peered":"2023-11-24T06:08:58.535697+0000","last_unstale":"2023-11-24T06:08:59.830522+0000","last_undegraded":"2023-11-24T06:08:59.830522+0000","last_fullsized":"2023-11-24T06:08:59.830522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:27:52.505428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517494+0000","last_change":"2023-11-24T06:08:58.523237+0000","last_active":"2023-11-24T06:08:59.517494+0000","last_peered":"2023-11-24T06:08:59.517494+0000","last_clean":"2023-11-24T06:08:59.517494+0000","last_became_active":"2023-11-24T06:08:58.522368+0000","last_became_peered":"2023-11-24T06:08:58.522368+0000","last_unstale":"2023-11-24T06:08:59.517494+0000","last_undegraded":"2023-11-24T06:08:59.517494+0000","last_fullsized":"2023-11-24T06:08:59.517494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:32.177428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829818+0000","last_change":"2023-11-24T06:08:58.533834+0000","last_active":"2023-11-24T06:08:59.829818+0000","last_peered":"2023-11-24T06:08:59.829818+0000","last_clean":"2023-11-24T06:08:59.829818+0000","last_became_active":"2023-11-24T06:08:58.533539+0000","last_became_peered":"2023-11-24T06:08:58.533539+0000","last_unstale":"2023-11-24T06:08:59.829818+0000","last_undegraded":"2023-11-24T06:08:59.829818+0000","last_fullsized":"2023-11-24T06:08:59.829818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:10.129141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829824+0000","last_change":"2023-11-24T06:08:58.531528+0000","last_active":"2023-11-24T06:08:59.829824+0000","last_peered":"2023-11-24T06:08:59.829824+0000","last_clean":"2023-11-24T06:08:59.829824+0000","last_became_active":"2023-11-24T06:08:58.531144+0000","last_became_peered":"2023-11-24T06:08:58.531144+0000","last_unstale":"2023-11-24T06:08:59.829824+0000","last_undegraded":"2023-11-24T06:08:59.829824+0000","last_fullsized":"2023-11-24T06:08:59.829824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:01.910093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830820+0000","last_change":"2023-11-24T06:08:58.542860+0000","last_active":"2023-11-24T06:08:59.830820+0000","last_peered":"2023-11-24T06:08:59.830820+0000","last_clean":"2023-11-24T06:08:59.830820+0000","last_became_active":"2023-11-24T06:08:58.542347+0000","last_became_peered":"2023-11-24T06:08:58.542347+0000","last_unstale":"2023-11-24T06:08:59.830820+0000","last_undegraded":"2023-11-24T06:08:59.830820+0000","last_fullsized":"2023-11-24T06:08:59.830820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:01.789347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830098+0000","last_change":"2023-11-24T06:08:58.529308+0000","last_active":"2023-11-24T06:08:59.830098+0000","last_peered":"2023-11-24T06:08:59.830098+0000","last_clean":"2023-11-24T06:08:59.830098+0000","last_became_active":"2023-11-24T06:08:58.528986+0000","last_became_peered":"2023-11-24T06:08:58.528986+0000","last_unstale":"2023-11-24T06:08:59.830098+0000","last_undegraded":"2023-11-24T06:08:59.830098+0000","last_fullsized":"2023-11-24T06:08:59.830098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829848+0000","last_change":"2023-11-24T06:08:58.530560+0000","last_active":"2023-11-24T06:08:59.829848+0000","last_peered":"2023-11-24T06:08:59.829848+0000","last_clean":"2023-11-24T06:08:59.829848+0000","last_became_active":"2023-11-24T06:08:58.530299+0000","last_became_peered":"2023-11-24T06:08:58.530299+0000","last_unstale":"2023-11-24T06:08:59.829848+0000","last_undegraded":"2023-11-24T06:08:59.829848+0000","last_fullsized":"2023-11-24T06:08:59.829848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517877+0000","last_change":"2023-11-24T06:08:58.529565+0000","last_active":"2023-11-24T06:08:59.517877+0000","last_peered":"2023-11-24T06:08:59.517877+0000","last_clean":"2023-11-24T06:08:59.517877+0000","last_became_active":"2023-11-24T06:08:58.529170+0000","last_became_peered":"2023-11-24T06:08:58.529170+0000","last_unstale":"2023-11-24T06:08:59.517877+0000","last_undegraded":"2023-11-24T06:08:59.517877+0000","last_fullsized":"2023-11-24T06:08:59.517877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829628+0000","last_change":"2023-11-24T06:08:58.521255+0000","last_active":"2023-11-24T06:08:59.829628+0000","last_peered":"2023-11-24T06:08:59.829628+0000","last_clean":"2023-11-24T06:08:59.829628+0000","last_became_active":"2023-11-24T06:08:58.520950+0000","last_became_peered":"2023-11-24T06:08:58.520950+0000","last_unstale":"2023-11-24T06:08:59.829628+0000","last_undegraded":"2023-11-24T06:08:59.829628+0000","last_fullsized":"2023-11-24T06:08:59.829628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:01.809823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829844+0000","last_change":"2023-11-24T06:08:58.525412+0000","last_active":"2023-11-24T06:08:59.829844+0000","last_peered":"2023-11-24T06:08:59.829844+0000","last_clean":"2023-11-24T06:08:59.829844+0000","last_became_active":"2023-11-24T06:08:58.525080+0000","last_became_peered":"2023-11-24T06:08:58.525080+0000","last_unstale":"2023-11-24T06:08:59.829844+0000","last_undegraded":"2023-11-24T06:08:59.829844+0000","last_fullsized":"2023-11-24T06:08:59.829844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:40.002717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516190+0000","last_change":"2023-11-24T06:08:58.529101+0000","last_active":"2023-11-24T06:08:59.516190+0000","last_peered":"2023-11-24T06:08:59.516190+0000","last_clean":"2023-11-24T06:08:59.516190+0000","last_became_active":"2023-11-24T06:08:58.528814+0000","last_became_peered":"2023-11-24T06:08:58.528814+0000","last_unstale":"2023-11-24T06:08:59.516190+0000","last_undegraded":"2023-11-24T06:08:59.516190+0000","last_fullsized":"2023-11-24T06:08:59.516190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516855+0000","last_change":"2023-11-24T06:08:58.523287+0000","last_active":"2023-11-24T06:08:59.516855+0000","last_peered":"2023-11-24T06:08:59.516855+0000","last_clean":"2023-11-24T06:08:59.516855+0000","last_became_active":"2023-11-24T06:08:58.523033+0000","last_became_peered":"2023-11-24T06:08:58.523033+0000","last_unstale":"2023-11-24T06:08:59.516855+0000","last_undegraded":"2023-11-24T06:08:59.516855+0000","last_fullsized":"2023-11-24T06:08:59.516855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:10.129141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517860+0000","last_change":"2023-11-24T06:08:58.543046+0000","last_active":"2023-11-24T06:08:59.517860+0000","last_peered":"2023-11-24T06:08:59.517860+0000","last_clean":"2023-11-24T06:08:59.517860+0000","last_became_active":"2023-11-24T06:08:58.542703+0000","last_became_peered":"2023-11-24T06:08:58.542703+0000","last_unstale":"2023-11-24T06:08:59.517860+0000","last_undegraded":"2023-11-24T06:08:59.517860+0000","last_fullsized":"2023-11-24T06:08:59.517860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:09.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829953+0000","last_change":"2023-11-24T06:08:58.521888+0000","last_active":"2023-11-24T06:08:59.829953+0000","last_peered":"2023-11-24T06:08:59.829953+0000","last_clean":"2023-11-24T06:08:59.829953+0000","last_became_active":"2023-11-24T06:08:58.521599+0000","last_became_peered":"2023-11-24T06:08:58.521599+0000","last_unstale":"2023-11-24T06:08:59.829953+0000","last_undegraded":"2023-11-24T06:08:59.829953+0000","last_fullsized":"2023-11-24T06:08:59.829953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830479+0000","last_change":"2023-11-24T06:08:58.536029+0000","last_active":"2023-11-24T06:08:59.830479+0000","last_peered":"2023-11-24T06:08:59.830479+0000","last_clean":"2023-11-24T06:08:59.830479+0000","last_became_active":"2023-11-24T06:08:58.535076+0000","last_became_peered":"2023-11-24T06:08:58.535076+0000","last_unstale":"2023-11-24T06:08:59.830479+0000","last_undegraded":"2023-11-24T06:08:59.830479+0000","last_fullsized":"2023-11-24T06:08:59.830479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:09.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.344516+0000","last_change":"2023-11-24T06:08:57.529523+0000","last_active":"2023-11-24T06:08:59.344516+0000","last_peered":"2023-11-24T06:08:59.344516+0000","last_clean":"2023-11-24T06:08:59.344516+0000","last_became_active":"2023-11-24T06:08:57.529207+0000","last_became_peered":"2023-11-24T06:08:57.529207+0000","last_unstale":"2023-11-24T06:08:59.344516+0000","last_undegraded":"2023-11-24T06:08:59.344516+0000","last_fullsized":"2023-11-24T06:08:59.344516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:52.525711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830189+0000","last_change":"2023-11-24T06:08:58.523242+0000","last_active":"2023-11-24T06:08:59.830189+0000","last_peered":"2023-11-24T06:08:59.830189+0000","last_clean":"2023-11-24T06:08:59.830189+0000","last_became_active":"2023-11-24T06:08:58.522951+0000","last_became_peered":"2023-11-24T06:08:58.522951+0000","last_unstale":"2023-11-24T06:08:59.830189+0000","last_undegraded":"2023-11-24T06:08:59.830189+0000","last_fullsized":"2023-11-24T06:08:59.830189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343204+0000","last_change":"2023-11-24T06:08:57.523196+0000","last_active":"2023-11-24T06:08:59.343204+0000","last_peered":"2023-11-24T06:08:59.343204+0000","last_clean":"2023-11-24T06:08:59.343204+0000","last_became_active":"2023-11-24T06:08:57.522871+0000","last_became_peered":"2023-11-24T06:08:57.522871+0000","last_unstale":"2023-11-24T06:08:59.343204+0000","last_undegraded":"2023-11-24T06:08:59.343204+0000","last_fullsized":"2023-11-24T06:08:59.343204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:22.630593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.831052+0000","last_change":"2023-11-24T06:08:58.530832+0000","last_active":"2023-11-24T06:08:59.831052+0000","last_peered":"2023-11-24T06:08:59.831052+0000","last_clean":"2023-11-24T06:08:59.831052+0000","last_became_active":"2023-11-24T06:08:58.530383+0000","last_became_peered":"2023-11-24T06:08:58.530383+0000","last_unstale":"2023-11-24T06:08:59.831052+0000","last_undegraded":"2023-11-24T06:08:59.831052+0000","last_fullsized":"2023-11-24T06:08:59.831052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339081+0000","last_change":"2023-11-24T06:08:57.528776+0000","last_active":"2023-11-24T06:08:59.339081+0000","last_peered":"2023-11-24T06:08:59.339081+0000","last_clean":"2023-11-24T06:08:59.339081+0000","last_became_active":"2023-11-24T06:08:57.528480+0000","last_became_peered":"2023-11-24T06:08:57.528480+0000","last_unstale":"2023-11-24T06:08:59.339081+0000","last_undegraded":"2023-11-24T06:08:59.339081+0000","last_fullsized":"2023-11-24T06:08:59.339081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518377+0000","last_change":"2023-11-24T06:08:58.524300+0000","last_active":"2023-11-24T06:08:59.518377+0000","last_peered":"2023-11-24T06:08:59.518377+0000","last_clean":"2023-11-24T06:08:59.518377+0000","last_became_active":"2023-11-24T06:08:58.522378+0000","last_became_peered":"2023-11-24T06:08:58.522378+0000","last_unstale":"2023-11-24T06:08:59.518377+0000","last_undegraded":"2023-11-24T06:08:59.518377+0000","last_fullsized":"2023-11-24T06:08:59.518377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:27:52.505428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339013+0000","last_change":"2023-11-24T06:08:57.526272+0000","last_active":"2023-11-24T06:08:59.339013+0000","last_peered":"2023-11-24T06:08:59.339013+0000","last_clean":"2023-11-24T06:08:59.339013+0000","last_became_active":"2023-11-24T06:08:57.525998+0000","last_became_peered":"2023-11-24T06:08:57.525998+0000","last_unstale":"2023-11-24T06:08:59.339013+0000","last_undegraded":"2023-11-24T06:08:59.339013+0000","last_fullsized":"2023-11-24T06:08:59.339013+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518271+0000","last_change":"2023-11-24T06:08:58.523547+0000","last_active":"2023-11-24T06:08:59.518271+0000","last_peered":"2023-11-24T06:08:59.518271+0000","last_clean":"2023-11-24T06:08:59.518271+0000","last_became_active":"2023-11-24T06:08:58.522784+0000","last_became_peered":"2023-11-24T06:08:58.522784+0000","last_unstale":"2023-11-24T06:08:59.518271+0000","last_undegraded":"2023-11-24T06:08:59.518271+0000","last_fullsized":"2023-11-24T06:08:59.518271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:33.338886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343116+0000","last_change":"2023-11-24T06:08:57.528824+0000","last_active":"2023-11-24T06:08:59.343116+0000","last_peered":"2023-11-24T06:08:59.343116+0000","last_clean":"2023-11-24T06:08:59.343116+0000","last_became_active":"2023-11-24T06:08:57.528517+0000","last_became_peered":"2023-11-24T06:08:57.528517+0000","last_unstale":"2023-11-24T06:08:59.343116+0000","last_undegraded":"2023-11-24T06:08:59.343116+0000","last_fullsized":"2023-11-24T06:08:59.343116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517286+0000","last_change":"2023-11-24T06:08:58.523181+0000","last_active":"2023-11-24T06:08:59.517286+0000","last_peered":"2023-11-24T06:08:59.517286+0000","last_clean":"2023-11-24T06:08:59.517286+0000","last_became_active":"2023-11-24T06:08:58.522917+0000","last_became_peered":"2023-11-24T06:08:58.522917+0000","last_unstale":"2023-11-24T06:08:59.517286+0000","last_undegraded":"2023-11-24T06:08:59.517286+0000","last_fullsized":"2023-11-24T06:08:59.517286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:17.360361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337315+0000","last_change":"2023-11-24T06:08:57.524473+0000","last_active":"2023-11-24T06:08:59.337315+0000","last_peered":"2023-11-24T06:08:59.337315+0000","last_clean":"2023-11-24T06:08:59.337315+0000","last_became_active":"2023-11-24T06:08:57.523947+0000","last_became_peered":"2023-11-24T06:08:57.523947+0000","last_unstale":"2023-11-24T06:08:59.337315+0000","last_undegraded":"2023-11-24T06:08:59.337315+0000","last_fullsized":"2023-11-24T06:08:59.337315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830977+0000","last_change":"2023-11-24T06:08:58.526170+0000","last_active":"2023-11-24T06:08:59.830977+0000","last_peered":"2023-11-24T06:08:59.830977+0000","last_clean":"2023-11-24T06:08:59.830977+0000","last_became_active":"2023-11-24T06:08:58.525874+0000","last_became_peered":"2023-11-24T06:08:58.525874+0000","last_unstale":"2023-11-24T06:08:59.830977+0000","last_undegraded":"2023-11-24T06:08:59.830977+0000","last_fullsized":"2023-11-24T06:08:59.830977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:32.599128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343057+0000","last_change":"2023-11-24T06:08:57.520895+0000","last_active":"2023-11-24T06:08:59.343057+0000","last_peered":"2023-11-24T06:08:59.343057+0000","last_clean":"2023-11-24T06:08:59.343057+0000","last_became_active":"2023-11-24T06:08:57.520367+0000","last_became_peered":"2023-11-24T06:08:57.520367+0000","last_unstale":"2023-11-24T06:08:59.343057+0000","last_undegraded":"2023-11-24T06:08:59.343057+0000","last_fullsized":"2023-11-24T06:08:59.343057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:55.763227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830105+0000","last_change":"2023-11-24T06:08:58.519434+0000","last_active":"2023-11-24T06:08:59.830105+0000","last_peered":"2023-11-24T06:08:59.830105+0000","last_clean":"2023-11-24T06:08:59.830105+0000","last_became_active":"2023-11-24T06:08:58.519148+0000","last_became_peered":"2023-11-24T06:08:58.519148+0000","last_unstale":"2023-11-24T06:08:59.830105+0000","last_undegraded":"2023-11-24T06:08:59.830105+0000","last_fullsized":"2023-11-24T06:08:59.830105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339366+0000","last_change":"2023-11-24T06:08:57.525692+0000","last_active":"2023-11-24T06:08:59.339366+0000","last_peered":"2023-11-24T06:08:59.339366+0000","last_clean":"2023-11-24T06:08:59.339366+0000","last_became_active":"2023-11-24T06:08:57.525451+0000","last_became_peered":"2023-11-24T06:08:57.525451+0000","last_unstale":"2023-11-24T06:08:59.339366+0000","last_undegraded":"2023-11-24T06:08:59.339366+0000","last_fullsized":"2023-11-24T06:08:59.339366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516297+0000","last_change":"2023-11-24T06:08:58.521741+0000","last_active":"2023-11-24T06:08:59.516297+0000","last_peered":"2023-11-24T06:08:59.516297+0000","last_clean":"2023-11-24T06:08:59.516297+0000","last_became_active":"2023-11-24T06:08:58.521275+0000","last_became_peered":"2023-11-24T06:08:58.521275+0000","last_unstale":"2023-11-24T06:08:59.516297+0000","last_undegraded":"2023-11-24T06:08:59.516297+0000","last_fullsized":"2023-11-24T06:08:59.516297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337411+0000","last_change":"2023-11-24T06:08:57.526466+0000","last_active":"2023-11-24T06:08:59.337411+0000","last_peered":"2023-11-24T06:08:59.337411+0000","last_clean":"2023-11-24T06:08:59.337411+0000","last_became_active":"2023-11-24T06:08:57.526174+0000","last_became_peered":"2023-11-24T06:08:57.526174+0000","last_unstale":"2023-11-24T06:08:59.337411+0000","last_undegraded":"2023-11-24T06:08:59.337411+0000","last_fullsized":"2023-11-24T06:08:59.337411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517356+0000","last_change":"2023-11-24T06:08:58.541089+0000","last_active":"2023-11-24T06:08:59.517356+0000","last_peered":"2023-11-24T06:08:59.517356+0000","last_clean":"2023-11-24T06:08:59.517356+0000","last_became_active":"2023-11-24T06:08:58.540792+0000","last_became_peered":"2023-11-24T06:08:58.540792+0000","last_unstale":"2023-11-24T06:08:59.517356+0000","last_undegraded":"2023-11-24T06:08:59.517356+0000","last_fullsized":"2023-11-24T06:08:59.517356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:51.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339541+0000","last_change":"2023-11-24T06:08:57.525181+0000","last_active":"2023-11-24T06:08:59.339541+0000","last_peered":"2023-11-24T06:08:59.339541+0000","last_clean":"2023-11-24T06:08:59.339541+0000","last_became_active":"2023-11-24T06:08:57.524906+0000","last_became_peered":"2023-11-24T06:08:57.524906+0000","last_unstale":"2023-11-24T06:08:59.339541+0000","last_undegraded":"2023-11-24T06:08:59.339541+0000","last_fullsized":"2023-11-24T06:08:59.339541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.831159+0000","last_change":"2023-11-24T06:08:58.530664+0000","last_active":"2023-11-24T06:08:59.831159+0000","last_peered":"2023-11-24T06:08:59.831159+0000","last_clean":"2023-11-24T06:08:59.831159+0000","last_became_active":"2023-11-24T06:08:58.530068+0000","last_became_peered":"2023-11-24T06:08:58.530068+0000","last_unstale":"2023-11-24T06:08:59.831159+0000","last_undegraded":"2023-11-24T06:08:59.831159+0000","last_fullsized":"2023-11-24T06:08:59.831159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340262+0000","last_change":"2023-11-24T06:08:57.525584+0000","last_active":"2023-11-24T06:08:59.340262+0000","last_peered":"2023-11-24T06:08:59.340262+0000","last_clean":"2023-11-24T06:08:59.340262+0000","last_became_active":"2023-11-24T06:08:57.525340+0000","last_became_peered":"2023-11-24T06:08:57.525340+0000","last_unstale":"2023-11-24T06:08:59.340262+0000","last_undegraded":"2023-11-24T06:08:59.340262+0000","last_fullsized":"2023-11-24T06:08:59.340262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517718+0000","last_change":"2023-11-24T06:08:58.537817+0000","last_active":"2023-11-24T06:08:59.517718+0000","last_peered":"2023-11-24T06:08:59.517718+0000","last_clean":"2023-11-24T06:08:59.517718+0000","last_became_active":"2023-11-24T06:08:58.537604+0000","last_became_peered":"2023-11-24T06:08:58.537604+0000","last_unstale":"2023-11-24T06:08:59.517718+0000","last_undegraded":"2023-11-24T06:08:59.517718+0000","last_fullsized":"2023-11-24T06:08:59.517718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:17.577240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337329+0000","last_change":"2023-11-24T06:08:57.520656+0000","last_active":"2023-11-24T06:08:59.337329+0000","last_peered":"2023-11-24T06:08:59.337329+0000","last_clean":"2023-11-24T06:08:59.337329+0000","last_became_active":"2023-11-24T06:08:57.520398+0000","last_became_peered":"2023-11-24T06:08:57.520398+0000","last_unstale":"2023-11-24T06:08:59.337329+0000","last_undegraded":"2023-11-24T06:08:59.337329+0000","last_fullsized":"2023-11-24T06:08:59.337329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830513+0000","last_change":"2023-11-24T06:08:58.533919+0000","last_active":"2023-11-24T06:08:59.830513+0000","last_peered":"2023-11-24T06:08:59.830513+0000","last_clean":"2023-11-24T06:08:59.830513+0000","last_became_active":"2023-11-24T06:08:58.533565+0000","last_became_peered":"2023-11-24T06:08:58.533565+0000","last_unstale":"2023-11-24T06:08:59.830513+0000","last_undegraded":"2023-11-24T06:08:59.830513+0000","last_fullsized":"2023-11-24T06:08:59.830513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:04:33.338886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.344421+0000","last_change":"2023-11-24T06:08:57.523298+0000","last_active":"2023-11-24T06:08:59.344421+0000","last_peered":"2023-11-24T06:08:59.344421+0000","last_clean":"2023-11-24T06:08:59.344421+0000","last_became_active":"2023-11-24T06:08:57.523000+0000","last_became_peered":"2023-11-24T06:08:57.523000+0000","last_unstale":"2023-11-24T06:08:59.344421+0000","last_undegraded":"2023-11-24T06:08:59.344421+0000","last_fullsized":"2023-11-24T06:08:59.344421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829711+0000","last_change":"2023-11-24T06:08:58.539907+0000","last_active":"2023-11-24T06:08:59.829711+0000","last_peered":"2023-11-24T06:08:59.829711+0000","last_clean":"2023-11-24T06:08:59.829711+0000","last_became_active":"2023-11-24T06:08:58.539663+0000","last_became_peered":"2023-11-24T06:08:58.539663+0000","last_unstale":"2023-11-24T06:08:59.829711+0000","last_undegraded":"2023-11-24T06:08:59.829711+0000","last_fullsized":"2023-11-24T06:08:59.829711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:26.004865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339449+0000","last_change":"2023-11-24T06:08:57.523074+0000","last_active":"2023-11-24T06:08:59.339449+0000","last_peered":"2023-11-24T06:08:59.339449+0000","last_clean":"2023-11-24T06:08:59.339449+0000","last_became_active":"2023-11-24T06:08:57.522527+0000","last_became_peered":"2023-11-24T06:08:57.522527+0000","last_unstale":"2023-11-24T06:08:59.339449+0000","last_undegraded":"2023-11-24T06:08:59.339449+0000","last_fullsized":"2023-11-24T06:08:59.339449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:31.591882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830543+0000","last_change":"2023-11-24T06:08:58.538966+0000","last_active":"2023-11-24T06:08:59.830543+0000","last_peered":"2023-11-24T06:08:59.830543+0000","last_clean":"2023-11-24T06:08:59.830543+0000","last_became_active":"2023-11-24T06:08:58.538575+0000","last_became_peered":"2023-11-24T06:08:58.538575+0000","last_unstale":"2023-11-24T06:08:59.830543+0000","last_undegraded":"2023-11-24T06:08:59.830543+0000","last_fullsized":"2023-11-24T06:08:59.830543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:45.776694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337286+0000","last_change":"2023-11-24T06:08:57.524582+0000","last_active":"2023-11-24T06:08:59.337286+0000","last_peered":"2023-11-24T06:08:59.337286+0000","last_clean":"2023-11-24T06:08:59.337286+0000","last_became_active":"2023-11-24T06:08:57.524351+0000","last_became_peered":"2023-11-24T06:08:57.524351+0000","last_unstale":"2023-11-24T06:08:59.337286+0000","last_undegraded":"2023-11-24T06:08:59.337286+0000","last_fullsized":"2023-11-24T06:08:59.337286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:22.630593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829397+0000","last_change":"2023-11-24T06:08:58.528140+0000","last_active":"2023-11-24T06:08:59.829397+0000","last_peered":"2023-11-24T06:08:59.829397+0000","last_clean":"2023-11-24T06:08:59.829397+0000","last_became_active":"2023-11-24T06:08:58.527685+0000","last_became_peered":"2023-11-24T06:08:58.527685+0000","last_unstale":"2023-11-24T06:08:59.829397+0000","last_undegraded":"2023-11-24T06:08:59.829397+0000","last_fullsized":"2023-11-24T06:08:59.829397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:01.910093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343108+0000","last_change":"2023-11-24T06:08:57.519672+0000","last_active":"2023-11-24T06:08:59.343108+0000","last_peered":"2023-11-24T06:08:59.343108+0000","last_clean":"2023-11-24T06:08:59.343108+0000","last_became_active":"2023-11-24T06:08:57.519379+0000","last_became_peered":"2023-11-24T06:08:57.519379+0000","last_unstale":"2023-11-24T06:08:59.343108+0000","last_undegraded":"2023-11-24T06:08:59.343108+0000","last_fullsized":"2023-11-24T06:08:59.343108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:43.426568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830278+0000","last_change":"2023-11-24T06:08:58.531525+0000","last_active":"2023-11-24T06:08:59.830278+0000","last_peered":"2023-11-24T06:08:59.830278+0000","last_clean":"2023-11-24T06:08:59.830278+0000","last_became_active":"2023-11-24T06:08:58.529801+0000","last_became_peered":"2023-11-24T06:08:58.529801+0000","last_unstale":"2023-11-24T06:08:59.830278+0000","last_undegraded":"2023-11-24T06:08:59.830278+0000","last_fullsized":"2023-11-24T06:08:59.830278+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:32.177428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.336436+0000","last_change":"2023-11-24T06:08:57.523021+0000","last_active":"2023-11-24T06:08:59.336436+0000","last_peered":"2023-11-24T06:08:59.336436+0000","last_clean":"2023-11-24T06:08:59.336436+0000","last_became_active":"2023-11-24T06:08:57.522474+0000","last_became_peered":"2023-11-24T06:08:57.522474+0000","last_unstale":"2023-11-24T06:08:59.336436+0000","last_undegraded":"2023-11-24T06:08:59.336436+0000","last_fullsized":"2023-11-24T06:08:59.336436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:43.891317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830650+0000","last_change":"2023-11-24T06:08:58.535871+0000","last_active":"2023-11-24T06:08:59.830650+0000","last_peered":"2023-11-24T06:08:59.830650+0000","last_clean":"2023-11-24T06:08:59.830650+0000","last_became_active":"2023-11-24T06:08:58.535563+0000","last_became_peered":"2023-11-24T06:08:58.535563+0000","last_unstale":"2023-11-24T06:08:59.830650+0000","last_undegraded":"2023-11-24T06:08:59.830650+0000","last_fullsized":"2023-11-24T06:08:59.830650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:55.981203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340214+0000","last_change":"2023-11-24T06:08:57.519605+0000","last_active":"2023-11-24T06:08:59.340214+0000","last_peered":"2023-11-24T06:08:59.340214+0000","last_clean":"2023-11-24T06:08:59.340214+0000","last_became_active":"2023-11-24T06:08:57.519308+0000","last_became_peered":"2023-11-24T06:08:57.519308+0000","last_unstale":"2023-11-24T06:08:59.340214+0000","last_undegraded":"2023-11-24T06:08:59.340214+0000","last_fullsized":"2023-11-24T06:08:59.340214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:17.272538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518512+0000","last_change":"2023-11-24T06:08:58.523994+0000","last_active":"2023-11-24T06:08:59.518512+0000","last_peered":"2023-11-24T06:08:59.518512+0000","last_clean":"2023-11-24T06:08:59.518512+0000","last_became_active":"2023-11-24T06:08:58.523750+0000","last_became_peered":"2023-11-24T06:08:58.523750+0000","last_unstale":"2023-11-24T06:08:59.518512+0000","last_undegraded":"2023-11-24T06:08:59.518512+0000","last_fullsized":"2023-11-24T06:08:59.518512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:45.776694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339437+0000","last_change":"2023-11-24T06:08:57.525492+0000","last_active":"2023-11-24T06:08:59.339437+0000","last_peered":"2023-11-24T06:08:59.339437+0000","last_clean":"2023-11-24T06:08:59.339437+0000","last_became_active":"2023-11-24T06:08:57.525231+0000","last_became_peered":"2023-11-24T06:08:57.525231+0000","last_unstale":"2023-11-24T06:08:59.339437+0000","last_undegraded":"2023-11-24T06:08:59.339437+0000","last_fullsized":"2023-11-24T06:08:59.339437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:35.934471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518156+0000","last_change":"2023-11-24T06:08:58.525233+0000","last_active":"2023-11-24T06:08:59.518156+0000","last_peered":"2023-11-24T06:08:59.518156+0000","last_clean":"2023-11-24T06:08:59.518156+0000","last_became_active":"2023-11-24T06:08:58.524919+0000","last_became_peered":"2023-11-24T06:08:58.524919+0000","last_unstale":"2023-11-24T06:08:59.518156+0000","last_undegraded":"2023-11-24T06:08:59.518156+0000","last_fullsized":"2023-11-24T06:08:59.518156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:56.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516155+0000","last_change":"2023-11-24T06:08:58.521720+0000","last_active":"2023-11-24T06:08:59.516155+0000","last_peered":"2023-11-24T06:08:59.516155+0000","last_clean":"2023-11-24T06:08:59.516155+0000","last_became_active":"2023-11-24T06:08:58.521358+0000","last_became_peered":"2023-11-24T06:08:58.521358+0000","last_unstale":"2023-11-24T06:08:59.516155+0000","last_undegraded":"2023-11-24T06:08:59.516155+0000","last_fullsized":"2023-11-24T06:08:59.516155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343132+0000","last_change":"2023-11-24T06:08:57.521087+0000","last_active":"2023-11-24T06:08:59.343132+0000","last_peered":"2023-11-24T06:08:59.343132+0000","last_clean":"2023-11-24T06:08:59.343132+0000","last_became_active":"2023-11-24T06:08:57.520758+0000","last_became_peered":"2023-11-24T06:08:57.520758+0000","last_unstale":"2023-11-24T06:08:59.343132+0000","last_undegraded":"2023-11-24T06:08:59.343132+0000","last_fullsized":"2023-11-24T06:08:59.343132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:28.366035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829947+0000","last_change":"2023-11-24T06:08:58.531268+0000","last_active":"2023-11-24T06:08:59.829947+0000","last_peered":"2023-11-24T06:08:59.829947+0000","last_clean":"2023-11-24T06:08:59.829947+0000","last_became_active":"2023-11-24T06:08:58.530982+0000","last_became_peered":"2023-11-24T06:08:58.530982+0000","last_unstale":"2023-11-24T06:08:59.829947+0000","last_undegraded":"2023-11-24T06:08:59.829947+0000","last_fullsized":"2023-11-24T06:08:59.829947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:32.599128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339463+0000","last_change":"2023-11-24T06:08:57.519233+0000","last_active":"2023-11-24T06:08:59.339463+0000","last_peered":"2023-11-24T06:08:59.339463+0000","last_clean":"2023-11-24T06:08:59.339463+0000","last_became_active":"2023-11-24T06:08:57.518413+0000","last_became_peered":"2023-11-24T06:08:57.518413+0000","last_unstale":"2023-11-24T06:08:59.339463+0000","last_undegraded":"2023-11-24T06:08:59.339463+0000","last_fullsized":"2023-11-24T06:08:59.339463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:58.516224+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:08:58.516224+0000","last_peered":"2023-11-24T06:08:58.516224+0000","last_clean":"2023-11-24T06:08:58.516224+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:08:58.516224+0000","last_undegraded":"2023-11-24T06:08:58.516224+0000","last_fullsized":"2023-11-24T06:08:58.516224+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829578+0000","last_change":"2023-11-24T06:08:58.528408+0000","last_active":"2023-11-24T06:08:59.829578+0000","last_peered":"2023-11-24T06:08:59.829578+0000","last_clean":"2023-11-24T06:08:59.829578+0000","last_became_active":"2023-11-24T06:08:58.528125+0000","last_became_peered":"2023-11-24T06:08:58.528125+0000","last_unstale":"2023-11-24T06:08:59.829578+0000","last_undegraded":"2023-11-24T06:08:59.829578+0000","last_fullsized":"2023-11-24T06:08:59.829578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:18.279785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.338944+0000","last_change":"2023-11-24T06:08:57.527351+0000","last_active":"2023-11-24T06:08:59.338944+0000","last_peered":"2023-11-24T06:08:59.338944+0000","last_clean":"2023-11-24T06:08:59.338944+0000","last_became_active":"2023-11-24T06:08:57.527085+0000","last_became_peered":"2023-11-24T06:08:57.527085+0000","last_unstale":"2023-11-24T06:08:59.338944+0000","last_undegraded":"2023-11-24T06:08:59.338944+0000","last_fullsized":"2023-11-24T06:08:59.338944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339066+0000","last_change":"2023-11-24T06:08:57.528763+0000","last_active":"2023-11-24T06:08:59.339066+0000","last_peered":"2023-11-24T06:08:59.339066+0000","last_clean":"2023-11-24T06:08:59.339066+0000","last_became_active":"2023-11-24T06:08:57.528469+0000","last_became_peered":"2023-11-24T06:08:57.528469+0000","last_unstale":"2023-11-24T06:08:59.339066+0000","last_undegraded":"2023-11-24T06:08:59.339066+0000","last_fullsized":"2023-11-24T06:08:59.339066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:00.802576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516266+0000","last_change":"2023-11-24T06:08:58.522186+0000","last_active":"2023-11-24T06:08:59.516266+0000","last_peered":"2023-11-24T06:08:59.516266+0000","last_clean":"2023-11-24T06:08:59.516266+0000","last_became_active":"2023-11-24T06:08:58.521844+0000","last_became_peered":"2023-11-24T06:08:58.521844+0000","last_unstale":"2023-11-24T06:08:59.516266+0000","last_undegraded":"2023-11-24T06:08:59.516266+0000","last_fullsized":"2023-11-24T06:08:59.516266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516554+0000","last_change":"2023-11-24T06:08:57.519266+0000","last_active":"2023-11-24T06:08:59.516554+0000","last_peered":"2023-11-24T06:08:59.516554+0000","last_clean":"2023-11-24T06:08:59.516554+0000","last_became_active":"2023-11-24T06:08:57.518952+0000","last_became_peered":"2023-11-24T06:08:57.518952+0000","last_unstale":"2023-11-24T06:08:59.516554+0000","last_undegraded":"2023-11-24T06:08:59.516554+0000","last_fullsized":"2023-11-24T06:08:59.516554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:00.902847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830207+0000","last_change":"2023-11-24T06:08:58.539562+0000","last_active":"2023-11-24T06:08:59.830207+0000","last_peered":"2023-11-24T06:08:59.830207+0000","last_clean":"2023-11-24T06:08:59.830207+0000","last_became_active":"2023-11-24T06:08:58.539268+0000","last_became_peered":"2023-11-24T06:08:58.539268+0000","last_unstale":"2023-11-24T06:08:59.830207+0000","last_undegraded":"2023-11-24T06:08:59.830207+0000","last_fullsized":"2023-11-24T06:08:59.830207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:49:10.129141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.337398+0000","last_change":"2023-11-24T06:08:57.525492+0000","last_active":"2023-11-24T06:08:59.337398+0000","last_peered":"2023-11-24T06:08:59.337398+0000","last_clean":"2023-11-24T06:08:59.337398+0000","last_became_active":"2023-11-24T06:08:57.525231+0000","last_became_peered":"2023-11-24T06:08:57.525231+0000","last_unstale":"2023-11-24T06:08:59.337398+0000","last_undegraded":"2023-11-24T06:08:59.337398+0000","last_fullsized":"2023-11-24T06:08:59.337398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:12.033515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.831038+0000","last_change":"2023-11-24T06:08:58.522482+0000","last_active":"2023-11-24T06:08:59.831038+0000","last_peered":"2023-11-24T06:08:59.831038+0000","last_clean":"2023-11-24T06:08:59.831038+0000","last_became_active":"2023-11-24T06:08:58.522237+0000","last_became_peered":"2023-11-24T06:08:58.522237+0000","last_unstale":"2023-11-24T06:08:59.831038+0000","last_undegraded":"2023-11-24T06:08:59.831038+0000","last_fullsized":"2023-11-24T06:08:59.831038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.338911+0000","last_change":"2023-11-24T06:08:57.525325+0000","last_active":"2023-11-24T06:08:59.338911+0000","last_peered":"2023-11-24T06:08:59.338911+0000","last_clean":"2023-11-24T06:08:59.338911+0000","last_became_active":"2023-11-24T06:08:57.525051+0000","last_became_peered":"2023-11-24T06:08:57.525051+0000","last_unstale":"2023-11-24T06:08:59.338911+0000","last_undegraded":"2023-11-24T06:08:59.338911+0000","last_fullsized":"2023-11-24T06:08:59.338911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:16.569994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517913+0000","last_change":"2023-11-24T06:08:58.534115+0000","last_active":"2023-11-24T06:08:59.517913+0000","last_peered":"2023-11-24T06:08:59.517913+0000","last_clean":"2023-11-24T06:08:59.517913+0000","last_became_active":"2023-11-24T06:08:58.525128+0000","last_became_peered":"2023-11-24T06:08:58.525128+0000","last_unstale":"2023-11-24T06:08:59.517913+0000","last_undegraded":"2023-11-24T06:08:59.517913+0000","last_fullsized":"2023-11-24T06:08:59.517913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:23.637839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340132+0000","last_change":"2023-11-24T06:08:57.530707+0000","last_active":"2023-11-24T06:08:59.340132+0000","last_peered":"2023-11-24T06:08:59.340132+0000","last_clean":"2023-11-24T06:08:59.340132+0000","last_became_active":"2023-11-24T06:08:57.529531+0000","last_became_peered":"2023-11-24T06:08:57.529531+0000","last_unstale":"2023-11-24T06:08:59.340132+0000","last_undegraded":"2023-11-24T06:08:59.340132+0000","last_fullsized":"2023-11-24T06:08:59.340132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:24.997619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829973+0000","last_change":"2023-11-24T06:08:58.523161+0000","last_active":"2023-11-24T06:08:59.829973+0000","last_peered":"2023-11-24T06:08:59.829973+0000","last_clean":"2023-11-24T06:08:59.829973+0000","last_became_active":"2023-11-24T06:08:58.522753+0000","last_became_peered":"2023-11-24T06:08:58.522753+0000","last_unstale":"2023-11-24T06:08:59.829973+0000","last_undegraded":"2023-11-24T06:08:59.829973+0000","last_fullsized":"2023-11-24T06:08:59.829973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340171+0000","last_change":"2023-11-24T06:08:57.521686+0000","last_active":"2023-11-24T06:08:59.340171+0000","last_peered":"2023-11-24T06:08:59.340171+0000","last_clean":"2023-11-24T06:08:59.340171+0000","last_became_active":"2023-11-24T06:08:57.520431+0000","last_became_peered":"2023-11-24T06:08:57.520431+0000","last_unstale":"2023-11-24T06:08:59.340171+0000","last_undegraded":"2023-11-24T06:08:59.340171+0000","last_fullsized":"2023-11-24T06:08:59.340171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:31.591882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830207+0000","last_change":"2023-11-24T06:08:58.519601+0000","last_active":"2023-11-24T06:08:59.830207+0000","last_peered":"2023-11-24T06:08:59.830207+0000","last_clean":"2023-11-24T06:08:59.830207+0000","last_became_active":"2023-11-24T06:08:58.519340+0000","last_became_peered":"2023-11-24T06:08:58.519340+0000","last_unstale":"2023-11-24T06:08:59.830207+0000","last_undegraded":"2023-11-24T06:08:59.830207+0000","last_fullsized":"2023-11-24T06:08:59.830207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:51.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.339001+0000","last_change":"2023-11-24T06:08:57.529494+0000","last_active":"2023-11-24T06:08:59.339001+0000","last_peered":"2023-11-24T06:08:59.339001+0000","last_clean":"2023-11-24T06:08:59.339001+0000","last_became_active":"2023-11-24T06:08:57.529231+0000","last_became_peered":"2023-11-24T06:08:57.529231+0000","last_unstale":"2023-11-24T06:08:59.339001+0000","last_undegraded":"2023-11-24T06:08:59.339001+0000","last_fullsized":"2023-11-24T06:08:59.339001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:06.359234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829452+0000","last_change":"2023-11-24T06:08:58.529994+0000","last_active":"2023-11-24T06:08:59.829452+0000","last_peered":"2023-11-24T06:08:59.829452+0000","last_clean":"2023-11-24T06:08:59.829452+0000","last_became_active":"2023-11-24T06:08:58.529528+0000","last_became_peered":"2023-11-24T06:08:58.529528+0000","last_unstale":"2023-11-24T06:08:59.829452+0000","last_undegraded":"2023-11-24T06:08:59.829452+0000","last_fullsized":"2023-11-24T06:08:59.829452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:32.599128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343170+0000","last_change":"2023-11-24T06:08:57.525323+0000","last_active":"2023-11-24T06:08:59.343170+0000","last_peered":"2023-11-24T06:08:59.343170+0000","last_clean":"2023-11-24T06:08:59.343170+0000","last_became_active":"2023-11-24T06:08:57.525087+0000","last_became_peered":"2023-11-24T06:08:57.525087+0000","last_unstale":"2023-11-24T06:08:59.343170+0000","last_undegraded":"2023-11-24T06:08:59.343170+0000","last_fullsized":"2023-11-24T06:08:59.343170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:30:43.891317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830689+0000","last_change":"2023-11-24T06:08:58.539679+0000","last_active":"2023-11-24T06:08:59.830689+0000","last_peered":"2023-11-24T06:08:59.830689+0000","last_clean":"2023-11-24T06:08:59.830689+0000","last_became_active":"2023-11-24T06:08:58.539279+0000","last_became_peered":"2023-11-24T06:08:58.539279+0000","last_unstale":"2023-11-24T06:08:59.830689+0000","last_undegraded":"2023-11-24T06:08:59.830689+0000","last_fullsized":"2023-11-24T06:08:59.830689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:13.221261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.340249+0000","last_change":"2023-11-24T06:08:57.523738+0000","last_active":"2023-11-24T06:08:59.340249+0000","last_peered":"2023-11-24T06:08:59.340249+0000","last_clean":"2023-11-24T06:08:59.340249+0000","last_became_active":"2023-11-24T06:08:57.523396+0000","last_became_peered":"2023-11-24T06:08:57.523396+0000","last_unstale":"2023-11-24T06:08:59.340249+0000","last_undegraded":"2023-11-24T06:08:59.340249+0000","last_fullsized":"2023-11-24T06:08:59.340249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:38.995471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.518526+0000","last_change":"2023-11-24T06:08:58.520593+0000","last_active":"2023-11-24T06:08:59.518526+0000","last_peered":"2023-11-24T06:08:59.518526+0000","last_clean":"2023-11-24T06:08:59.518526+0000","last_became_active":"2023-11-24T06:08:58.520297+0000","last_became_peered":"2023-11-24T06:08:58.520297+0000","last_unstale":"2023-11-24T06:08:59.518526+0000","last_undegraded":"2023-11-24T06:08:59.518526+0000","last_fullsized":"2023-11-24T06:08:59.518526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:55.981203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.343206+0000","last_change":"2023-11-24T06:08:57.521315+0000","last_active":"2023-11-24T06:08:59.343206+0000","last_peered":"2023-11-24T06:08:59.343206+0000","last_clean":"2023-11-24T06:08:59.343206+0000","last_became_active":"2023-11-24T06:08:57.520718+0000","last_became_peered":"2023-11-24T06:08:57.520718+0000","last_unstale":"2023-11-24T06:08:59.343206+0000","last_undegraded":"2023-11-24T06:08:59.343206+0000","last_fullsized":"2023-11-24T06:08:59.343206+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:56.491864+0000","last_clean_scrub_stamp":"2023-11-24T06:08:56.491864+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:16.353114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830728+0000","last_change":"2023-11-24T06:08:58.542621+0000","last_active":"2023-11-24T06:08:59.830728+0000","last_peered":"2023-11-24T06:08:59.830728+0000","last_clean":"2023-11-24T06:08:59.830728+0000","last_became_active":"2023-11-24T06:08:58.541632+0000","last_became_peered":"2023-11-24T06:08:58.541632+0000","last_unstale":"2023-11-24T06:08:59.830728+0000","last_undegraded":"2023-11-24T06:08:59.830728+0000","last_fullsized":"2023-11-24T06:08:59.830728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:40.188112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829349+0000","last_change":"2023-11-24T06:08:58.527955+0000","last_active":"2023-11-24T06:08:59.829349+0000","last_peered":"2023-11-24T06:08:59.829349+0000","last_clean":"2023-11-24T06:08:59.829349+0000","last_became_active":"2023-11-24T06:08:58.527336+0000","last_became_peered":"2023-11-24T06:08:58.527336+0000","last_unstale":"2023-11-24T06:08:59.829349+0000","last_undegraded":"2023-11-24T06:08:59.829349+0000","last_fullsized":"2023-11-24T06:08:59.829349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:01.809823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830151+0000","last_change":"2023-11-24T06:08:58.530318+0000","last_active":"2023-11-24T06:08:59.830151+0000","last_peered":"2023-11-24T06:08:59.830151+0000","last_clean":"2023-11-24T06:08:59.830151+0000","last_became_active":"2023-11-24T06:08:58.530015+0000","last_became_peered":"2023-11-24T06:08:58.530015+0000","last_unstale":"2023-11-24T06:08:59.830151+0000","last_undegraded":"2023-11-24T06:08:59.830151+0000","last_fullsized":"2023-11-24T06:08:59.830151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:13.040762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517268+0000","last_change":"2023-11-24T06:08:58.522901+0000","last_active":"2023-11-24T06:08:59.517268+0000","last_peered":"2023-11-24T06:08:59.517268+0000","last_clean":"2023-11-24T06:08:59.517268+0000","last_became_active":"2023-11-24T06:08:58.522620+0000","last_became_peered":"2023-11-24T06:08:58.522620+0000","last_unstale":"2023-11-24T06:08:59.517268+0000","last_undegraded":"2023-11-24T06:08:59.517268+0000","last_fullsized":"2023-11-24T06:08:59.517268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:01:07.366480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830900+0000","last_change":"2023-11-24T06:08:58.519217+0000","last_active":"2023-11-24T06:08:59.830900+0000","last_peered":"2023-11-24T06:08:59.830900+0000","last_clean":"2023-11-24T06:08:59.830900+0000","last_became_active":"2023-11-24T06:08:58.518577+0000","last_became_peered":"2023-11-24T06:08:58.518577+0000","last_unstale":"2023-11-24T06:08:59.830900+0000","last_undegraded":"2023-11-24T06:08:59.830900+0000","last_fullsized":"2023-11-24T06:08:59.830900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:22:01.910093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830247+0000","last_change":"2023-11-24T06:08:58.523627+0000","last_active":"2023-11-24T06:08:59.830247+0000","last_peered":"2023-11-24T06:08:59.830247+0000","last_clean":"2023-11-24T06:08:59.830247+0000","last_became_active":"2023-11-24T06:08:58.522543+0000","last_became_peered":"2023-11-24T06:08:58.522543+0000","last_unstale":"2023-11-24T06:08:59.830247+0000","last_undegraded":"2023-11-24T06:08:59.830247+0000","last_fullsized":"2023-11-24T06:08:59.830247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:09.611451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.516517+0000","last_change":"2023-11-24T06:08:58.521811+0000","last_active":"2023-11-24T06:08:59.516517+0000","last_peered":"2023-11-24T06:08:59.516517+0000","last_clean":"2023-11-24T06:08:59.516517+0000","last_became_active":"2023-11-24T06:08:58.521129+0000","last_became_peered":"2023-11-24T06:08:58.521129+0000","last_unstale":"2023-11-24T06:08:59.516517+0000","last_undegraded":"2023-11-24T06:08:59.516517+0000","last_fullsized":"2023-11-24T06:08:59.516517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:51.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829914+0000","last_change":"2023-11-24T06:08:58.539747+0000","last_active":"2023-11-24T06:08:59.829914+0000","last_peered":"2023-11-24T06:08:59.829914+0000","last_clean":"2023-11-24T06:08:59.829914+0000","last_became_active":"2023-11-24T06:08:58.539449+0000","last_became_peered":"2023-11-24T06:08:58.539449+0000","last_unstale":"2023-11-24T06:08:59.829914+0000","last_undegraded":"2023-11-24T06:08:59.829914+0000","last_fullsized":"2023-11-24T06:08:59.829914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830096+0000","last_change":"2023-11-24T06:08:58.523632+0000","last_active":"2023-11-24T06:08:59.830096+0000","last_peered":"2023-11-24T06:08:59.830096+0000","last_clean":"2023-11-24T06:08:59.830096+0000","last_became_active":"2023-11-24T06:08:58.523365+0000","last_became_peered":"2023-11-24T06:08:58.523365+0000","last_unstale":"2023-11-24T06:08:59.830096+0000","last_undegraded":"2023-11-24T06:08:59.830096+0000","last_fullsized":"2023-11-24T06:08:59.830096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517020+0000","last_change":"2023-11-24T06:08:58.527956+0000","last_active":"2023-11-24T06:08:59.517020+0000","last_peered":"2023-11-24T06:08:59.517020+0000","last_clean":"2023-11-24T06:08:59.517020+0000","last_became_active":"2023-11-24T06:08:58.527700+0000","last_became_peered":"2023-11-24T06:08:58.527700+0000","last_unstale":"2023-11-24T06:08:59.517020+0000","last_undegraded":"2023-11-24T06:08:59.517020+0000","last_fullsized":"2023-11-24T06:08:59.517020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:35.785026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830320+0000","last_change":"2023-11-24T06:08:58.529061+0000","last_active":"2023-11-24T06:08:59.830320+0000","last_peered":"2023-11-24T06:08:59.830320+0000","last_clean":"2023-11-24T06:08:59.830320+0000","last_became_active":"2023-11-24T06:08:58.528753+0000","last_became_peered":"2023-11-24T06:08:58.528753+0000","last_unstale":"2023-11-24T06:08:59.830320+0000","last_undegraded":"2023-11-24T06:08:59.830320+0000","last_fullsized":"2023-11-24T06:08:59.830320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:10:32.177428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.517097+0000","last_change":"2023-11-24T06:08:58.541017+0000","last_active":"2023-11-24T06:08:59.517097+0000","last_peered":"2023-11-24T06:08:59.517097+0000","last_clean":"2023-11-24T06:08:59.517097+0000","last_became_active":"2023-11-24T06:08:58.540746+0000","last_became_peered":"2023-11-24T06:08:58.540746+0000","last_unstale":"2023-11-24T06:08:59.517097+0000","last_undegraded":"2023-11-24T06:08:59.517097+0000","last_fullsized":"2023-11-24T06:08:59.517097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:59.790244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.830345+0000","last_change":"2023-11-24T06:08:58.540211+0000","last_active":"2023-11-24T06:08:59.830345+0000","last_peered":"2023-11-24T06:08:59.830345+0000","last_clean":"2023-11-24T06:08:59.830345+0000","last_became_active":"2023-11-24T06:08:58.539978+0000","last_became_peered":"2023-11-24T06:08:58.539978+0000","last_unstale":"2023-11-24T06:08:59.830345+0000","last_undegraded":"2023-11-24T06:08:59.830345+0000","last_fullsized":"2023-11-24T06:08:59.830345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:01.809823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T06:08:59.829744+0000","last_change":"2023-11-24T06:08:58.529024+0000","last_active":"2023-11-24T06:08:59.829744+0000","last_peered":"2023-11-24T06:08:59.829744+0000","last_clean":"2023-11-24T06:08:59.829744+0000","last_became_active":"2023-11-24T06:08:58.528704+0000","last_became_peered":"2023-11-24T06:08:58.528704+0000","last_unstale":"2023-11-24T06:08:59.829744+0000","last_undegraded":"2023-11-24T06:08:59.829744+0000","last_fullsized":"2023-11-24T06:08:59.829744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:57.499111+0000","last_clean_scrub_stamp":"2023-11-24T06:08:57.499111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:36.941717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.245}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:09:07.655 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:09:07.655 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T06:09:07.655 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T06:09:07.655 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T06:09:07.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.809+0000 7f7431529700 1 Processor -- start 2023-11-24T06:09:07.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.809+0000 7f7431529700 1 -- start start 2023-11-24T06:09:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f7431529700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742c15e1e0 0x7f742c0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f7431529700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f7431529700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742c0d20b0 con 0x7f742c0d0db0 2023-11-24T06:09:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f7431529700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742c0d3150 con 0x7f742c15e1e0 2023-11-24T06:09:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f7431529700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0d1240 con 0x7f742c0d3a00 2023-11-24T06:09:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742c15e1e0 0x7f742c0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f742c15e1e0 0x7f742c0d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45518/0 (socket says 172.21.15.97:45518) 2023-11-24T06:09:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742affd700 1 -- 172.21.15.97:0/1336772463 learned_addr learned my addr 172.21.15.97:0/1336772463 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f742c0d0db0 0x7f742c06aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37974/0 (socket says 172.21.15.97:37974) 2023-11-24T06:09:07.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742a7fc700 1 --2- 172.21.15.97:0/1336772463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742affd700 1 -- 172.21.15.97:0/1336772463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 msgr2=0x7f742c0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742affd700 1 --2- 172.21.15.97:0/1336772463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742affd700 1 -- 172.21.15.97:0/1336772463 >> v1:172.21.15.97:6789/0 conn(0x7f742c0d0db0 legacy=0x7f742c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f7429ffb700 1 -- 172.21.15.97:0/1336772463 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181361814 0 0) 0x7f742c0d20b0 con 0x7f742c0d0db0 2023-11-24T06:09:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742affd700 1 -- 172.21.15.97:0/1336772463 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c0d2390 con 0x7f742c15e1e0 2023-11-24T06:09:07.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.810+0000 7f742a7fc700 1 --2- 172.21.15.97:0/1336772463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f742affd700 1 --2- 172.21.15.97:0/1336772463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c15e1e0 0x7f742c0d34c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74200098f0 tx=0x7f7420009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcf7175a50742e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7429ffb700 1 -- 172.21.15.97:0/1336772463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000b8a0 con 0x7f742c15e1e0 2023-11-24T06:09:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7429ffb700 1 -- 172.21.15.97:0/1336772463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742000ba00 con 0x7f742c15e1e0 2023-11-24T06:09:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7429ffb700 1 -- 172.21.15.97:0/1336772463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000bd90 con 0x7f742c15e1e0 2023-11-24T06:09:07.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7431529700 1 -- 172.21.15.97:0/1336772463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c15e1e0 msgr2=0x7f742c0d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7431529700 1 --2- 172.21.15.97:0/1336772463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c15e1e0 0x7f742c0d34c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74200098f0 tx=0x7f7420009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7431529700 1 -- 172.21.15.97:0/1336772463 shutdown_connections 2023-11-24T06:09:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7431529700 1 --2- 172.21.15.97:0/1336772463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c15e1e0 0x7f742c0d34c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7431529700 1 --2- 172.21.15.97:0/1336772463 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7431529700 1 -- 172.21.15.97:0/1336772463 >> 172.21.15.97:0/1336772463 conn(0x7f742c0b5b50 msgr2=0x7f742c0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7431529700 1 -- 172.21.15.97:0/1336772463 shutdown_connections 2023-11-24T06:09:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.811+0000 7f7431529700 1 -- 172.21.15.97:0/1336772463 wait complete. 2023-11-24T06:09:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.813+0000 7f7431529700 1 Processor -- start 2023-11-24T06:09:07.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.813+0000 7f7431529700 1 -- start start 2023-11-24T06:09:07.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.813+0000 7f7431529700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c0d0db0 0x7f742c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.813+0000 7f7431529700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.813+0000 7f7431529700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f742c15e1e0 0x7f742c1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.813+0000 7f7431529700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c152ec0 con 0x7f742c0d0db0 2023-11-24T06:09:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.813+0000 7f7431529700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0cb470 con 0x7f742c15e1e0 2023-11-24T06:09:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.813+0000 7f7431529700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c1430a0 con 0x7f742c0d3a00 2023-11-24T06:09:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c0d0db0 0x7f742c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c0d0db0 0x7f742c0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45524/0 (socket says 172.21.15.97:45524) 2023-11-24T06:09:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 -- 172.21.15.97:0/60319864 learned_addr learned my addr 172.21.15.97:0/60319864 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 -- 172.21.15.97:0/60319864 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 msgr2=0x7f742c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742a7fc700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742b7fe700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f742c15e1e0 0x7f742c1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 -- 172.21.15.97:0/60319864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f742c15e1e0 msgr2=0x7f742c1f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f742c15e1e0 0x7f742c1f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 -- 172.21.15.97:0/60319864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7420009550 con 0x7f742c0d0db0 2023-11-24T06:09:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f742affd700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c0d0db0 0x7f742c0cd3b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7420009c40 tx=0x7f7420004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f74137fe700 1 -- 172.21.15.97:0/60319864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200038a0 con 0x7f742c0d0db0 2023-11-24T06:09:07.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f74137fe700 1 -- 172.21.15.97:0/60319864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742001e070 con 0x7f742c0d0db0 2023-11-24T06:09:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c0cda50 con 0x7f742c0d0db0 2023-11-24T06:09:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f74137fe700 1 -- 172.21.15.97:0/60319864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420003c20 con 0x7f742c0d0db0 2023-11-24T06:09:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.814+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c153110 con 0x7f742c0d0db0 2023-11-24T06:09:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.815+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c0751e0 con 0x7f742c0d0db0 2023-11-24T06:09:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.816+0000 7f74137fe700 1 -- 172.21.15.97:0/60319864 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f742001a420 con 0x7f742c0d0db0 2023-11-24T06:09:07.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.816+0000 7f74137fe700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f741806aed0 0x7f741806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.816+0000 7f74137fe700 1 -- 172.21.15.97:0/60319864 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7420020070 con 0x7f742c0d0db0 2023-11-24T06:09:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.816+0000 7f742a7fc700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f741806aed0 0x7f741806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:07.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.817+0000 7f742a7fc700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f741806aed0 0x7f741806d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f741c006fd0 tx=0x7f741c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:07.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:07.820+0000 7f74137fe700 1 -- 172.21.15.97:0/60319864 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7420055d70 con 0x7f742c0d0db0 2023-11-24T06:09:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.052+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f742c0b5480 con 0x7f742c0d0db0 2023-11-24T06:09:08.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.052+0000 7f74137fe700 1 -- 172.21.15.97:0/60319864 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7420059390 con 0x7f742c0d0db0 2023-11-24T06:09:08.055 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:08.055 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T06:09:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f741806aed0 msgr2=0x7f741806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f741806aed0 0x7f741806d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f741c006fd0 tx=0x7f741c008040 comp rx=0 tx=0).stop 2023-11-24T06:09:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c0d0db0 msgr2=0x7f742c0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c0d0db0 0x7f742c0cd3b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7420009c40 tx=0x7f7420004750 comp rx=0 tx=0).stop 2023-11-24T06:09:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 shutdown_connections 2023-11-24T06:09:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f742c15e1e0 0x7f742c1f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f742c0d0db0 0x7f742c0cd3b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f741806aed0 0x7f741806d390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 --2- 172.21.15.97:0/60319864 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f742c0d3a00 0x7f742c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 >> 172.21.15.97:0/60319864 conn(0x7f742c0b5b50 msgr2=0x7f742c0ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 shutdown_connections 2023-11-24T06:09:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.054+0000 7f7431529700 1 -- 172.21.15.97:0/60319864 wait complete. 2023-11-24T06:09:08.069 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T06:09:08.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.186+0000 7fd835635700 1 Processor -- start 2023-11-24T06:09:08.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.186+0000 7fd835635700 1 -- start start 2023-11-24T06:09:08.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd835635700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd83015e1e0 0x7fd8300d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd835635700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd8300d0f80 0x7fd8300b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd835635700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300d20b0 con 0x7fd8300d3a00 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd835635700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8300d3150 con 0x7fd8300d0f80 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd835635700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83015e5c0 con 0x7fd83015e1e0 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82f7fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd8300d0f80 0x7fd8300b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82effd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd83015e1e0 0x7fd8300d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82f7fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd8300d0f80 0x7fd8300b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:39694/0 (socket says 172.21.15.97:39694) 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82effd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd83015e1e0 0x7fd8300d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:60800/0 (socket says 172.21.15.97:60800) 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82f7fe700 1 -- 172.21.15.97:0/1475666841 learned_addr learned my addr 172.21.15.97:0/1475666841 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82effd700 1 -- 172.21.15.97:0/1475666841 >> v2:172.21.15.183:3300/0 conn(0x7fd8300d0f80 msgr2=0x7fd8300b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82effd700 1 --2- 172.21.15.97:0/1475666841 >> v2:172.21.15.183:3300/0 conn(0x7fd8300d0f80 0x7fd8300b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82effd700 1 -- 172.21.15.97:0/1475666841 >> v1:172.21.15.183:6789/0 conn(0x7fd8300d3a00 legacy=0x7fd8300d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82effd700 1 -- 172.21.15.97:0/1475666841 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300d2390 con 0x7fd83015e1e0 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82f7fe700 1 --2- 172.21.15.97:0/1475666841 >> v2:172.21.15.183:3300/0 conn(0x7fd8300d0f80 0x7fd8300b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.187+0000 7fd82effd700 1 --2- 172.21.15.97:0/1475666841 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd83015e1e0 0x7fd8300d34c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd820009840 tx=0x7fd820009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c866277a33d549b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd82dffb700 1 -- 172.21.15.97:0/1475666841 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820011830 con 0x7fd83015e1e0 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd82dffb700 1 -- 172.21.15.97:0/1475666841 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd820011990 con 0x7fd83015e1e0 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd82dffb700 1 -- 172.21.15.97:0/1475666841 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd820011d20 con 0x7fd83015e1e0 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd835635700 1 -- 172.21.15.97:0/1475666841 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd83015e1e0 msgr2=0x7fd8300d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd835635700 1 --2- 172.21.15.97:0/1475666841 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd83015e1e0 0x7fd8300d34c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd820009840 tx=0x7fd820009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd835635700 1 -- 172.21.15.97:0/1475666841 shutdown_connections 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd835635700 1 --2- 172.21.15.97:0/1475666841 >> v2:172.21.15.183:3300/0 conn(0x7fd8300d0f80 0x7fd8300b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd835635700 1 --2- 172.21.15.97:0/1475666841 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd83015e1e0 0x7fd8300d34c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd835635700 1 -- 172.21.15.97:0/1475666841 >> 172.21.15.97:0/1475666841 conn(0x7fd8300b5b50 msgr2=0x7fd8300b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:08.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd835635700 1 -- 172.21.15.97:0/1475666841 shutdown_connections 2023-11-24T06:09:08.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.188+0000 7fd835635700 1 -- 172.21.15.97:0/1475666841 wait complete. 2023-11-24T06:09:08.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd835635700 1 Processor -- start 2023-11-24T06:09:08.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd835635700 1 -- start start 2023-11-24T06:09:08.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd835635700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8300d0f80 0x7fd8300cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd835635700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8300d3a00 0x7fd83015a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd835635700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd83015e1e0 0x7fd8300c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd835635700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300cdb40 con 0x7fd83015e1e0 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd835635700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300ca700 con 0x7fd8300d0f80 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd835635700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300ccf30 con 0x7fd8300d3a00 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd82f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd83015e1e0 0x7fd8300c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd82f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd83015e1e0 0x7fd8300c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45540/0 (socket says 172.21.15.97:45540) 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.190+0000 7fd82f7fe700 1 -- 172.21.15.97:0/575219302 learned_addr learned my addr 172.21.15.97:0/575219302 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82e7fc700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8300d3a00 0x7fd83015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82effd700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8300d0f80 0x7fd8300cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82f7fe700 1 -- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8300d3a00 msgr2=0x7fd83015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82f7fe700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8300d3a00 0x7fd83015a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82f7fe700 1 -- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8300d0f80 msgr2=0x7fd8300cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82f7fe700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8300d0f80 0x7fd8300cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82f7fe700 1 -- 172.21.15.97:0/575219302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828009b30 con 0x7fd83015e1e0 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82e7fc700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8300d3a00 0x7fd83015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82effd700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8300d0f80 0x7fd8300cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd82f7fe700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd83015e1e0 0x7fd8300c8a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd82800ef90 tx=0x7fd828009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd8177fe700 1 -- 172.21.15.97:0/575219302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828018070 con 0x7fd83015e1e0 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd8177fe700 1 -- 172.21.15.97:0/575219302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800f040 con 0x7fd83015e1e0 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8200094a0 con 0x7fd83015e1e0 2023-11-24T06:09:08.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd8177fe700 1 -- 172.21.15.97:0/575219302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828003c60 con 0x7fd83015e1e0 2023-11-24T06:09:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.191+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8300cd4f0 con 0x7fd83015e1e0 2023-11-24T06:09:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.192+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8300cdcb0 con 0x7fd83015e1e0 2023-11-24T06:09:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.192+0000 7fd8177fe700 1 -- 172.21.15.97:0/575219302 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd82800c9e0 con 0x7fd83015e1e0 2023-11-24T06:09:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.193+0000 7fd8177fe700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd81c06aed0 0x7fd81c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.193+0000 7fd8177fe700 1 -- 172.21.15.97:0/575219302 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd828089890 con 0x7fd83015e1e0 2023-11-24T06:09:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.193+0000 7fd82effd700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd81c06aed0 0x7fd81c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.193+0000 7fd82effd700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd81c06aed0 0x7fd81c06d390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd820014040 tx=0x7fd820008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.197+0000 7fd8177fe700 1 -- 172.21.15.97:0/575219302 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd828054950 con 0x7fd83015e1e0 2023-11-24T06:09:08.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.390+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8300751e0 con 0x7fd83015e1e0 2023-11-24T06:09:08.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.391+0000 7fd8177fe700 1 -- 172.21.15.97:0/575219302 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7fd828057f70 con 0x7fd83015e1e0 2023-11-24T06:09:08.394 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:08.394 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd81c06aed0 msgr2=0x7fd81c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd81c06aed0 0x7fd81c06d390 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd820014040 tx=0x7fd820008030 comp rx=0 tx=0).stop 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd83015e1e0 msgr2=0x7fd8300c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd83015e1e0 0x7fd8300c8a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd82800ef90 tx=0x7fd828009f90 comp rx=0 tx=0).stop 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 shutdown_connections 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd83015e1e0 0x7fd8300c8a20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8300d0f80 0x7fd8300cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd81c06aed0 0x7fd81c06d390 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 --2- 172.21.15.97:0/575219302 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8300d3a00 0x7fd83015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.393+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 >> 172.21.15.97:0/575219302 conn(0x7fd8300b5b50 msgr2=0x7fd83015f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:08.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.394+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 shutdown_connections 2023-11-24T06:09:08.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.394+0000 7fd835635700 1 -- 172.21.15.97:0/575219302 wait complete. 2023-11-24T06:09:08.409 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:08.409 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:08.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.529+0000 7f0f28819700 1 Processor -- start 2023-11-24T06:09:08.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.530+0000 7f0f28819700 1 -- start start 2023-11-24T06:09:08.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.530+0000 7f0f28819700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0f241587f0 0x7f0f2406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.530+0000 7f0f28819700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f240b4370 0x7f0f2415b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.530+0000 7f0f28819700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2406b530 con 0x7f0f2415b370 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f28819700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f240ae5e0 con 0x7f0f241587f0 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f28819700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2415be30 con 0x7f0f240b4370 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f21d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0f2415b370 0x7f0f240b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:37998/0 (socket says 172.21.15.97:37998) 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f21d9b700 1 -- 172.21.15.97:0/2113902736 learned_addr learned my addr 172.21.15.97:0/2113902736 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2259c700 1 --2- 172.21.15.97:0/2113902736 >> v2:172.21.15.183:3300/0 conn(0x7f0f241587f0 0x7f0f2406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f22d9d700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f240b4370 0x7f0f2415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626863166 0 0) 0x7f0f2406b530 con 0x7f0f2415b370 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f0c0057c0 con 0x7f0f2415b370 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3006894031 0 0) 0x7f0f0c0057c0 con 0x7f0f2415b370 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f240b4370 msgr2=0x7f0f2415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f240b4370 0x7f0f2415b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 >> v2:172.21.15.183:3300/0 conn(0x7f0f241587f0 msgr2=0x7f0f2406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:08.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 --2- 172.21.15.97:0/2113902736 >> v2:172.21.15.183:3300/0 conn(0x7f0f241587f0 0x7f0f2406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2415c110 con 0x7f0f2415b370 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f0f14001670 con 0x7f0f2415b370 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 >> v1:172.21.15.97:6789/0 conn(0x7f0f2415b370 legacy=0x7f0f240b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f0c005c10 0x7f0f0c006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f240b4370 0x7f0f0c006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c006fa0 0x7f0f0c007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240ae5e0 con 0x7f0f0c005c10 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2415be30 con 0x7f0f0c006fa0 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c00dc20 con 0x7f0f240b4370 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f2259c700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f0c005c10 0x7f0f0c006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f21d9b700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f240b4370 0x7f0f0c006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.531+0000 7f0f22d9d700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c006fa0 0x7f0f0c007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2259c700 1 -- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f240b4370 msgr2=0x7f0f0c006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2259c700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f240b4370 0x7f0f0c006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2259c700 1 -- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c006fa0 msgr2=0x7f0f0c007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2259c700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f0c006fa0 0x7f0f0c007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2259c700 1 -- 172.21.15.97:0/2113902736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0057c0 con 0x7f0f0c005c10 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2259c700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f0c005c10 0x7f0f0c006030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009b90 tx=0x7f0f0c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabfdb9c88d94d13 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1800e6c0 con 0x7f0f0c005c10 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1800e820 con 0x7f0f0c005c10 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f2159a700 1 -- 172.21.15.97:0/2113902736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1800eb90 con 0x7f0f0c005c10 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f28819700 1 -- 172.21.15.97:0/2113902736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f0c005c10 msgr2=0x7f0f0c006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f28819700 1 --2- 172.21.15.97:0/2113902736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f0c005c10 0x7f0f0c006030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009b90 tx=0x7f0f0c007a90 comp rx=0 tx=0).stop 2023-11-24T06:09:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f22d9d700 1 -- 172.21.15.97:0/2113902736 reap_dead start 2023-11-24T06:09:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f28819700 1 -- 172.21.15.97:0/2113902736 shutdown_connections 2023-11-24T06:09:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.532+0000 7f0f28819700 1 -- 172.21.15.97:0/2113902736 >> 172.21.15.97:0/2113902736 conn(0x7f0f240d0bf0 msgr2=0x7f0f240d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.533+0000 7f0f28819700 1 -- 172.21.15.97:0/2113902736 shutdown_connections 2023-11-24T06:09:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.533+0000 7f0f28819700 1 -- 172.21.15.97:0/2113902736 wait complete. 2023-11-24T06:09:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.534+0000 7f0f28819700 1 Processor -- start 2023-11-24T06:09:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.534+0000 7f0f28819700 1 -- start start 2023-11-24T06:09:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f28819700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f0c005c10 0x7f0f24159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.542 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f28819700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f240afac0 0x7f0f2415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f28819700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f240aff40 0x7f0f2415ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f28819700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2415cf50 con 0x7f0f240aff40 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f28819700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2415d450 con 0x7f0f240afac0 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f28819700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2415d170 con 0x7f0f0c005c10 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f240aff40 0x7f0f2415ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f240aff40 0x7f0f2415ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45560/0 (socket says 172.21.15.97:45560) 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 -- 172.21.15.97:0/2335120921 learned_addr learned my addr 172.21.15.97:0/2335120921 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f21d9b700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f240afac0 0x7f0f2415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f2259c700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f0c005c10 0x7f0f24159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 -- 172.21.15.97:0/2335120921 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f0c005c10 msgr2=0x7f0f24159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f0c005c10 0x7f0f24159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 -- 172.21.15.97:0/2335120921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f240afac0 msgr2=0x7f0f2415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f240afac0 0x7f0f2415c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 -- 172.21.15.97:0/2335120921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f240aefe0 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f22d9d700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f240aff40 0x7f0f2415ca10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f24072de0 tx=0x7f0f1c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f12ffd700 1 -- 172.21.15.97:0/2335120921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00e040 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f12ffd700 1 -- 172.21.15.97:0/2335120921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00f040 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f12ffd700 1 -- 172.21.15.97:0/2335120921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00fc40 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f240af2c0 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.535+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2415a500 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.536+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f24068fb0 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.537+0000 7f0f12ffd700 1 -- 172.21.15.97:0/2335120921 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f1c015430 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.537+0000 7f0f12ffd700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0f0406aed0 0x7f0f0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.537+0000 7f0f12ffd700 1 -- 172.21.15.97:0/2335120921 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f1c089900 con 0x7f0f240aff40 2023-11-24T06:09:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.537+0000 7f0f2259c700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0f0406aed0 0x7f0f0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.538+0000 7f0f2259c700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0f0406aed0 0x7f0f0406d390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f180090b0 tx=0x7f0f1801f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.542+0000 7f0f12ffd700 1 -- 172.21.15.97:0/2335120921 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f1c054a40 con 0x7f0f240aff40 2023-11-24T06:09:08.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.731+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f24074b20 con 0x7f0f240aff40 2023-11-24T06:09:08.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.731+0000 7f0f12ffd700 1 -- 172.21.15.97:0/2335120921 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f1c058060 con 0x7f0f240aff40 2023-11-24T06:09:08.734 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:08.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0f0406aed0 msgr2=0x7f0f0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0f0406aed0 0x7f0f0406d390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f180090b0 tx=0x7f0f1801f040 comp rx=0 tx=0).stop 2023-11-24T06:09:08.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f240aff40 msgr2=0x7f0f2415ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f240aff40 0x7f0f2415ca10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f24072de0 tx=0x7f0f1c011ed0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 shutdown_connections 2023-11-24T06:09:08.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f240aff40 0x7f0f2415ca10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0f0406aed0 0x7f0f0406d390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0f0c005c10 0x7f0f24159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 --2- 172.21.15.97:0/2335120921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f240afac0 0x7f0f2415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.733+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 >> 172.21.15.97:0/2335120921 conn(0x7f0f240d0bf0 msgr2=0x7f0f24073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.734+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 shutdown_connections 2023-11-24T06:09:08.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.734+0000 7f0f28819700 1 -- 172.21.15.97:0/2335120921 wait complete. 2023-11-24T06:09:08.749 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.869+0000 7efd377d6700 1 Processor -- start 2023-11-24T06:09:08.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.869+0000 7efd377d6700 1 -- start start 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd377d6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd300746a0 0x7efd300b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd377d6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd300b3c60 con 0x7efd3015e160 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd377d6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd300b58c0 con 0x7efd300afc90 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd377d6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b4500 con 0x7efd300746a0 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd35572700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efd3015e160 0x7efd30073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38016/0 (socket says 172.21.15.97:38016) 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd35572700 1 -- 172.21.15.97:0/3376603175 learned_addr learned my addr 172.21.15.97:0/3376603175 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd34d71700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd300746a0 0x7efd300b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889199999 0 0) 0x7efd300b3c60 con 0x7efd3015e160 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.870+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd1c0035b0 con 0x7efd3015e160 2023-11-24T06:09:08.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 646444940 0 0) 0x7efd1c0035b0 con 0x7efd3015e160 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd300746a0 msgr2=0x7efd300b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd300746a0 0x7efd300b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 >> v1:172.21.15.183:6789/0 conn(0x7efd300afc90 legacy=0x7efd3006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300b0140 con 0x7efd3015e160 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7efd2c001670 con 0x7efd3015e160 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 >> v1:172.21.15.97:6789/0 conn(0x7efd3015e160 legacy=0x7efd30073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd300746a0 0x7efd1c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c003fb0 0x7efd1c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c004d90 0x7efd1c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd300b4500 con 0x7efd1c004d90 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00baf0 con 0x7efd1c003fb0 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00bcb0 con 0x7efd300746a0 2023-11-24T06:09:08.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd35d73700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c004d90 0x7efd1c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd35572700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd300746a0 0x7efd1c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd34d71700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c003fb0 0x7efd1c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd35d73700 1 -- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd300746a0 msgr2=0x7efd1c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd35d73700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd300746a0 0x7efd1c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd35d73700 1 -- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c003fb0 msgr2=0x7efd1c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd35d73700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd1c003fb0 0x7efd1c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd35d73700 1 -- 172.21.15.97:0/3376603175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c0035b0 con 0x7efd1c004d90 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd35d73700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c004d90 0x7efd1c005210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c190 tx=0x7efd1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdbb324ba308595 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28001db0 con 0x7efd1c004d90 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2800ccd0 con 0x7efd1c004d90 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.871+0000 7efd27fff700 1 -- 172.21.15.97:0/3376603175 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28003c40 con 0x7efd1c004d90 2023-11-24T06:09:08.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.872+0000 7efd377d6700 1 -- 172.21.15.97:0/3376603175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c004d90 msgr2=0x7efd1c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.872+0000 7efd377d6700 1 --2- 172.21.15.97:0/3376603175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd1c004d90 0x7efd1c005210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c190 tx=0x7efd1c005880 comp rx=0 tx=0).stop 2023-11-24T06:09:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.872+0000 7efd35d73700 1 -- 172.21.15.97:0/3376603175 reap_dead start 2023-11-24T06:09:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.872+0000 7efd377d6700 1 -- 172.21.15.97:0/3376603175 shutdown_connections 2023-11-24T06:09:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.872+0000 7efd377d6700 1 -- 172.21.15.97:0/3376603175 >> 172.21.15.97:0/3376603175 conn(0x7efd300d0bf0 msgr2=0x7efd300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.872+0000 7efd377d6700 1 -- 172.21.15.97:0/3376603175 shutdown_connections 2023-11-24T06:09:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.872+0000 7efd377d6700 1 -- 172.21.15.97:0/3376603175 wait complete. 2023-11-24T06:09:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.873+0000 7efd377d6700 1 Processor -- start 2023-11-24T06:09:08.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.873+0000 7efd377d6700 1 -- start start 2023-11-24T06:09:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd377d6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd1c004d90 0x7efd300c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd377d6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd3015e770 0x7efd3014e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd377d6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30160040 0x7efd301604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd377d6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3015efa0 con 0x7efd30160040 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd377d6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3015f280 con 0x7efd3015e770 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd377d6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3015f560 con 0x7efd1c004d90 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30160040 0x7efd301604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30160040 0x7efd301604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45580/0 (socket says 172.21.15.97:45580) 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 -- 172.21.15.97:0/4135633867 learned_addr learned my addr 172.21.15.97:0/4135633867 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35572700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd1c004d90 0x7efd300c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd34d71700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd3015e770 0x7efd3014e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 -- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd1c004d90 msgr2=0x7efd300c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd1c004d90 0x7efd300c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 -- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd3015e770 msgr2=0x7efd3014e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd3015e770 0x7efd3014e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:08.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 -- 172.21.15.97:0/4135633867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3015f840 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd34d71700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd3015e770 0x7efd3014e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.874+0000 7efd35d73700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30160040 0x7efd301604d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd28001220 tx=0x7efd280014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.875+0000 7efd25ffb700 1 -- 172.21.15.97:0/4135633867 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28003a00 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.875+0000 7efd25ffb700 1 -- 172.21.15.97:0/4135633867 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2800cad0 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.875+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3015fb20 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.875+0000 7efd25ffb700 1 -- 172.21.15.97:0/4135633867 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd280052e0 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.875+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3015ebf0 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.876+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3015fe90 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.876+0000 7efd25ffb700 1 -- 172.21.15.97:0/4135633867 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd28019070 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.876+0000 7efd25ffb700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd1406aed0 0x7efd1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.876+0000 7efd25ffb700 1 -- 172.21.15.97:0/4135633867 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd2808a1d0 con 0x7efd30160040 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.876+0000 7efd35572700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd1406aed0 0x7efd1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:08.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.877+0000 7efd35572700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd1406aed0 0x7efd1406d390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efd2c001050 tx=0x7efd2c001470 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:08.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:08.881+0000 7efd25ffb700 1 -- 172.21.15.97:0/4135633867 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd28055310 con 0x7efd30160040 2023-11-24T06:09:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.069+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd3015fcb0 con 0x7efd30160040 2023-11-24T06:09:09.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.070+0000 7efd25ffb700 1 -- 172.21.15.97:0/4135633867 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd2805de60 con 0x7efd30160040 2023-11-24T06:09:09.073 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:09.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd1406aed0 msgr2=0x7efd1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:09.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd1406aed0 0x7efd1406d390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efd2c001050 tx=0x7efd2c001470 comp rx=0 tx=0).stop 2023-11-24T06:09:09.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30160040 msgr2=0x7efd301604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:09.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30160040 0x7efd301604d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efd28001220 tx=0x7efd280014a0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 shutdown_connections 2023-11-24T06:09:09.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd30160040 0x7efd301604d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd1406aed0 0x7efd1406d390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efd1c004d90 0x7efd300c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 --2- 172.21.15.97:0/4135633867 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efd3015e770 0x7efd3014e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 >> 172.21.15.97:0/4135633867 conn(0x7efd300d0bf0 msgr2=0x7efd3015ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 shutdown_connections 2023-11-24T06:09:09.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.072+0000 7efd377d6700 1 -- 172.21.15.97:0/4135633867 wait complete. 2023-11-24T06:09:09.087 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:09.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.207+0000 7fb5cd701700 1 Processor -- start 2023-11-24T06:09:09.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.207+0000 7fb5cd701700 1 -- start start 2023-11-24T06:09:09.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.207+0000 7fb5cd701700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5c815bec0 0x7fb5c806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.207+0000 7fb5cd701700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c80702e0 0x7fb5c80d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.207+0000 7fb5cd701700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb5c80d2000 0x7fb5c815f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.207+0000 7fb5cd701700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80cc4a0 con 0x7fb5c815bec0 2023-11-24T06:09:09.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.207+0000 7fb5cd701700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c8070740 con 0x7fb5c80d2000 2023-11-24T06:09:09.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.207+0000 7fb5cd701700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80d2470 con 0x7fb5c80702e0 2023-11-24T06:09:09.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5c815bec0 0x7fb5c806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5c815bec0 0x7fb5c806fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45584/0 (socket says 172.21.15.97:45584) 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 -- 172.21.15.97:0/1914565687 learned_addr learned my addr 172.21.15.97:0/1914565687 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c67fc700 1 --2- 172.21.15.97:0/1914565687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c80702e0 0x7fb5c80d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c77fe700 1 --2- 172.21.15.97:0/1914565687 >> v2:172.21.15.183:3300/0 conn(0x7fb5c80d2000 0x7fb5c815f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 -- 172.21.15.97:0/1914565687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c80702e0 msgr2=0x7fb5c80d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 --2- 172.21.15.97:0/1914565687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c80702e0 0x7fb5c80d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 -- 172.21.15.97:0/1914565687 >> v2:172.21.15.183:3300/0 conn(0x7fb5c80d2000 msgr2=0x7fb5c815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 --2- 172.21.15.97:0/1914565687 >> v2:172.21.15.183:3300/0 conn(0x7fb5c80d2000 0x7fb5c815f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 -- 172.21.15.97:0/1914565687 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80d03e0 con 0x7fb5c815bec0 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c6ffd700 1 --2- 172.21.15.97:0/1914565687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c815bec0 0x7fb5c806fda0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80cc340 tx=0x7fb5b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f827b8700ee0710e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c5ffb700 1 -- 172.21.15.97:0/1914565687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800b870 con 0x7fb5c815bec0 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c5ffb700 1 -- 172.21.15.97:0/1914565687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b800b9d0 con 0x7fb5c815bec0 2023-11-24T06:09:09.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.208+0000 7fb5c5ffb700 1 -- 172.21.15.97:0/1914565687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b800bd60 con 0x7fb5c815bec0 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 -- 172.21.15.97:0/1914565687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c815bec0 msgr2=0x7fb5c806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 --2- 172.21.15.97:0/1914565687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c815bec0 0x7fb5c806fda0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5c80cc340 tx=0x7fb5b8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 -- 172.21.15.97:0/1914565687 shutdown_connections 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 --2- 172.21.15.97:0/1914565687 >> v2:172.21.15.183:3300/0 conn(0x7fb5c80d2000 0x7fb5c815f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 --2- 172.21.15.97:0/1914565687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c815bec0 0x7fb5c806fda0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 --2- 172.21.15.97:0/1914565687 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c80702e0 0x7fb5c80d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 -- 172.21.15.97:0/1914565687 >> 172.21.15.97:0/1914565687 conn(0x7fb5c80730a0 msgr2=0x7fb5c809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 -- 172.21.15.97:0/1914565687 shutdown_connections 2023-11-24T06:09:09.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.209+0000 7fb5cd701700 1 -- 172.21.15.97:0/1914565687 wait complete. 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.210+0000 7fb5cd701700 1 Processor -- start 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.210+0000 7fb5cd701700 1 -- start start 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5cd701700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5c80702e0 0x7fb5c80cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5cd701700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c80d2000 0x7fb5c8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5cd701700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c815bec0 0x7fb5c80c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5cd701700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80cb340 con 0x7fb5c80d2000 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5cd701700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80ca730 con 0x7fb5c80702e0 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5cd701700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80caa10 con 0x7fb5c815bec0 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c80d2000 0x7fb5c8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c80d2000 0x7fb5c8155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45592/0 (socket says 172.21.15.97:45592) 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 -- 172.21.15.97:0/2696290726 learned_addr learned my addr 172.21.15.97:0/2696290726 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:09.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c6ffd700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5c80702e0 0x7fb5c80cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c77fe700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c815bec0 0x7fb5c80c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 -- 172.21.15.97:0/2696290726 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c815bec0 msgr2=0x7fb5c80c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c815bec0 0x7fb5c80c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 -- 172.21.15.97:0/2696290726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5c80702e0 msgr2=0x7fb5c80cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5c80702e0 0x7fb5c80cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 -- 172.21.15.97:0/2696290726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8009550 con 0x7fb5c80d2000 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.211+0000 7fb5c67fc700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c80d2000 0x7fb5c8155d60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d800 tx=0x7fb5bc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.212+0000 7fb5af7fe700 1 -- 172.21.15.97:0/2696290726 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0125f0 con 0x7fb5c80d2000 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.212+0000 7fb5af7fe700 1 -- 172.21.15.97:0/2696290726 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc012750 con 0x7fb5c80d2000 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.212+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c80cacf0 con 0x7fb5c80d2000 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.212+0000 7fb5af7fe700 1 -- 172.21.15.97:0/2696290726 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc012ae0 con 0x7fb5c80d2000 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.212+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c81562a0 con 0x7fb5c80d2000 2023-11-24T06:09:09.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.212+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c80cb620 con 0x7fb5c80d2000 2023-11-24T06:09:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.213+0000 7fb5af7fe700 1 -- 172.21.15.97:0/2696290726 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb5bc012c40 con 0x7fb5c80d2000 2023-11-24T06:09:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.213+0000 7fb5af7fe700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5b406aed0 0x7fb5b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.213+0000 7fb5af7fe700 1 -- 172.21.15.97:0/2696290726 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5bc089670 con 0x7fb5c80d2000 2023-11-24T06:09:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.213+0000 7fb5c6ffd700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5b406aed0 0x7fb5b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.214+0000 7fb5c6ffd700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5b406aed0 0x7fb5b406d390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8000c00 tx=0x7fb5b8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:09.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.218+0000 7fb5af7fe700 1 -- 172.21.15.97:0/2696290726 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5bc054750 con 0x7fb5c80d2000 2023-11-24T06:09:09.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.409+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5c80cb480 con 0x7fb5c80d2000 2023-11-24T06:09:09.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.410+0000 7fb5af7fe700 1 -- 172.21.15.97:0/2696290726 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5c80cb480 con 0x7fb5c80d2000 2023-11-24T06:09:09.413 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5b406aed0 msgr2=0x7fb5b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5b406aed0 0x7fb5b406d390 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8000c00 tx=0x7fb5b8008030 comp rx=0 tx=0).stop 2023-11-24T06:09:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c80d2000 msgr2=0x7fb5c8155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c80d2000 0x7fb5c8155d60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00d800 tx=0x7fb5bc00db10 comp rx=0 tx=0).stop 2023-11-24T06:09:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 shutdown_connections 2023-11-24T06:09:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5b406aed0 0x7fb5b406d390 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5c815bec0 0x7fb5c80c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5c80702e0 0x7fb5c80cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 --2- 172.21.15.97:0/2696290726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5c80d2000 0x7fb5c8155d60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:09.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 >> 172.21.15.97:0/2696290726 conn(0x7fb5c80730a0 msgr2=0x7fb5c80cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:09.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 shutdown_connections 2023-11-24T06:09:09.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.412+0000 7fb5cd701700 1 -- 172.21.15.97:0/2696290726 wait complete. 2023-11-24T06:09:09.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:09.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.545+0000 7f3339832700 1 Processor -- start 2023-11-24T06:09:09.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.545+0000 7f3339832700 1 -- start start 2023-11-24T06:09:09.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3339832700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 0x7f33340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3339832700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f333406c710 0x7f33340afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:09.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3339832700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33340ae430 con 0x7f33340af4d0 2023-11-24T06:09:09.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3339832700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33340af0b0 con 0x7f333406c710 2023-11-24T06:09:09.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3339832700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b05a0 con 0x7f333415e080 2023-11-24T06:09:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f33327fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f33340af4d0 0x7f333406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38054/0 (socket says 172.21.15.97:38054) 2023-11-24T06:09:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f33327fc700 1 -- 172.21.15.97:0/3792767797 learned_addr learned my addr 172.21.15.97:0/3792767797 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:13.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3332ffd700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 0x7f33340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653728556 0 0) 0x7f33340ae430 con 0x7f33340af4d0 2023-11-24T06:09:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f331c0035b0 con 0x7f33340af4d0 2023-11-24T06:09:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f33337fe700 1 --2- 172.21.15.97:0/3792767797 >> v2:172.21.15.183:3300/0 conn(0x7f333406c710 0x7f33340afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202775829 0 0) 0x7f331c0035b0 con 0x7f33340af4d0 2023-11-24T06:09:13.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 msgr2=0x7f33340b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 0x7f33340b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 >> v2:172.21.15.183:3300/0 conn(0x7f333406c710 msgr2=0x7f33340afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 --2- 172.21.15.97:0/3792767797 >> v2:172.21.15.183:3300/0 conn(0x7f333406c710 0x7f33340afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340ae710 con 0x7f33340af4d0 2023-11-24T06:09:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.546+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f3324001670 con 0x7f33340af4d0 2023-11-24T06:09:13.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 >> v1:172.21.15.97:6789/0 conn(0x7f33340af4d0 legacy=0x7f333406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f331c003a30 0x7f331c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 0x7f331c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f331c004d90 0x7f331c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340af0b0 con 0x7f331c003a30 2023-11-24T06:09:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b05a0 con 0x7f331c004d90 2023-11-24T06:09:13.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c00ba10 con 0x7f333415e080 2023-11-24T06:09:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3332ffd700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f331c003a30 0x7f331c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f33327fc700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 0x7f331c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f33337fe700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f331c004d90 0x7f331c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3332ffd700 1 -- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 msgr2=0x7f331c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3332ffd700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 0x7f331c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3332ffd700 1 -- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f331c004d90 msgr2=0x7f331c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3332ffd700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f331c004d90 0x7f331c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3332ffd700 1 -- 172.21.15.97:0/3792767797 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0035b0 con 0x7f331c003a30 2023-11-24T06:09:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f33337fe700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f331c004d90 0x7f331c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f33327fc700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f333415e080 0x7f331c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:13.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3332ffd700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f331c003a30 0x7f331c003e50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3328009b90 tx=0x7f331c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bd7222b22e5f29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800e6c0 con 0x7f331c003a30 2023-11-24T06:09:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332800e820 con 0x7f331c003a30 2023-11-24T06:09:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.547+0000 7f3331ffb700 1 -- 172.21.15.97:0/3792767797 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332800eb90 con 0x7f331c003a30 2023-11-24T06:09:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.548+0000 7f3339832700 1 -- 172.21.15.97:0/3792767797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f331c003a30 msgr2=0x7f331c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.548+0000 7f3339832700 1 --2- 172.21.15.97:0/3792767797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f331c003a30 0x7f331c003e50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3328009b90 tx=0x7f331c005880 comp rx=0 tx=0).stop 2023-11-24T06:09:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.548+0000 7f33337fe700 1 -- 172.21.15.97:0/3792767797 reap_dead start 2023-11-24T06:09:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.548+0000 7f3339832700 1 -- 172.21.15.97:0/3792767797 shutdown_connections 2023-11-24T06:09:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.548+0000 7f3339832700 1 -- 172.21.15.97:0/3792767797 >> 172.21.15.97:0/3792767797 conn(0x7f3334071b90 msgr2=0x7f3334071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.548+0000 7f3339832700 1 -- 172.21.15.97:0/3792767797 shutdown_connections 2023-11-24T06:09:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.548+0000 7f3339832700 1 -- 172.21.15.97:0/3792767797 wait complete. 2023-11-24T06:09:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.549+0000 7f3339832700 1 Processor -- start 2023-11-24T06:09:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.549+0000 7f3339832700 1 -- start start 2023-11-24T06:09:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f3339832700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f331c003a30 0x7f333414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f3339832700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33340cf770 0x7f33340c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f3339832700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333414f930 0x7f33340cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f3339832700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333414fdc0 con 0x7f333414f930 2023-11-24T06:09:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f3339832700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340ce9b0 con 0x7f33340cf770 2023-11-24T06:09:13.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f3339832700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33340cec90 con 0x7f331c003a30 2023-11-24T06:09:13.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333414f930 0x7f33340cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333414f930 0x7f33340cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45614/0 (socket says 172.21.15.97:45614) 2023-11-24T06:09:13.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 -- 172.21.15.97:0/2519956575 learned_addr learned my addr 172.21.15.97:0/2519956575 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:13.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33327fc700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33340cf770 0x7f33340c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f3332ffd700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f331c003a30 0x7f333414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 -- 172.21.15.97:0/2519956575 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f331c003a30 msgr2=0x7f333414c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f331c003a30 0x7f333414c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 -- 172.21.15.97:0/2519956575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33340cf770 msgr2=0x7f33340c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33340cf770 0x7f33340c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 -- 172.21.15.97:0/2519956575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340cef70 con 0x7f333414f930 2023-11-24T06:09:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.550+0000 7f33337fe700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333414f930 0x7f33340cc640 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3334072a90 tx=0x7f332c0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.551+0000 7f331bfff700 1 -- 172.21.15.97:0/2519956575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c01b070 con 0x7f333414f930 2023-11-24T06:09:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.551+0000 7f331bfff700 1 -- 172.21.15.97:0/2519956575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c00f040 con 0x7f333414f930 2023-11-24T06:09:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.551+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33340cf250 con 0x7f333414f930 2023-11-24T06:09:13.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.551+0000 7f331bfff700 1 -- 172.21.15.97:0/2519956575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c00e050 con 0x7f333414f930 2023-11-24T06:09:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.551+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33340ccb80 con 0x7f333414f930 2023-11-24T06:09:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.552+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33340753f0 con 0x7f333414f930 2023-11-24T06:09:13.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.552+0000 7f331bfff700 1 -- 172.21.15.97:0/2519956575 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f332c015440 con 0x7f333414f930 2023-11-24T06:09:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.553+0000 7f331bfff700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006aed0 0x7f332006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.553+0000 7f3332ffd700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006aed0 0x7f332006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.553+0000 7f331bfff700 1 -- 172.21.15.97:0/2519956575 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f332c089710 con 0x7f333414f930 2023-11-24T06:09:13.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.553+0000 7f3332ffd700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006aed0 0x7f332006d390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33280090b0 tx=0x7f3328008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.557+0000 7f331bfff700 1 -- 172.21.15.97:0/2519956575 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f332c054900 con 0x7f333414f930 2023-11-24T06:09:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.788+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33340d08b0 con 0x7f333414f930 2023-11-24T06:09:13.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.789+0000 7f331bfff700 1 -- 172.21.15.97:0/2519956575 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f332c057f20 con 0x7f333414f930 2023-11-24T06:09:13.085 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:13.085 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:08:57.902510+0000","modified":"2023-11-24T06:09:00.921422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006aed0 msgr2=0x7f332006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006aed0 0x7f332006d390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f33280090b0 tx=0x7f3328008030 comp rx=0 tx=0).stop 2023-11-24T06:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333414f930 msgr2=0x7f33340cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333414f930 0x7f33340cc640 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3334072a90 tx=0x7f332c0104a0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 shutdown_connections 2023-11-24T06:09:13.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f333414f930 0x7f33340cc640 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006aed0 0x7f332006d390 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f331c003a30 0x7f333414c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 --2- 172.21.15.97:0/2519956575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33340cf770 0x7f33340c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 >> 172.21.15.97:0/2519956575 conn(0x7f3334071b90 msgr2=0x7f33340b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 shutdown_connections 2023-11-24T06:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:09.791+0000 7f3339832700 1 -- 172.21.15.97:0/2519956575 wait complete. 2023-11-24T06:09:13.091 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-11-24T06:09:13.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:13.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.221+0000 7fe3c07d4700 1 Processor -- start 2023-11-24T06:09:13.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.221+0000 7fe3c07d4700 1 -- start start 2023-11-24T06:09:13.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3c07d4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80b3370 0x7fe3b81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3c07d4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3b80d1c70 0x7fe3b80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3c07d4700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b80b22d0 con 0x7fe3b815bdd0 2023-11-24T06:09:13.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3c07d4700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b80b2f50 con 0x7fe3b80d1c70 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3c07d4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80d2640 con 0x7fe3b80b3370 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bed71700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3b80d1c70 0x7fe3b80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bed71700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe3b80d1c70 0x7fe3b80d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45622/0 (socket says 172.21.15.97:45622) 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bed71700 1 -- 172.21.15.97:0/1568655949 learned_addr learned my addr 172.21.15.97:0/1568655949 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3be570700 1 --1- 172.21.15.97:0/1568655949 >> v1:172.21.15.97:6789/0 conn(0x7fe3b815bdd0 0x7fe3b80b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38070/0 (socket says 172.21.15.97:38070) 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bed71700 1 -- 172.21.15.97:0/1568655949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80b3370 msgr2=0x7fe3b81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bdd6f700 1 --2- 172.21.15.97:0/1568655949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80b3370 0x7fe3b81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bd56e700 1 -- 172.21.15.97:0/1568655949 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965903183 0 0) 0x7fe3b80b22d0 con 0x7fe3b815bdd0 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bed71700 1 --2- 172.21.15.97:0/1568655949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80b3370 0x7fe3b81604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bed71700 1 -- 172.21.15.97:0/1568655949 >> v1:172.21.15.97:6789/0 conn(0x7fe3b815bdd0 legacy=0x7fe3b80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bed71700 1 -- 172.21.15.97:0/1568655949 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80b25b0 con 0x7fe3b80d1c70 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.222+0000 7fe3bed71700 1 --2- 172.21.15.97:0/1568655949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80d1c70 0x7fe3b80d2100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009e50 tx=0x7fe3b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef59028a3c0b2189 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3bd56e700 1 -- 172.21.15.97:0/1568655949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000e070 con 0x7fe3b80d1c70 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3bd56e700 1 -- 172.21.15.97:0/1568655949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b000b9c0 con 0x7fe3b80d1c70 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3bd56e700 1 -- 172.21.15.97:0/1568655949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000bd70 con 0x7fe3b80d1c70 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3c07d4700 1 -- 172.21.15.97:0/1568655949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80d1c70 msgr2=0x7fe3b80d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/1568655949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80d1c70 0x7fe3b80d2100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009e50 tx=0x7fe3b000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3c07d4700 1 -- 172.21.15.97:0/1568655949 shutdown_connections 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/1568655949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80d1c70 0x7fe3b80d2100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/1568655949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b80b3370 0x7fe3b81604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3c07d4700 1 -- 172.21.15.97:0/1568655949 >> 172.21.15.97:0/1568655949 conn(0x7fe3b80731a0 msgr2=0x7fe3b809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3c07d4700 1 -- 172.21.15.97:0/1568655949 shutdown_connections 2023-11-24T06:09:13.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.223+0000 7fe3c07d4700 1 -- 172.21.15.97:0/1568655949 wait complete. 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.224+0000 7fe3c07d4700 1 Processor -- start 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3c07d4700 1 -- start start 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3c07d4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80b3370 0x7fe3b80c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3c07d4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b80d1c70 0x7fe3b81564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3c07d4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b815bdd0 0x7fe3b81569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3c07d4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8156f20 con 0x7fe3b80b3370 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3c07d4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80c5fa0 con 0x7fe3b80d1c70 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3c07d4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80c5970 con 0x7fe3b815bdd0 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3be570700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80b3370 0x7fe3b80c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3be570700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80b3370 0x7fe3b80c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45624/0 (socket says 172.21.15.97:45624) 2023-11-24T06:09:13.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3be570700 1 -- 172.21.15.97:0/957009054 learned_addr learned my addr 172.21.15.97:0/957009054 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3bed71700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b815bdd0 0x7fe3b81569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.225+0000 7fe3bdd6f700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b80d1c70 0x7fe3b81564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3be570700 1 -- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b815bdd0 msgr2=0x7fe3b81569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3be570700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b815bdd0 0x7fe3b81569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3be570700 1 -- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b80d1c70 msgr2=0x7fe3b81564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3be570700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b80d1c70 0x7fe3b81564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3be570700 1 -- 172.21.15.97:0/957009054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b0009b30 con 0x7fe3b80b3370 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3bdd6f700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b80d1c70 0x7fe3b81564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3be570700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80b3370 0x7fe3b80c7f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400c550 tx=0x7fe3b400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3aeffd700 1 -- 172.21.15.97:0/957009054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4004c80 con 0x7fe3b80b3370 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3aeffd700 1 -- 172.21.15.97:0/957009054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b4004e00 con 0x7fe3b80b3370 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b81e6410 con 0x7fe3b80b3370 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3aeffd700 1 -- 172.21.15.97:0/957009054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4007310 con 0x7fe3b80b3370 2023-11-24T06:09:13.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.226+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b80c8490 con 0x7fe3b80b3370 2023-11-24T06:09:13.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.227+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b80c8680 con 0x7fe3b80b3370 2023-11-24T06:09:13.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.227+0000 7fe3aeffd700 1 -- 172.21.15.97:0/957009054 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3b4007470 con 0x7fe3b80b3370 2023-11-24T06:09:13.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.228+0000 7fe3aeffd700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe39c06aed0 0x7fe39c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.228+0000 7fe3aeffd700 1 -- 172.21.15.97:0/957009054 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3b4089170 con 0x7fe3b80b3370 2023-11-24T06:09:13.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.228+0000 7fe3bdd6f700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe39c06aed0 0x7fe39c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.228+0000 7fe3bdd6f700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe39c06aed0 0x7fe39c06d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80b7c30 tx=0x7fe3a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.232+0000 7fe3aeffd700 1 -- 172.21.15.97:0/957009054 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3b40542b0 con 0x7fe3b80b3370 2023-11-24T06:09:13.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.426+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3b80c57d0 con 0x7fe3b80b3370 2023-11-24T06:09:13.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.427+0000 7fe3aeffd700 1 -- 172.21.15.97:0/957009054 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7fe3b80c57d0 con 0x7fe3b80b3370 2023-11-24T06:09:13.432 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:13.432 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:13.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe39c06aed0 msgr2=0x7fe39c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe39c06aed0 0x7fe39c06d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80b7c30 tx=0x7fe3a8008040 comp rx=0 tx=0).stop 2023-11-24T06:09:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80b3370 msgr2=0x7fe3b80c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80b3370 0x7fe3b80c7f50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400c550 tx=0x7fe3b400c860 comp rx=0 tx=0).stop 2023-11-24T06:09:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 shutdown_connections 2023-11-24T06:09:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe39c06aed0 0x7fe39c06d390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3b815bdd0 0x7fe3b81569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3b80b3370 0x7fe3b80c7f50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 --2- 172.21.15.97:0/957009054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3b80d1c70 0x7fe3b81564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 >> 172.21.15.97:0/957009054 conn(0x7fe3b80731a0 msgr2=0x7fe3b80cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 shutdown_connections 2023-11-24T06:09:13.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.429+0000 7fe3c07d4700 1 -- 172.21.15.97:0/957009054 wait complete. 2023-11-24T06:09:13.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:13.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:13.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.566+0000 7f2743d6f700 1 Processor -- start 2023-11-24T06:09:13.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.567+0000 7f2743d6f700 1 -- start start 2023-11-24T06:09:13.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.567+0000 7f2743d6f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c0746a0 0x7f273c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2743d6f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273c0b3c60 con 0x7f273c15e160 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2743d6f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273c0b58c0 con 0x7f273c0afc90 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2743d6f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0b4500 con 0x7f273c0746a0 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2741b0b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f273c15e160 0x7f273c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38084/0 (socket says 172.21.15.97:38084) 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2741b0b700 1 -- 172.21.15.97:0/3384601150 learned_addr learned my addr 172.21.15.97:0/3384601150 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f274130a700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c0746a0 0x7f273c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84753163 0 0) 0x7f273c0b3c60 con 0x7f273c15e160 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27280035b0 con 0x7f273c15e160 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165689677 0 0) 0x7f273c0b58c0 con 0x7f273c0afc90 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f273c0b3c60 con 0x7f273c0afc90 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734118772 0 0) 0x7f27280035b0 con 0x7f273c15e160 2023-11-24T06:09:13.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c0746a0 msgr2=0x7f273c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c0746a0 0x7f273c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 >> v1:172.21.15.183:6789/0 conn(0x7f273c0afc90 legacy=0x7f273c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0b0140 con 0x7f273c15e160 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f272c001670 con 0x7f273c15e160 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 >> v1:172.21.15.97:6789/0 conn(0x7f273c15e160 legacy=0x7f273c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c0746a0 0x7f2728003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2728004120 0x7f2728004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2728004d90 0x7f2728005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c0b4500 con 0x7f2728004d90 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272800baf0 con 0x7f2728004120 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272800bcb0 con 0x7f273c0746a0 2023-11-24T06:09:13.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f274230c700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2728004d90 0x7f2728005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f2741b0b700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c0746a0 0x7f2728003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.568+0000 7f274130a700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2728004120 0x7f2728004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f274230c700 1 -- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c0746a0 msgr2=0x7f2728003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f274230c700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c0746a0 0x7f2728003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f274230c700 1 -- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2728004120 msgr2=0x7f2728004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f274230c700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2728004120 0x7f2728004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f274230c700 1 -- 172.21.15.97:0/3384601150 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27280035b0 con 0x7f2728004d90 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f274230c700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2728004d90 0x7f2728005210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f273400c2f0 tx=0x7f2728005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd45ac2fdd6c6ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734001f30 con 0x7f2728004d90 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273400ce90 con 0x7f2728004d90 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f2740b09700 1 -- 172.21.15.97:0/3384601150 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2734003c40 con 0x7f2728004d90 2023-11-24T06:09:13.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f2743d6f700 1 -- 172.21.15.97:0/3384601150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2728004d90 msgr2=0x7f2728005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f2743d6f700 1 --2- 172.21.15.97:0/3384601150 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2728004d90 0x7f2728005210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f273400c2f0 tx=0x7f2728005880 comp rx=0 tx=0).stop 2023-11-24T06:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f274230c700 1 -- 172.21.15.97:0/3384601150 reap_dead start 2023-11-24T06:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f2743d6f700 1 -- 172.21.15.97:0/3384601150 shutdown_connections 2023-11-24T06:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.569+0000 7f2743d6f700 1 -- 172.21.15.97:0/3384601150 >> 172.21.15.97:0/3384601150 conn(0x7f273c0d0bf0 msgr2=0x7f273c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.570+0000 7f2743d6f700 1 -- 172.21.15.97:0/3384601150 shutdown_connections 2023-11-24T06:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.570+0000 7f2743d6f700 1 -- 172.21.15.97:0/3384601150 wait complete. 2023-11-24T06:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f2743d6f700 1 Processor -- start 2023-11-24T06:09:13.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f2743d6f700 1 -- start start 2023-11-24T06:09:13.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f2743d6f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c14b690 0x7f273c160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f2743d6f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f273c157240 0x7f273c1611b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f2743d6f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f273c157f30 0x7f273c1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f2743d6f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c1561a0 con 0x7f273c157240 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f2743d6f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c156480 con 0x7f273c157f30 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f2743d6f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c156760 con 0x7f273c14b690 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f274130a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f273c157240 0x7f273c1611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f274130a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f273c157240 0x7f273c1611b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45646/0 (socket says 172.21.15.97:45646) 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.571+0000 7f274130a700 1 -- 172.21.15.97:0/2487199957 learned_addr learned my addr 172.21.15.97:0/2487199957 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f274230c700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f273c157f30 0x7f273c1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f2741b0b700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c14b690 0x7f273c160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f274130a700 1 -- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c14b690 msgr2=0x7f273c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f274130a700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c14b690 0x7f273c160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f274130a700 1 -- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f273c157f30 msgr2=0x7f273c1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f274130a700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f273c157f30 0x7f273c1583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f274130a700 1 -- 172.21.15.97:0/2487199957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c156a40 con 0x7f273c157240 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f2741b0b700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c14b690 0x7f273c160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f274230c700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f273c157f30 0x7f273c1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f274130a700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f273c157240 0x7f273c1611b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2738011aa0 tx=0x7f2738011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f27327fc700 1 -- 172.21.15.97:0/2487199957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800f980 con 0x7f273c157240 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f27327fc700 1 -- 172.21.15.97:0/2487199957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273800fae0 con 0x7f273c157240 2023-11-24T06:09:13.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c156d20 con 0x7f273c157240 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f27327fc700 1 -- 172.21.15.97:0/2487199957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800fe90 con 0x7f273c157240 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.572+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c14ba70 con 0x7f273c157240 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.573+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c068fb0 con 0x7f273c157240 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.573+0000 7f27327fc700 1 -- 172.21.15.97:0/2487199957 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2738014440 con 0x7f273c157240 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.574+0000 7f27327fc700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f272006aed0 0x7f272006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.574+0000 7f27327fc700 1 -- 172.21.15.97:0/2487199957 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2738089da0 con 0x7f273c157240 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.574+0000 7f2741b0b700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f272006aed0 0x7f272006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.574+0000 7f2741b0b700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f272006aed0 0x7f272006d390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f272c007c90 tx=0x7f272c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.578+0000 7f27327fc700 1 -- 172.21.15.97:0/2487199957 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2738054fc0 con 0x7f273c157240 2023-11-24T06:09:13.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.767+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f273c06c140 con 0x7f273c157240 2023-11-24T06:09:13.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.768+0000 7f27327fc700 1 -- 172.21.15.97:0/2487199957 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27380585e0 con 0x7f273c157240 2023-11-24T06:09:13.772 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:13.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f272006aed0 msgr2=0x7f272006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f272006aed0 0x7f272006d390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f272c007c90 tx=0x7f272c009040 comp rx=0 tx=0).stop 2023-11-24T06:09:13.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f273c157240 msgr2=0x7f273c1611b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f273c157240 0x7f273c1611b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2738011aa0 tx=0x7f2738011e60 comp rx=0 tx=0).stop 2023-11-24T06:09:13.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 shutdown_connections 2023-11-24T06:09:13.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f273c157f30 0x7f273c1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f272006aed0 0x7f272006d390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f273c14b690 0x7f273c160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 --2- 172.21.15.97:0/2487199957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f273c157240 0x7f273c1611b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 >> 172.21.15.97:0/2487199957 conn(0x7f273c0d0bf0 msgr2=0x7f273c06ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:13.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 shutdown_connections 2023-11-24T06:09:13.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.770+0000 7f2743d6f700 1 -- 172.21.15.97:0/2487199957 wait complete. 2023-11-24T06:09:13.786 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.906+0000 7f66f267c700 1 Processor -- start 2023-11-24T06:09:13.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.906+0000 7f66f267c700 1 -- start start 2023-11-24T06:09:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.906+0000 7f66f267c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f66ec15e160 0x7f66ec0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.906+0000 7f66f267c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f66ec0afc90 0x7f66ec073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.906+0000 7f66f267c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0744d0 0x7f66ec06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.906+0000 7f66f267c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ec0b4500 con 0x7f66ec15e160 2023-11-24T06:09:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.906+0000 7f66f267c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ec074960 con 0x7f66ec0afc90 2023-11-24T06:09:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.906+0000 7f66f267c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec0b3c60 con 0x7f66ec0744d0 2023-11-24T06:09:13.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f66ec15e160 0x7f66ec0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f66ec15e160 0x7f66ec0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45652/0 (socket says 172.21.15.97:45652) 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 -- 172.21.15.97:0/1127709560 learned_addr learned my addr 172.21.15.97:0/1127709560 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66f0c19700 1 --2- 172.21.15.97:0/1127709560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0744d0 0x7f66ec06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66eb7fe700 1 --2- 172.21.15.97:0/1127709560 >> v2:172.21.15.183:3300/0 conn(0x7f66ec0afc90 0x7f66ec073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 -- 172.21.15.97:0/1127709560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0744d0 msgr2=0x7f66ec06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 --2- 172.21.15.97:0/1127709560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0744d0 0x7f66ec06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 -- 172.21.15.97:0/1127709560 >> v2:172.21.15.183:3300/0 conn(0x7f66ec0afc90 msgr2=0x7f66ec073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 --2- 172.21.15.97:0/1127709560 >> v2:172.21.15.183:3300/0 conn(0x7f66ec0afc90 0x7f66ec073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 -- 172.21.15.97:0/1127709560 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec0b58c0 con 0x7f66ec15e160 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66f0c19700 1 --2- 172.21.15.97:0/1127709560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0744d0 0x7f66ec06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66ebfff700 1 --2- 172.21.15.97:0/1127709560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b5da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0d0880 tx=0x7f66dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aefad7b4f0f6a04 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66eaffd700 1 -- 172.21.15.97:0/1127709560 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc00b870 con 0x7f66ec15e160 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66eaffd700 1 -- 172.21.15.97:0/1127709560 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc00b9d0 con 0x7f66ec15e160 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.907+0000 7f66eaffd700 1 -- 172.21.15.97:0/1127709560 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc00bd60 con 0x7f66ec15e160 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 -- 172.21.15.97:0/1127709560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 msgr2=0x7f66ec0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 --2- 172.21.15.97:0/1127709560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b5da0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0d0880 tx=0x7f66dc009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 -- 172.21.15.97:0/1127709560 shutdown_connections 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 --2- 172.21.15.97:0/1127709560 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0744d0 0x7f66ec06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 --2- 172.21.15.97:0/1127709560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b5da0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 --2- 172.21.15.97:0/1127709560 >> v2:172.21.15.183:3300/0 conn(0x7f66ec0afc90 0x7f66ec073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 -- 172.21.15.97:0/1127709560 >> 172.21.15.97:0/1127709560 conn(0x7f66ec0d0bf0 msgr2=0x7f66ec0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 -- 172.21.15.97:0/1127709560 shutdown_connections 2023-11-24T06:09:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.908+0000 7f66f267c700 1 -- 172.21.15.97:0/1127709560 wait complete. 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.909+0000 7f66f267c700 1 Processor -- start 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.909+0000 7f66f267c700 1 -- start start 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f267c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66ec0744d0 0x7f66ec14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f267c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0afc90 0x7f66ec153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f267c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f267c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec1500e0 con 0x7f66ec15e160 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f267c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec154190 con 0x7f66ec0744d0 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f267c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec1f74c0 con 0x7f66ec0afc90 2023-11-24T06:09:13.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45660/0 (socket says 172.21.15.97:45660) 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 -- 172.21.15.97:0/197867693 learned_addr learned my addr 172.21.15.97:0/197867693 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66ebfff700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66ec0744d0 0x7f66ec14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66eb7fe700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0afc90 0x7f66ec153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 -- 172.21.15.97:0/197867693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0afc90 msgr2=0x7f66ec153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0afc90 0x7f66ec153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 -- 172.21.15.97:0/197867693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66ec0744d0 msgr2=0x7f66ec14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66ec0744d0 0x7f66ec14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 -- 172.21.15.97:0/197867693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc009550 con 0x7f66ec15e160 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66f0c19700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b1780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f66e4007ef0 tx=0x7f66e400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.910+0000 7f66e8ff9700 1 -- 172.21.15.97:0/197867693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400dd80 con 0x7f66ec15e160 2023-11-24T06:09:13.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.911+0000 7f66e8ff9700 1 -- 172.21.15.97:0/197867693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e400f040 con 0x7f66ec15e160 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.911+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ec1f77a0 con 0x7f66ec15e160 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.911+0000 7f66e8ff9700 1 -- 172.21.15.97:0/197867693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4003c60 con 0x7f66ec15e160 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.911+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ec1f7d60 con 0x7f66ec15e160 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.912+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ec15e8b0 con 0x7f66ec15e160 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.912+0000 7f66e8ff9700 1 -- 172.21.15.97:0/197867693 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66e4004030 con 0x7f66ec15e160 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.912+0000 7f66e8ff9700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66cc06aed0 0x7f66cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.913+0000 7f66e8ff9700 1 -- 172.21.15.97:0/197867693 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66e4089590 con 0x7f66ec15e160 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.913+0000 7f66ebfff700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66cc06aed0 0x7f66cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:13.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.913+0000 7f66ebfff700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66cc06aed0 0x7f66cc06d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66dc014040 tx=0x7f66dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:13.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:13.917+0000 7f66e8ff9700 1 -- 172.21.15.97:0/197867693 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66e40546d0 con 0x7f66ec15e160 2023-11-24T06:09:14.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.107+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66ec1f7600 con 0x7f66ec15e160 2023-11-24T06:09:14.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.107+0000 7f66e8ff9700 1 -- 172.21.15.97:0/197867693 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f66ec1f7600 con 0x7f66ec15e160 2023-11-24T06:09:14.111 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66cc06aed0 msgr2=0x7f66cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66cc06aed0 0x7f66cc06d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66dc014040 tx=0x7f66dc008030 comp rx=0 tx=0).stop 2023-11-24T06:09:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 msgr2=0x7f66ec0b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b1780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f66e4007ef0 tx=0x7f66e400d3b0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 shutdown_connections 2023-11-24T06:09:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ec15e160 0x7f66ec0b1780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66ec0744d0 0x7f66ec14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66cc06aed0 0x7f66cc06d390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 --2- 172.21.15.97:0/197867693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f66ec0afc90 0x7f66ec153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 >> 172.21.15.97:0/197867693 conn(0x7f66ec0d0bf0 msgr2=0x7f66ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 shutdown_connections 2023-11-24T06:09:14.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.109+0000 7f66f267c700 1 -- 172.21.15.97:0/197867693 wait complete. 2023-11-24T06:09:14.125 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.244+0000 7effb0388700 1 Processor -- start 2023-11-24T06:09:14.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.244+0000 7effb0388700 1 -- start start 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.245+0000 7effb0388700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7effa815e160 0x7effa80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.245+0000 7effb0388700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa80b3c60 con 0x7effa80746a0 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.245+0000 7effb0388700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa80b58c0 con 0x7effa80afc90 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.245+0000 7effb0388700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa80b4500 con 0x7effa815e160 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effae925700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7effa80746a0 0x7effa806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38122/0 (socket says 172.21.15.97:38122) 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effae925700 1 -- 172.21.15.97:0/1054971365 learned_addr learned my addr 172.21.15.97:0/1054971365 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effae124700 1 --2- 172.21.15.97:0/1054971365 >> v2:172.21.15.183:3300/0 conn(0x7effa815e160 0x7effa80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effad122700 1 -- 172.21.15.97:0/1054971365 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226371910 0 0) 0x7effa80b3c60 con 0x7effa80746a0 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effad122700 1 -- 172.21.15.97:0/1054971365 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff940035b0 con 0x7effa80746a0 2023-11-24T06:09:14.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effad122700 1 -- 172.21.15.97:0/1054971365 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948918468 0 0) 0x7effa80b58c0 con 0x7effa80afc90 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effad122700 1 -- 172.21.15.97:0/1054971365 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa80b3c60 con 0x7effa80afc90 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effae124700 1 -- 172.21.15.97:0/1054971365 >> v1:172.21.15.183:6789/0 conn(0x7effa80afc90 legacy=0x7effa8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effae124700 1 -- 172.21.15.97:0/1054971365 >> v1:172.21.15.97:6789/0 conn(0x7effa80746a0 legacy=0x7effa806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effae124700 1 -- 172.21.15.97:0/1054971365 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80b0140 con 0x7effa815e160 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effae124700 1 --2- 172.21.15.97:0/1054971365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa815e160 0x7effa80b5da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff98009840 tx=0x7eff98009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5577d2130561dfb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effad122700 1 -- 172.21.15.97:0/1054971365 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98011830 con 0x7effa815e160 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effad122700 1 -- 172.21.15.97:0/1054971365 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98011990 con 0x7effa815e160 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.246+0000 7effad122700 1 -- 172.21.15.97:0/1054971365 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98011d20 con 0x7effa815e160 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.247+0000 7effb0388700 1 -- 172.21.15.97:0/1054971365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa815e160 msgr2=0x7effa80b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.247+0000 7effb0388700 1 --2- 172.21.15.97:0/1054971365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa815e160 0x7effa80b5da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff98009840 tx=0x7eff98009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.247+0000 7effb0388700 1 -- 172.21.15.97:0/1054971365 shutdown_connections 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.247+0000 7effb0388700 1 --2- 172.21.15.97:0/1054971365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa815e160 0x7effa80b5da0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.247+0000 7effb0388700 1 -- 172.21.15.97:0/1054971365 >> 172.21.15.97:0/1054971365 conn(0x7effa80d0bf0 msgr2=0x7effa80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:14.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.247+0000 7effb0388700 1 -- 172.21.15.97:0/1054971365 shutdown_connections 2023-11-24T06:09:14.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.247+0000 7effb0388700 1 -- 172.21.15.97:0/1054971365 wait complete. 2023-11-24T06:09:14.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.248+0000 7effb0388700 1 Processor -- start 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.248+0000 7effb0388700 1 -- start start 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effb0388700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa80746a0 0x7effa8144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effb0388700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effa80afc90 0x7effa8160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effb0388700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effa815e160 0x7effa8156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effb0388700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa81468a0 con 0x7effa815e160 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effb0388700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8145460 con 0x7effa80746a0 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effb0388700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa8145c20 con 0x7effa80afc90 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae925700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effa815e160 0x7effa8156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae925700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effa815e160 0x7effa8156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45676/0 (socket says 172.21.15.97:45676) 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae925700 1 -- 172.21.15.97:0/4059910251 learned_addr learned my addr 172.21.15.97:0/4059910251 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:14.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae124700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa80746a0 0x7effa8144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effad923700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effa80afc90 0x7effa8160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae925700 1 -- 172.21.15.97:0/4059910251 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effa80afc90 msgr2=0x7effa8160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae925700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effa80afc90 0x7effa8160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae925700 1 -- 172.21.15.97:0/4059910251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa80746a0 msgr2=0x7effa8144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae925700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa80746a0 0x7effa8144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.249+0000 7effae925700 1 -- 172.21.15.97:0/4059910251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980094a0 con 0x7effa815e160 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.250+0000 7effae925700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effa815e160 0x7effa8156100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7effa000cba0 tx=0x7effa000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.250+0000 7eff9effd700 1 -- 172.21.15.97:0/4059910251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000a9e0 con 0x7effa815e160 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.250+0000 7eff9effd700 1 -- 172.21.15.97:0/4059910251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa000ab40 con 0x7effa815e160 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.250+0000 7eff9effd700 1 -- 172.21.15.97:0/4059910251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0012400 con 0x7effa815e160 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.250+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa8145f00 con 0x7effa815e160 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.250+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa81464c0 con 0x7effa815e160 2023-11-24T06:09:14.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.251+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa81469e0 con 0x7effa815e160 2023-11-24T06:09:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.251+0000 7eff9effd700 1 -- 172.21.15.97:0/4059910251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effa001b3b0 con 0x7effa815e160 2023-11-24T06:09:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.252+0000 7eff9effd700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7eff8c06aed0 0x7eff8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.252+0000 7eff9effd700 1 -- 172.21.15.97:0/4059910251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effa0089150 con 0x7effa815e160 2023-11-24T06:09:14.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.252+0000 7effae124700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7eff8c06aed0 0x7eff8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.252+0000 7effae124700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7eff8c06aed0 0x7eff8c06d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff98009fd0 tx=0x7eff9801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.256+0000 7eff9effd700 1 -- 172.21.15.97:0/4059910251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7effa81469e0 con 0x7effa815e160 2023-11-24T06:09:14.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.448+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7effa81469e0 con 0x7effa815e160 2023-11-24T06:09:14.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.449+0000 7eff9effd700 1 -- 172.21.15.97:0/4059910251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7effa81469e0 con 0x7effa815e160 2023-11-24T06:09:14.453 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7eff8c06aed0 msgr2=0x7eff8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7eff8c06aed0 0x7eff8c06d390 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff98009fd0 tx=0x7eff9801c040 comp rx=0 tx=0).stop 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effa815e160 msgr2=0x7effa8156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effa815e160 0x7effa8156100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7effa000cba0 tx=0x7effa000ceb0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 shutdown_connections 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7effa815e160 0x7effa8156100 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effa80746a0 0x7effa8144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7eff8c06aed0 0x7eff8c06d390 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 --2- 172.21.15.97:0/4059910251 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7effa80afc90 0x7effa8160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 >> 172.21.15.97:0/4059910251 conn(0x7effa80d0bf0 msgr2=0x7effa815ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 shutdown_connections 2023-11-24T06:09:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.451+0000 7effb0388700 1 -- 172.21.15.97:0/4059910251 wait complete. 2023-11-24T06:09:14.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:14.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.587+0000 7f15b168d700 1 Processor -- start 2023-11-24T06:09:14.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.587+0000 7f15b168d700 1 -- start start 2023-11-24T06:09:14.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.587+0000 7f15b168d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 0x7f15ac0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.587+0000 7f15b168d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15ac0cf0d0 0x7f15ac0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.587+0000 7f15b168d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac0cf530 con 0x7f15ac15e0f0 2023-11-24T06:09:14.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.587+0000 7f15b168d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac06bc00 con 0x7f15ac0cf0d0 2023-11-24T06:09:14.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.587+0000 7f15b168d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac15e4d0 con 0x7f15ac0b2b60 2023-11-24T06:09:14.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15ab7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15ac0cf0d0 0x7f15ac0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15ab7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15ac0cf0d0 0x7f15ac0d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45684/0 (socket says 172.21.15.97:45684) 2023-11-24T06:09:14.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15ab7fe700 1 -- 172.21.15.97:0/3656656954 learned_addr learned my addr 172.21.15.97:0/3656656954 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15aa7fc700 1 --2- 172.21.15.97:0/3656656954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 0x7f15ac0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15ab7fe700 1 -- 172.21.15.97:0/3656656954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 msgr2=0x7f15ac0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15ab7fe700 1 --2- 172.21.15.97:0/3656656954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 0x7f15ac0ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15ab7fe700 1 -- 172.21.15.97:0/3656656954 >> v1:172.21.15.183:6789/0 conn(0x7f15ac15e0f0 legacy=0x7f15ac0b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15ab7fe700 1 -- 172.21.15.97:0/3656656954 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac06aff0 con 0x7f15ac0cf0d0 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15ab7fe700 1 --2- 172.21.15.97:0/3656656954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac0cf0d0 0x7f15ac0d1f60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009e50 tx=0x7f15a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9101adb3c79cdd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15a9ffb700 1 -- 172.21.15.97:0/3656656954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a400e070 con 0x7f15ac0cf0d0 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15a9ffb700 1 -- 172.21.15.97:0/3656656954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a4003a20 con 0x7f15ac0cf0d0 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.588+0000 7f15a9ffb700 1 -- 172.21.15.97:0/3656656954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4003dd0 con 0x7f15ac0cf0d0 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.589+0000 7f15b168d700 1 -- 172.21.15.97:0/3656656954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac0cf0d0 msgr2=0x7f15ac0d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.589+0000 7f15b168d700 1 --2- 172.21.15.97:0/3656656954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac0cf0d0 0x7f15ac0d1f60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009e50 tx=0x7f15a400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:14.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.589+0000 7f15b168d700 1 -- 172.21.15.97:0/3656656954 shutdown_connections 2023-11-24T06:09:14.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.589+0000 7f15b168d700 1 --2- 172.21.15.97:0/3656656954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac0cf0d0 0x7f15ac0d1f60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.589+0000 7f15b168d700 1 --2- 172.21.15.97:0/3656656954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 0x7f15ac0ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.589+0000 7f15b168d700 1 -- 172.21.15.97:0/3656656954 >> 172.21.15.97:0/3656656954 conn(0x7f15ac0b5ce0 msgr2=0x7f15ac0b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:14.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.589+0000 7f15b168d700 1 -- 172.21.15.97:0/3656656954 shutdown_connections 2023-11-24T06:09:14.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.589+0000 7f15b168d700 1 -- 172.21.15.97:0/3656656954 wait complete. 2023-11-24T06:09:14.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.590+0000 7f15b168d700 1 Processor -- start 2023-11-24T06:09:14.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.590+0000 7f15b168d700 1 -- start start 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15b168d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 0x7f15ac14e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15b168d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15ac0cf0d0 0x7f15ac0c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15b168d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac15e0f0 0x7f15ac15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15b168d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac14e7d0 con 0x7f15ac15e0f0 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15b168d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1587a0 con 0x7f15ac0cf0d0 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15b168d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac15e7b0 con 0x7f15ac0b2b60 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac15e0f0 0x7f15ac15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac15e0f0 0x7f15ac15a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45690/0 (socket says 172.21.15.97:45690) 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 -- 172.21.15.97:0/1849965916 learned_addr learned my addr 172.21.15.97:0/1849965916 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15aaffd700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 0x7f15ac14e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15aa7fc700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15ac0cf0d0 0x7f15ac0c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 -- 172.21.15.97:0/1849965916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 msgr2=0x7f15ac14e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 0x7f15ac14e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 -- 172.21.15.97:0/1849965916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15ac0cf0d0 msgr2=0x7f15ac0c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15ac0cf0d0 0x7f15ac0c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 -- 172.21.15.97:0/1849965916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a4009b30 con 0x7f15ac15e0f0 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15ab7fe700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac15e0f0 0x7f15ac15a040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15ac08c570 tx=0x7f15a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.591+0000 7f15a37fe700 1 -- 172.21.15.97:0/1849965916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4004440 con 0x7f15ac15e0f0 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.592+0000 7f15a37fe700 1 -- 172.21.15.97:0/1849965916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a40045c0 con 0x7f15ac15e0f0 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.592+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac15ea90 con 0x7f15ac15e0f0 2023-11-24T06:09:14.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.592+0000 7f15a37fe700 1 -- 172.21.15.97:0/1849965916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a4004970 con 0x7f15ac15e0f0 2023-11-24T06:09:14.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.592+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac15f050 con 0x7f15ac15e0f0 2023-11-24T06:09:14.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.592+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac0751e0 con 0x7f15ac15e0f0 2023-11-24T06:09:14.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.593+0000 7f15a37fe700 1 -- 172.21.15.97:0/1849965916 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15a4025450 con 0x7f15ac15e0f0 2023-11-24T06:09:14.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.593+0000 7f15a37fe700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f158c06aed0 0x7f158c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.593+0000 7f15aaffd700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f158c06aed0 0x7f158c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.594+0000 7f15a37fe700 1 -- 172.21.15.97:0/1849965916 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15a408abf0 con 0x7f15ac15e0f0 2023-11-24T06:09:14.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.594+0000 7f15aaffd700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f158c06aed0 0x7f158c06d390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f159c009960 tx=0x7f159c0090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.598+0000 7f15a37fe700 1 -- 172.21.15.97:0/1849965916 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15a40597c0 con 0x7f15ac15e0f0 2023-11-24T06:09:14.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.835+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15ac0683f0 con 0x7f15ac15e0f0 2023-11-24T06:09:14.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.835+0000 7f15a37fe700 1 -- 172.21.15.97:0/1849965916 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f15a4059350 con 0x7f15ac15e0f0 2023-11-24T06:09:14.839 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:14.839 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:08:57.902510+0000","modified":"2023-11-24T06:09:00.921422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f158c06aed0 msgr2=0x7f158c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f158c06aed0 0x7f158c06d390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f159c009960 tx=0x7f159c0090e0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac15e0f0 msgr2=0x7f15ac15a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac15e0f0 0x7f15ac15a040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15ac08c570 tx=0x7f15a4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 shutdown_connections 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15ac15e0f0 0x7f15ac15a040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f158c06aed0 0x7f158c06d390 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15ac0b2b60 0x7f15ac14e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 --2- 172.21.15.97:0/1849965916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15ac0cf0d0 0x7f15ac0c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 >> 172.21.15.97:0/1849965916 conn(0x7f15ac0b5ce0 msgr2=0x7f15ac0b92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 shutdown_connections 2023-11-24T06:09:14.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.837+0000 7f15b168d700 1 -- 172.21.15.97:0/1849965916 wait complete. 2023-11-24T06:09:14.843 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-11-24T06:09:14.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.969+0000 7f3d900f5700 1 Processor -- start 2023-11-24T06:09:14.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.969+0000 7f3d900f5700 1 -- start start 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.970+0000 7f3d900f5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3d8815e080 0x7f3d880b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.970+0000 7f3d900f5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8806c710 0x7f3d880afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.970+0000 7f3d900f5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d880ae430 con 0x7f3d880af4d0 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.970+0000 7f3d900f5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d880af0b0 con 0x7f3d8815e080 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.970+0000 7f3d900f5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880b05a0 con 0x7f3d8806c710 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.970+0000 7f3d8d690700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3d880af4d0 0x7f3d8806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38146/0 (socket says 172.21.15.97:38146) 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.970+0000 7f3d8d690700 1 -- 172.21.15.97:0/3952907071 learned_addr learned my addr 172.21.15.97:0/3952907071 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8e692700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8806c710 0x7f3d880afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8de91700 1 --2- 172.21.15.97:0/3952907071 >> v2:172.21.15.183:3300/0 conn(0x7f3d8815e080 0x7f3d880b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454024257 0 0) 0x7f3d880ae430 con 0x7f3d880af4d0 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d740035b0 con 0x7f3d880af4d0 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3773698954 0 0) 0x7f3d740035b0 con 0x7f3d880af4d0 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8806c710 msgr2=0x7f3d880afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8806c710 0x7f3d880afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 >> v2:172.21.15.183:3300/0 conn(0x7f3d8815e080 msgr2=0x7f3d880b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 --2- 172.21.15.97:0/3952907071 >> v2:172.21.15.183:3300/0 conn(0x7f3d8815e080 0x7f3d880b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880ae710 con 0x7f3d880af4d0 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f3d84001670 con 0x7f3d880af4d0 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 >> v1:172.21.15.97:6789/0 conn(0x7f3d880af4d0 legacy=0x7f3d8806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d74003a30 0x7f3d74003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d74004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d74005210 0x7f3d7400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880af0b0 con 0x7f3d74004850 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880b05a0 con 0x7f3d74003a30 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7400bf50 con 0x7f3d74005210 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8de91700 1 --2- 172.21.15.97:0/3952907071 >> v2:172.21.15.183:3300/0 conn(0x7f3d8815e080 0x7f3d880b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8e692700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8806c710 0x7f3d880afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8d690700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d74004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8de91700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d74003a30 0x7f3d74003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8e692700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d74005210 0x7f3d7400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8d690700 1 -- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d74005210 msgr2=0x7f3d7400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8d690700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d74005210 0x7f3d7400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8d690700 1 -- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d74003a30 msgr2=0x7f3d74003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:14.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8d690700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d74003a30 0x7f3d74003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8d690700 1 -- 172.21.15.97:0/3952907071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740035b0 con 0x7f3d74004850 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.971+0000 7f3d8de91700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d74003a30 0x7f3d74003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d8d690700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d74004cd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400c210 tx=0x7f3d74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2a7b2d7550443f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84007c10 con 0x7f3d74004850 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8400cd70 con 0x7f3d74004850 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d8ce8f700 1 -- 172.21.15.97:0/3952907071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84005ba0 con 0x7f3d74004850 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d900f5700 1 -- 172.21.15.97:0/3952907071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 msgr2=0x7f3d74004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d900f5700 1 --2- 172.21.15.97:0/3952907071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d74004cd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400c210 tx=0x7f3d74005880 comp rx=0 tx=0).stop 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d8e692700 1 -- 172.21.15.97:0/3952907071 reap_dead start 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d900f5700 1 -- 172.21.15.97:0/3952907071 shutdown_connections 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d900f5700 1 -- 172.21.15.97:0/3952907071 >> 172.21.15.97:0/3952907071 conn(0x7f3d88071b90 msgr2=0x7f3d88071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d900f5700 1 -- 172.21.15.97:0/3952907071 shutdown_connections 2023-11-24T06:09:14.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.972+0000 7f3d900f5700 1 -- 172.21.15.97:0/3952907071 wait complete. 2023-11-24T06:09:14.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d900f5700 1 Processor -- start 2023-11-24T06:09:14.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d900f5700 1 -- start start 2023-11-24T06:09:14.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d900f5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d8813e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d900f5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151540 0x7f3d8814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d900f5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8814a8f0 0x7f3d881561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d900f5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8814b0e0 con 0x7f3d74004850 2023-11-24T06:09:14.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d900f5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8813ea10 con 0x7f3d88151540 2023-11-24T06:09:14.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d900f5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d881510b0 con 0x7f3d8814a8f0 2023-11-24T06:09:14.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d8de91700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d8813e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d8de91700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d8813e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45704/0 (socket says 172.21.15.97:45704) 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.974+0000 7f3d8de91700 1 -- 172.21.15.97:0/4129001648 learned_addr learned my addr 172.21.15.97:0/4129001648 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d8e692700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8814a8f0 0x7f3d881561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d8de91700 1 -- 172.21.15.97:0/4129001648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8814a8f0 msgr2=0x7f3d881561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d8d690700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151540 0x7f3d8814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d8de91700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8814a8f0 0x7f3d881561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d8de91700 1 -- 172.21.15.97:0/4129001648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151540 msgr2=0x7f3d8814fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d8de91700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151540 0x7f3d8814fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d8de91700 1 -- 172.21.15.97:0/4129001648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d881504a0 con 0x7f3d74004850 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d8de91700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d8813e4d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d780109e0 tx=0x7f3d78010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d7e7fc700 1 -- 172.21.15.97:0/4129001648 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800e8d0 con 0x7f3d74004850 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d7e7fc700 1 -- 172.21.15.97:0/4129001648 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7800ea30 con 0x7f3d74004850 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d88150780 con 0x7f3d74004850 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d7e7fc700 1 -- 172.21.15.97:0/4129001648 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800eda0 con 0x7f3d74004850 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.975+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d88150d40 con 0x7f3d74004850 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.976+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d880753f0 con 0x7f3d74004850 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.976+0000 7f3d7e7fc700 1 -- 172.21.15.97:0/4129001648 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d78016460 con 0x7f3d74004850 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.977+0000 7f3d7e7fc700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06aed0 0x7f3d6c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.977+0000 7f3d7e7fc700 1 -- 172.21.15.97:0/4129001648 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d78089aa0 con 0x7f3d74004850 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.977+0000 7f3d8d690700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06aed0 0x7f3d6c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:14.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.977+0000 7f3d8d690700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06aed0 0x7f3d6c06d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d84015040 tx=0x7f3d84007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:14.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:14.981+0000 7f3d7e7fc700 1 -- 172.21.15.97:0/4129001648 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d78054be0 con 0x7f3d74004850 2023-11-24T06:09:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.172+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d880d09b0 con 0x7f3d74004850 2023-11-24T06:09:15.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.173+0000 7f3d7e7fc700 1 -- 172.21.15.97:0/4129001648 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f3d78058200 con 0x7f3d74004850 2023-11-24T06:09:15.178 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:15.178 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06aed0 msgr2=0x7f3d6c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06aed0 0x7f3d6c06d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3d84015040 tx=0x7f3d84007000 comp rx=0 tx=0).stop 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 msgr2=0x7f3d8813e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d8813e4d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d780109e0 tx=0x7f3d78010da0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 shutdown_connections 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06aed0 0x7f3d6c06d390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d8814a8f0 0x7f3d881561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d74004850 0x7f3d8813e4d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 --2- 172.21.15.97:0/4129001648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151540 0x7f3d8814fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 >> 172.21.15.97:0/4129001648 conn(0x7f3d88071b90 msgr2=0x7f3d880b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 shutdown_connections 2023-11-24T06:09:15.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.176+0000 7f3d900f5700 1 -- 172.21.15.97:0/4129001648 wait complete. 2023-11-24T06:09:15.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.308+0000 7f9565971700 1 Processor -- start 2023-11-24T06:09:15.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.308+0000 7f9565971700 1 -- start start 2023-11-24T06:09:15.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f9565971700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f956015e1e0 0x7f95600d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f9565971700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f95600d3a00 0x7f95600d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f9565971700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95600d1200 con 0x7f95600d0db0 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f9565971700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95600d3150 con 0x7f956015e1e0 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f9565971700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956015e5c0 con 0x7f95600d3a00 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f955effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f956015e1e0 0x7f95600d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f955effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f956015e1e0 0x7f95600d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45712/0 (socket says 172.21.15.97:45712) 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f955effd700 1 -- 172.21.15.97:0/842854934 learned_addr learned my addr 172.21.15.97:0/842854934 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f955e7fc700 1 --2- 172.21.15.97:0/842854934 >> v2:172.21.15.183:3300/0 conn(0x7f95600d3a00 0x7f95600d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f955effd700 1 -- 172.21.15.97:0/842854934 >> v2:172.21.15.183:3300/0 conn(0x7f95600d3a00 msgr2=0x7f95600d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f955effd700 1 --2- 172.21.15.97:0/842854934 >> v2:172.21.15.183:3300/0 conn(0x7f95600d3a00 0x7f95600d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f955effd700 1 -- 172.21.15.97:0/842854934 >> v1:172.21.15.183:6789/0 conn(0x7f95600d0db0 legacy=0x7f956006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:15.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.309+0000 7f955effd700 1 -- 172.21.15.97:0/842854934 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600d20b0 con 0x7f956015e1e0 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f955effd700 1 --2- 172.21.15.97:0/842854934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f956015e1e0 0x7f95600d34c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9560161ba0 tx=0x7f9554009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec8bfaf4021bca7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f955dffb700 1 -- 172.21.15.97:0/842854934 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554011830 con 0x7f956015e1e0 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f955dffb700 1 -- 172.21.15.97:0/842854934 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9554011990 con 0x7f956015e1e0 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f955dffb700 1 -- 172.21.15.97:0/842854934 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554011d20 con 0x7f956015e1e0 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f9565971700 1 -- 172.21.15.97:0/842854934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f956015e1e0 msgr2=0x7f95600d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f9565971700 1 --2- 172.21.15.97:0/842854934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f956015e1e0 0x7f95600d34c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9560161ba0 tx=0x7f9554009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f9565971700 1 -- 172.21.15.97:0/842854934 shutdown_connections 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f9565971700 1 --2- 172.21.15.97:0/842854934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f956015e1e0 0x7f95600d34c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f9565971700 1 --2- 172.21.15.97:0/842854934 >> v2:172.21.15.183:3300/0 conn(0x7f95600d3a00 0x7f95600d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f9565971700 1 -- 172.21.15.97:0/842854934 >> 172.21.15.97:0/842854934 conn(0x7f95600b5b50 msgr2=0x7f95600b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f9565971700 1 -- 172.21.15.97:0/842854934 shutdown_connections 2023-11-24T06:09:15.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.310+0000 7f9565971700 1 -- 172.21.15.97:0/842854934 wait complete. 2023-11-24T06:09:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.311+0000 7f9565971700 1 Processor -- start 2023-11-24T06:09:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f9565971700 1 -- start start 2023-11-24T06:09:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f9565971700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95600d0db0 0x7f9560152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f9565971700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95600d3a00 0x7f95601f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f9565971700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f956015e1e0 0x7f95601f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f9565971700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600cadd0 con 0x7f95600d0db0 2023-11-24T06:09:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f9565971700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600cc8c0 con 0x7f95600d3a00 2023-11-24T06:09:15.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f9565971700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956014c3c0 con 0x7f956015e1e0 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f955effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95600d0db0 0x7f9560152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f955effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95600d0db0 0x7f9560152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45720/0 (socket says 172.21.15.97:45720) 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.312+0000 7f955effd700 1 -- 172.21.15.97:0/3033729212 learned_addr learned my addr 172.21.15.97:0/3033729212 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f955f7fe700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f956015e1e0 0x7f95601f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f955e7fc700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95600d3a00 0x7f95601f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f955effd700 1 -- 172.21.15.97:0/3033729212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f956015e1e0 msgr2=0x7f95601f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f955effd700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f956015e1e0 0x7f95601f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f955effd700 1 -- 172.21.15.97:0/3033729212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95600d3a00 msgr2=0x7f95601f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f955effd700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95600d3a00 0x7f95601f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f955effd700 1 -- 172.21.15.97:0/3033729212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9554009550 con 0x7f95600d0db0 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f955effd700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95600d0db0 0x7f9560152260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9554009c10 tx=0x7f9554004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f9547fff700 1 -- 172.21.15.97:0/3033729212 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554004510 con 0x7f95600d0db0 2023-11-24T06:09:15.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f9547fff700 1 -- 172.21.15.97:0/3033729212 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955401e070 con 0x7f95600d0db0 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956014c6a0 con 0x7f95600d0db0 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f9547fff700 1 -- 172.21.15.97:0/3033729212 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95540038e0 con 0x7f95600d0db0 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.313+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956014cc60 con 0x7f95600d0db0 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.314+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95600751e0 con 0x7f95600d0db0 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.314+0000 7f9547fff700 1 -- 172.21.15.97:0/3033729212 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9554003a40 con 0x7f95600d0db0 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.315+0000 7f9547fff700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954c06aed0 0x7f954c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.315+0000 7f9547fff700 1 -- 172.21.15.97:0/3033729212 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f955408a370 con 0x7f95600d0db0 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.315+0000 7f955e7fc700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954c06aed0 0x7f954c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:15.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.315+0000 7f955e7fc700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954c06aed0 0x7f954c06d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95600b7360 tx=0x7f9550009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:15.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.319+0000 7f9547fff700 1 -- 172.21.15.97:0/3033729212 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95540554b0 con 0x7f95600d0db0 2023-11-24T06:09:15.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.552+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95600008d0 con 0x7f95600d0db0 2023-11-24T06:09:15.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.553+0000 7f9547fff700 1 -- 172.21.15.97:0/3033729212 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f9554058ad0 con 0x7f95600d0db0 2023-11-24T06:09:15.557 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:15.557 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:08:57.902510+0000","modified":"2023-11-24T06:09:00.921422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:09:15.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954c06aed0 msgr2=0x7f954c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:15.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954c06aed0 0x7f954c06d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f95600b7360 tx=0x7f9550009210 comp rx=0 tx=0).stop 2023-11-24T06:09:15.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95600d0db0 msgr2=0x7f9560152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:15.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95600d0db0 0x7f9560152260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9554009c10 tx=0x7f9554004750 comp rx=0 tx=0).stop 2023-11-24T06:09:15.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 shutdown_connections 2023-11-24T06:09:15.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954c06aed0 0x7f954c06d390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f956015e1e0 0x7f95601f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95600d0db0 0x7f9560152260 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 --2- 172.21.15.97:0/3033729212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95600d3a00 0x7f95601f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:15.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 >> 172.21.15.97:0/3033729212 conn(0x7f95600b5b50 msgr2=0x7f95600ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:15.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 shutdown_connections 2023-11-24T06:09:15.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:15.555+0000 7f9565971700 1 -- 172.21.15.97:0/3033729212 wait complete. 2023-11-24T06:09:15.560 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-11-24T06:09:15.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:08:57.902510+0000', 'modified': '2023-11-24T06:09:00.921422+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6834/401436267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 401436267}, {'type': 'v1', 'addr': '172.21.15.183:6834', 'nonce': 401436267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:09:15.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:09:15.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-24T06:09:16.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.095+0000 7fe381f54700 1 Processor -- start 2023-11-24T06:09:16.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.095+0000 7fe381f54700 1 -- start start 2023-11-24T06:09:16.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.095+0000 7fe381f54700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe37c158a50 0x7fe37c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.095+0000 7fe381f54700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe37c162a30 0x7fe37c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.095+0000 7fe381f54700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c0cfff0 con 0x7fe37c158670 2023-11-24T06:09:16.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.095+0000 7fe381f54700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0629b0 con 0x7fe37c162a30 2023-11-24T06:09:16.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.095+0000 7fe381f54700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0cfd10 con 0x7fe37c158a50 2023-11-24T06:09:16.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe37c162a30 0x7fe37c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe37c162a30 0x7fe37c0c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45728/0 (socket says 172.21.15.97:45728) 2023-11-24T06:09:16.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37bfff700 1 -- 172.21.15.97:0/1135874223 learned_addr learned my addr 172.21.15.97:0/1135874223 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:16.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37affd700 1 --2- 172.21.15.97:0/1135874223 >> v2:172.21.15.183:3300/0 conn(0x7fe37c158a50 0x7fe37c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37bfff700 1 -- 172.21.15.97:0/1135874223 >> v2:172.21.15.183:3300/0 conn(0x7fe37c158a50 msgr2=0x7fe37c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37bfff700 1 --2- 172.21.15.97:0/1135874223 >> v2:172.21.15.183:3300/0 conn(0x7fe37c158a50 0x7fe37c0c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37bfff700 1 -- 172.21.15.97:0/1135874223 >> v1:172.21.15.183:6789/0 conn(0x7fe37c158670 legacy=0x7fe37c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37bfff700 1 -- 172.21.15.97:0/1135874223 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c161f50 con 0x7fe37c162a30 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37bfff700 1 --2- 172.21.15.97:0/1135874223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c162a30 0x7fe37c0c9df0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0ca730 tx=0x7fe37400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1d375ddfca7afb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37a7fc700 1 -- 172.21.15.97:0/1135874223 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400e070 con 0x7fe37c162a30 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37a7fc700 1 -- 172.21.15.97:0/1135874223 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374003a20 con 0x7fe37c162a30 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.096+0000 7fe37a7fc700 1 -- 172.21.15.97:0/1135874223 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374003dd0 con 0x7fe37c162a30 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.097+0000 7fe381f54700 1 -- 172.21.15.97:0/1135874223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c162a30 msgr2=0x7fe37c0c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.097+0000 7fe381f54700 1 --2- 172.21.15.97:0/1135874223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c162a30 0x7fe37c0c9df0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0ca730 tx=0x7fe37400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.097+0000 7fe381f54700 1 -- 172.21.15.97:0/1135874223 shutdown_connections 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.097+0000 7fe381f54700 1 --2- 172.21.15.97:0/1135874223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c162a30 0x7fe37c0c9df0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.097+0000 7fe381f54700 1 --2- 172.21.15.97:0/1135874223 >> v2:172.21.15.183:3300/0 conn(0x7fe37c158a50 0x7fe37c0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.097+0000 7fe381f54700 1 -- 172.21.15.97:0/1135874223 >> 172.21.15.97:0/1135874223 conn(0x7fe37c0b57b0 msgr2=0x7fe37c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:16.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.097+0000 7fe381f54700 1 -- 172.21.15.97:0/1135874223 shutdown_connections 2023-11-24T06:09:16.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.097+0000 7fe381f54700 1 -- 172.21.15.97:0/1135874223 wait complete. 2023-11-24T06:09:16.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.098+0000 7fe381f54700 1 Processor -- start 2023-11-24T06:09:16.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.099+0000 7fe381f54700 1 -- start start 2023-11-24T06:09:16.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.099+0000 7fe381f54700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c158670 0x7fe37c0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.099+0000 7fe381f54700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe37c158a50 0x7fe37c150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.099+0000 7fe381f54700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe37c162a30 0x7fe37c06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.099+0000 7fe381f54700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c151230 con 0x7fe37c158670 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.099+0000 7fe381f54700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0b4080 con 0x7fe37c158a50 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.099+0000 7fe381f54700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0b4360 con 0x7fe37c162a30 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c158670 0x7fe37c0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c158670 0x7fe37c0b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45730/0 (socket says 172.21.15.97:45730) 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1426946007 learned_addr learned my addr 172.21.15.97:0/1426946007 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37affd700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe37c158a50 0x7fe37c150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37bfff700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe37c162a30 0x7fe37c06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe37c162a30 msgr2=0x7fe37c06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe37c162a30 0x7fe37c06e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe37c158a50 msgr2=0x7fe37c150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe37c158a50 0x7fe37c150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1426946007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe374009b30 con 0x7fe37c158670 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37affd700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe37c158a50 0x7fe37c150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37bfff700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe37c162a30 0x7fe37c06e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c158670 0x7fe37c0b5120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c4d0 tx=0x7fe36c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe363fff700 1 -- 172.21.15.97:0/1426946007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c011790 con 0x7fe37c158670 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe363fff700 1 -- 172.21.15.97:0/1426946007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c0118f0 con 0x7fe37c158670 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe363fff700 1 -- 172.21.15.97:0/1426946007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c011ca0 con 0x7fe37c158670 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe381f54700 1 -- 172.21.15.97:0/1426946007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c0b4640 con 0x7fe37c158670 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.100+0000 7fe381f54700 1 -- 172.21.15.97:0/1426946007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c0b4c00 con 0x7fe37c158670 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.102+0000 7fe363fff700 1 -- 172.21.15.97:0/1426946007 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe36c01c460 con 0x7fe37c158670 2023-11-24T06:09:16.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.102+0000 7fe363fff700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe36806aed0 0x7fe36806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.102+0000 7fe37affd700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe36806aed0 0x7fe36806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.102+0000 7fe363fff700 1 -- 172.21.15.97:0/1426946007 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe36c0894b0 con 0x7fe37c158670 2023-11-24T06:09:16.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.102+0000 7fe37affd700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe36806aed0 0x7fe36806d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0cdf40 tx=0x7fe370008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.104+0000 7fe38e14db80 1 Processor -- start 2023-11-24T06:09:16.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.104+0000 7fe38e14db80 1 -- start start 2023-11-24T06:09:16.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 0x558531675840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558531644ed0 0x558531675d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531658e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe38e14db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585315a2770 con 0x55853162c490 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe38e14db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558531589750 con 0x558531644ed0 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe38e14db80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x558531587070 con 0x55853162f080 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531658e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531658e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45742/0 (socket says 172.21.15.97:45742) 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 -- 172.21.15.97:0/106328713 learned_addr learned my addr 172.21.15.97:0/106328713 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 0x558531675840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37affd700 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558531644ed0 0x558531675d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 -- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 msgr2=0x558531675840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 0x558531675840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 -- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558531644ed0 msgr2=0x558531675d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558531644ed0 0x558531675d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 -- 172.21.15.97:0/106328713 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585313ff5e0 con 0x55853162c490 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37affd700 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558531644ed0 0x558531675d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.105+0000 7fe37bfff700 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531658e70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe37400b370 tx=0x7fe374010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfd5a7a468ba5f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe3617fa700 1 -- 172.21.15.97:0/106328713 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374003e10 con 0x55853162c490 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe3617fa700 1 -- 172.21.15.97:0/106328713 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37400b980 con 0x55853162c490 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe3617fa700 1 -- 172.21.15.97:0/106328713 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400bd10 con 0x55853162c490 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 -- 172.21.15.97:0/106328713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 msgr2=0x558531658e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531658e70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe37400b370 tx=0x7fe374010000 comp rx=0 tx=0).stop 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 -- 172.21.15.97:0/106328713 shutdown_connections 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531658e70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 0x558531675840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 --2- 172.21.15.97:0/106328713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558531644ed0 0x558531675d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 -- 172.21.15.97:0/106328713 >> 172.21.15.97:0/106328713 conn(0x55853155d310 msgr2=0x558531673e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 -- 172.21.15.97:0/106328713 shutdown_connections 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.106+0000 7fe38e14db80 1 -- 172.21.15.97:0/106328713 wait complete. 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.107+0000 7fe38e14db80 1 -- start start 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.107+0000 7fe38e14db80 1 Processor -- start 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x55853166f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531644580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531623330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558531582960 con 0x55853162c490 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853153c510 con 0x55853162f080 2023-11-24T06:09:16.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55853153c1b0 con 0x558531644ed0 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 10 client.0 _ll_register_callbacks cb 0x558531467a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x55853166f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x55853166f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45744/0 (socket says 172.21.15.97:45744) 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1043672723 learned_addr learned my addr 172.21.15.97:0/1043672723 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37affd700 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531644580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37bfff700 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531623330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 msgr2=0x558531623330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531623330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 msgr2=0x558531644580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531644580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1043672723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c161f50 con 0x55853162c490 2023-11-24T06:09:16.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37affd700 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531644580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x55853166f640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe36c057f20 tx=0x7fe36c0582a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca6005f3a2d5925 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe35a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe35a7fc700 1 -- 172.21.15.97:0/1043672723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c060520 con 0x55853162c490 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe35a7fc700 1 -- 172.21.15.97:0/1043672723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe36c011e00 con 0x55853162c490 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 1 -- 172.21.15.97:0/1043672723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558531621a40 con 0x55853162c490 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe35a7fc700 1 -- 172.21.15.97:0/1043672723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe36c0890b0 con 0x55853162c490 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 10 client.4632 fetch_fsmap learned FSMap version 8 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.108+0000 7fe38e14db80 1 -- 172.21.15.97:0/1043672723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5585313abd90 con 0x55853162c490 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe35a7fc700 1 -- 172.21.15.97:0/1043672723 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558531621a40 con 0x55853162c490 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe35a7fc700 1 -- 172.21.15.97:0/1043672723 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe36c058480 con 0x55853162c490 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe38e14db80 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:09:16.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe38e14db80 10 client.4632 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe38e14db80 20 client.4632 populate_metadata read hostname 'smithi097' 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] conn(0x558531686d90 0x558531690950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe38e14db80 4 client.4632 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe38e14db80 1 -- 172.21.15.97:0/1043672723 --> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585315fd840 con 0x558531686d90 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.109+0000 7fe37affd700 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] conn(0x558531686d90 0x558531690950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.110+0000 7fe35a7fc700 10 client.4632 ms_handle_connect on v2:172.21.15.183:6832/401436267 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.110+0000 7fe37affd700 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] conn(0x558531686d90 0x558531690950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=922b996110195a84 server_cookie=83ab6006c1b27bd in_seq=0 out_seq=0 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe35a7fc700 1 -- 172.21.15.97:0/1043672723 <== mds.0 v2:172.21.15.183:6832/401436267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585315fd840 con 0x558531686d90 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe35a7fc700 10 client.4632 handle_command_reply: tid=0 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 client.4632 shutdown 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 -- 172.21.15.97:0/1043672723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 msgr2=0x55853166f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x55853166f640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe36c057f20 tx=0x7fe36c0582a0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 -- 172.21.15.97:0/1043672723 shutdown_connections 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] conn(0x558531686d90 0x558531690950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531623330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe35a7fc700 0 client.4632 ms_handle_reset on v2:172.21.15.183:6832/401436267 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x55853166f640 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1043672723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531644580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 -- 172.21.15.97:0/1043672723 >> 172.21.15.97:0/1043672723 conn(0x55853155d310 msgr2=0x558531677c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 -- 172.21.15.97:0/1043672723 shutdown_connections 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 1 -- 172.21.15.97:0/1043672723 wait complete. 2023-11-24T06:09:16.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.111+0000 7fe38e14db80 20 client.4632 trim_cache size 0 max 16384 2023-11-24T06:09:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.129+0000 7fe38e14db80 1 Processor -- start 2023-11-24T06:09:16.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.129+0000 7fe38e14db80 1 -- start start 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531673d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531622ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531694ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe38e14db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585316739d0 con 0x55853162c490 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe38e14db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585314018d0 con 0x55853162f080 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe38e14db80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x558531686840 con 0x558531644ed0 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531673d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531673d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45752/0 (socket says 172.21.15.97:45752) 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1229497624 learned_addr learned my addr 172.21.15.97:0/1229497624 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:16.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37affd700 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531622ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37bfff700 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531694ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 msgr2=0x558531694ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531694ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 msgr2=0x558531622ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531622ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37b7fe700 1 -- 172.21.15.97:0/1229497624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558531394c10 con 0x55853162c490 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.130+0000 7fe37bfff700 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531694ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531673d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe36c05c250 tx=0x7fe36c05a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4687e3d1ecdd9fa6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe361ffb700 1 -- 172.21.15.97:0/1229497624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c07b070 con 0x55853162c490 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe361ffb700 1 -- 172.21.15.97:0/1229497624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c00e040 con 0x55853162c490 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe361ffb700 1 -- 172.21.15.97:0/1229497624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0039b0 con 0x55853162c490 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 -- 172.21.15.97:0/1229497624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 msgr2=0x558531673d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531673d20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe36c05c250 tx=0x7fe36c05a000 comp rx=0 tx=0).stop 2023-11-24T06:09:16.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 -- 172.21.15.97:0/1229497624 shutdown_connections 2023-11-24T06:09:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558531644ed0 0x558531694ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55853162c490 0x558531673d20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 --2- 172.21.15.97:0/1229497624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162f080 0x558531622ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 -- 172.21.15.97:0/1229497624 >> 172.21.15.97:0/1229497624 conn(0x55853155d310 msgr2=0x55853166f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 -- 172.21.15.97:0/1229497624 shutdown_connections 2023-11-24T06:09:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.131+0000 7fe38e14db80 1 -- 172.21.15.97:0/1229497624 wait complete. 2023-11-24T06:09:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.132+0000 7fe38e14db80 1 -- start start 2023-11-24T06:09:16.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.132+0000 7fe38e14db80 1 Processor -- start 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162c490 0x558531676c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 0x558531682570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558531644ed0 0x558531682ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558531694970 con 0x558531644ed0 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55853166dcb0 con 0x55853162c490 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x558531694b80 con 0x55853162f080 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 10 client.0 _ll_register_callbacks cb 0x558531467a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558531644ed0 0x558531682ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558531644ed0 0x558531682ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45762/0 (socket says 172.21.15.97:45762) 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 -- 172.21.15.97:0/67238933 learned_addr learned my addr 172.21.15.97:0/67238933 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37affd700 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 0x558531682570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162c490 0x558531676c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 -- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 msgr2=0x558531682570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 0x558531682570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 -- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162c490 msgr2=0x558531676c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162c490 0x558531676c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 -- 172.21.15.97:0/67238933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585313abd90 con 0x558531644ed0 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe37bfff700 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558531644ed0 0x558531682ab0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0722a0 tx=0x7fe374004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a023098c380c3f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe35a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:09:16.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe35a7fc700 1 -- 172.21.15.97:0/67238933 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374004500 con 0x558531644ed0 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe35a7fc700 1 -- 172.21.15.97:0/67238933 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe374004660 con 0x558531644ed0 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 1 -- 172.21.15.97:0/67238933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558531548f80 con 0x558531644ed0 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe35a7fc700 1 -- 172.21.15.97:0/67238933 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe374004820 con 0x558531644ed0 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.133+0000 7fe38e14db80 10 client.4638 fetch_fsmap learned FSMap version 8 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe38e14db80 1 -- 172.21.15.97:0/67238933 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5585313d2200 con 0x558531644ed0 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe35a7fc700 1 -- 172.21.15.97:0/67238933 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558531548f80 con 0x558531644ed0 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe35a7fc700 1 -- 172.21.15.97:0/67238933 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe37402e430 con 0x558531644ed0 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe38e14db80 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe38e14db80 10 client.4638 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe38e14db80 20 client.4638 populate_metadata read hostname 'smithi097' 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe38e14db80 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] conn(0x55853166d1a0 0x5585316773f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe38e14db80 4 client.4638 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe38e14db80 1 -- 172.21.15.97:0/67238933 --> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55853153b440 con 0x55853166d1a0 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] conn(0x55853166d1a0 0x5585316773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe35a7fc700 10 client.4638 ms_handle_connect on v2:172.21.15.183:6832/401436267 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe37b7fe700 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] conn(0x55853166d1a0 0x5585316773f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2896066180a2699d server_cookie=2cfbca65595b2348 in_seq=0 out_seq=0 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe35a7fc700 1 -- 172.21.15.97:0/67238933 <== mds.0 v2:172.21.15.183:6832/401436267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55853153b440 con 0x55853166d1a0 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe35a7fc700 10 client.4638 handle_command_reply: tid=0 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.134+0000 7fe38e14db80 1 client.4638 shutdown 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 -- 172.21.15.97:0/67238933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558531644ed0 msgr2=0x558531682ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558531644ed0 0x558531682ab0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0722a0 tx=0x7fe374004b40 comp rx=0 tx=0).stop 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 -- 172.21.15.97:0/67238933 shutdown_connections 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:6832/401436267,v1:172.21.15.183:6834/401436267] conn(0x55853166d1a0 0x5585316773f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558531644ed0 0x558531682ab0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe35a7fc700 0 client.4638 ms_handle_reset on v2:172.21.15.183:6832/401436267 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55853162c490 0x558531676c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 --2- 172.21.15.97:0/67238933 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55853162f080 0x558531682570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 -- 172.21.15.97:0/67238933 >> 172.21.15.97:0/67238933 conn(0x55853155d310 msgr2=0x558531670f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 -- 172.21.15.97:0/67238933 shutdown_connections 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 1 -- 172.21.15.97:0/67238933 wait complete. 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe38e14db80 20 client.4638 trim_cache size 0 max 16384 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9", 2023-11-24T06:09:16.146 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 4291, 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 17.620378336000002, 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 8, 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 20, 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 20, 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 25.234249393999999 2023-11-24T06:09:16.147 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:09:16.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 -- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe36806aed0 msgr2=0x7fe36806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe36806aed0 0x7fe36806d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0cdf40 tx=0x7fe370008040 comp rx=0 tx=0).stop 2023-11-24T06:09:16.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 -- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c158670 msgr2=0x7fe37c0b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c158670 0x7fe37c0b5120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00c4d0 tx=0x7fe36c00c890 comp rx=0 tx=0).stop 2023-11-24T06:09:16.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 -- 172.21.15.97:0/1426946007 shutdown_connections 2023-11-24T06:09:16.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe36806aed0 0x7fe36806d390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe37c162a30 0x7fe37c06e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe37c158670 0x7fe37c0b5120 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 --2- 172.21.15.97:0/1426946007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe37c158a50 0x7fe37c150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 -- 172.21.15.97:0/1426946007 >> 172.21.15.97:0/1426946007 conn(0x7fe37c0b57b0 msgr2=0x7fe37c0b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:16.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 -- 172.21.15.97:0/1426946007 shutdown_connections 2023-11-24T06:09:16.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.135+0000 7fe381f54700 1 -- 172.21.15.97:0/1426946007 wait complete. 2023-11-24T06:09:16.151 INFO:teuthology.run_tasks:Running task kclient... 2023-11-24T06:09:16.173 INFO:tasks.kclient:Mounting kernel clients... 2023-11-24T06:09:16.173 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T06:09:16.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.173 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:09:16.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-11-24T06:09:16.258 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2023-11-24T06:09:16.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:09:16.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.260 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:09:16.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.317 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-11-24T06:09:16.384 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2023-11-24T06:09:16.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:09:16.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.386 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:09:16.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.408 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link delete ceph-brx 2023-11-24T06:09:16.483 INFO:teuthology.orchestra.run.smithi183.stderr:Cannot find device "ceph-brx" 2023-11-24T06:09:16.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:09:16.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.484 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:09:16.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link delete ceph-brx 2023-11-24T06:09:16.613 INFO:teuthology.orchestra.run.smithi183.stderr:Cannot find device "ceph-brx" 2023-11-24T06:09:16.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:09:16.615 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-24T06:09:16.615 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:09:16.615 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:09:16.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:09:16.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.733+0000 7fe1a00c2700 1 Processor -- start 2023-11-24T06:09:16.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.733+0000 7fe1a00c2700 1 -- start start 2023-11-24T06:09:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe1a00c2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 0x7fe19806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe1a00c2700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1980d0630 0x7fe19806c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe1a00c2700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19815c4b0 con 0x7fe1980b49a0 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe1a00c2700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19815c790 con 0x7fe1980d0630 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe1a00c2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19815ca70 con 0x7fe198152cc0 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe19e65f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1980d0630 0x7fe19806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe19e65f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe1980d0630 0x7fe19806c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45770/0 (socket says 172.21.15.97:45770) 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe19e65f700 1 -- 172.21.15.97:0/337075916 learned_addr learned my addr 172.21.15.97:0/337075916 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe19de5e700 1 --2- 172.21.15.97:0/337075916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 0x7fe19806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe19e65f700 1 -- 172.21.15.97:0/337075916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 msgr2=0x7fe19806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe19e65f700 1 --2- 172.21.15.97:0/337075916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 0x7fe19806c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe19e65f700 1 -- 172.21.15.97:0/337075916 >> v1:172.21.15.183:6789/0 conn(0x7fe1980b49a0 legacy=0x7fe1980cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.734+0000 7fe19e65f700 1 -- 172.21.15.97:0/337075916 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19815cd50 con 0x7fe1980d0630 2023-11-24T06:09:16.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe19e65f700 1 --2- 172.21.15.97:0/337075916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe19806c540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe190009e50 tx=0x7fe19000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa020043cabab23d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe19ce5c700 1 -- 172.21.15.97:0/337075916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19000e070 con 0x7fe1980d0630 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe19ce5c700 1 -- 172.21.15.97:0/337075916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19000b910 con 0x7fe1980d0630 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe19ce5c700 1 -- 172.21.15.97:0/337075916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19000bcc0 con 0x7fe1980d0630 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe1a00c2700 1 -- 172.21.15.97:0/337075916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 msgr2=0x7fe19806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/337075916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe19806c540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe190009e50 tx=0x7fe19000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe1a00c2700 1 -- 172.21.15.97:0/337075916 shutdown_connections 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/337075916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe19806c540 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/337075916 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 0x7fe19806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe1a00c2700 1 -- 172.21.15.97:0/337075916 >> 172.21.15.97:0/337075916 conn(0x7fe198071b90 msgr2=0x7fe198071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe1a00c2700 1 -- 172.21.15.97:0/337075916 shutdown_connections 2023-11-24T06:09:16.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.735+0000 7fe1a00c2700 1 -- 172.21.15.97:0/337075916 wait complete. 2023-11-24T06:09:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.736+0000 7fe1a00c2700 1 Processor -- start 2023-11-24T06:09:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.736+0000 7fe1a00c2700 1 -- start start 2023-11-24T06:09:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe1a00c2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1980b49a0 0x7fe198158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe1a00c2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe1981f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe1a00c2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 0x7fe1980b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe1a00c2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe198158cb0 con 0x7fe1980d0630 2023-11-24T06:09:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe1a00c2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe198158f50 con 0x7fe1980b49a0 2023-11-24T06:09:16.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe1a00c2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1981532b0 con 0x7fe198152cc0 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe19d65d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe1981f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe19d65d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe1981f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45774/0 (socket says 172.21.15.97:45774) 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe19d65d700 1 -- 172.21.15.97:0/2915768760 learned_addr learned my addr 172.21.15.97:0/2915768760 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe19e65f700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 0x7fe1980b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.737+0000 7fe19de5e700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1980b49a0 0x7fe198158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe19d65d700 1 -- 172.21.15.97:0/2915768760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 msgr2=0x7fe1980b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe19d65d700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 0x7fe1980b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe19d65d700 1 -- 172.21.15.97:0/2915768760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1980b49a0 msgr2=0x7fe198158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe19d65d700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1980b49a0 0x7fe198158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe19d65d700 1 -- 172.21.15.97:0/2915768760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe190009b30 con 0x7fe1980d0630 2023-11-24T06:09:16.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe19d65d700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe1981f0e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c4e0 tx=0x7fe19400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe18e7fc700 1 -- 172.21.15.97:0/2915768760 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1940052d0 con 0x7fe1980d0630 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe18e7fc700 1 -- 172.21.15.97:0/2915768760 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe194005430 con 0x7fe1980d0630 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe198153590 con 0x7fe1980d0630 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe18e7fc700 1 -- 172.21.15.97:0/2915768760 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1940038c0 con 0x7fe1980d0630 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.738+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe198153b50 con 0x7fe1980d0630 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.739+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe198147af0 con 0x7fe1980d0630 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.739+0000 7fe18e7fc700 1 -- 172.21.15.97:0/2915768760 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe194003a20 con 0x7fe1980d0630 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.740+0000 7fe18e7fc700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe17c06af00 0x7fe17c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.740+0000 7fe18e7fc700 1 -- 172.21.15.97:0/2915768760 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1940889f0 con 0x7fe1980d0630 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.740+0000 7fe19de5e700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe17c06af00 0x7fe17c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:16.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.740+0000 7fe19de5e700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe17c06af00 0x7fe17c06d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe188009b40 tx=0x7fe188009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:16.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.744+0000 7fe18e7fc700 1 -- 172.21.15.97:0/2915768760 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe194053a10 con 0x7fe1980d0630 2023-11-24T06:09:16.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.979+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe1980753f0 con 0x7fe1980d0630 2023-11-24T06:09:16.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.980+0000 7fe18e7fc700 1 -- 172.21.15.97:0/2915768760 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe194017020 con 0x7fe1980d0630 2023-11-24T06:09:16.984 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:09:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe17c06af00 msgr2=0x7fe17c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe17c06af00 0x7fe17c06d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe188009b40 tx=0x7fe188009210 comp rx=0 tx=0).stop 2023-11-24T06:09:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 msgr2=0x7fe1981f0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe1981f0e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe19400c4e0 tx=0x7fe19400c7f0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 shutdown_connections 2023-11-24T06:09:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe17c06af00 0x7fe17c06d3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe198152cc0 0x7fe1980b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1980b49a0 0x7fe198158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 --2- 172.21.15.97:0/2915768760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1980d0630 0x7fe1981f0e00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 >> 172.21.15.97:0/2915768760 conn(0x7fe198071b90 msgr2=0x7fe198159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 shutdown_connections 2023-11-24T06:09:16.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:16.982+0000 7fe1a00c2700 1 -- 172.21.15.97:0/2915768760 wait complete. 2023-11-24T06:09:16.998 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:09:16.999 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:09:16.999 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-24T06:09:16.999 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:09:16.999 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:09:16.999 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:09:16.999 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:09:16.999 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:09:16.999 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:09:16.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:16.999 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-24T06:09:17.016 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:09:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:09:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:09:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:09:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:17.016 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:17.016 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42180sec preferred_lft 42180sec 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c9d4:801c:12b3:b866/64 scope link noprefixroute 2023-11-24T06:09:17.017 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:17.018 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:09:17.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.018 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:17.018 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:17.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:09:17.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2023-11-24T06:09:17.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2023-11-24T06:09:17.018 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:09:17.018 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:17.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.200 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:09:17.228 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-24T06:09:17.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.230 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:09:17.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.300 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:17.300 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:17.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:09:17.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:09:17.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:09:17.300 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:17.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.579 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:09:17.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.596 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2023-11-24T06:09:17.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.652 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:17.652 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:17.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:09:17.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T06:09:17.652 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:17.746 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T06:09:17.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.746 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:17.746 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:17.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T06:09:17.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:09:17.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T06:09:17.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T06:09:17.747 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T06:09:17.747 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:17.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:17.954 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:17.955 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:17.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-24T06:09:17.955 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:09:17.955 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:18.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:09:18.054 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:09:18.083 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:09:18.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:09:18.083 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:09:18.139 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:09:18.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:18.140 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:18.234 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:18.234 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:18.357 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:18.357 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:18.357 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:09:18.357 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:09:18.357 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:09:18.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:09:18.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:09:18.358 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:09:18.434 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:09:18.435 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:09:18.435 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:09:18.437 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:09:18.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:18.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:09:18.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:18.464 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:18.551 INFO:teuthology.orchestra.run.smithi097.stdout:client4621 2023-11-24T06:09:18.552 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:09:18.552 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:18.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:18.569 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:18.656 INFO:teuthology.orchestra.run.smithi097.stdout:client4621 2023-11-24T06:09:18.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:18.656 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run.smithi097.stdout:client4621 2023-11-24T06:09:18.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:18.704 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.client4621/status') 2023-11-24T06:09:18.756 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4621 (3)172.21.15.97:0/3050357935 2023-11-24T06:09:18.756 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2023-11-24T06:09:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2023-11-24T06:09:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2023-11-24T06:09:18.757 INFO:teuthology.orchestra.run.smithi097.stderr:70 bytes copied, 3.6115e-05 s, 1.9 MB/s 2023-11-24T06:09:18.758 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-24T06:09:18.758 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:09:18.759 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:09:18.759 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:09:18.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.876+0000 7f7b9c1bf700 1 Processor -- start 2023-11-24T06:09:18.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.876+0000 7f7b9c1bf700 1 -- start start 2023-11-24T06:09:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b9c1bf700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b9415a3b0 0x7f7b940b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:18.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b9c1bf700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b940cf360 0x7f7b940b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b9c1bf700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9415a790 con 0x7f7b940b3da0 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b9c1bf700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9415eec0 con 0x7f7b9415a3b0 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b9c1bf700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b940cf8b0 con 0x7f7b940cf360 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b99f5b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b9415a3b0 0x7f7b940b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b9a75c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7b940b3da0 0x7f7b9406dd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38248/0 (socket says 172.21.15.97:38248) 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b99f5b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7b9415a3b0 0x7f7b940b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45792/0 (socket says 172.21.15.97:45792) 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b9a75c700 1 -- 172.21.15.97:0/2014590919 learned_addr learned my addr 172.21.15.97:0/2014590919 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b9975a700 1 --2- 172.21.15.97:0/2014590919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b940cf360 0x7f7b940b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b99f5b700 1 -- 172.21.15.97:0/2014590919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b940cf360 msgr2=0x7f7b940b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:18.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b99f5b700 1 --2- 172.21.15.97:0/2014590919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b940cf360 0x7f7b940b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b99f5b700 1 -- 172.21.15.97:0/2014590919 >> v1:172.21.15.97:6789/0 conn(0x7f7b940b3da0 legacy=0x7f7b9406dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b99f5b700 1 -- 172.21.15.97:0/2014590919 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b940b4c40 con 0x7f7b9415a3b0 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b98f59700 1 -- 172.21.15.97:0/2014590919 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545886492 0 0) 0x7f7b9415a790 con 0x7f7b940b3da0 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.877+0000 7f7b99f5b700 1 --2- 172.21.15.97:0/2014590919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9415a3b0 0x7f7b940b3320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009870 tx=0x7f7b90009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12f5951f2678aa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b98f59700 1 -- 172.21.15.97:0/2014590919 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90011830 con 0x7f7b9415a3b0 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b98f59700 1 -- 172.21.15.97:0/2014590919 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b90011990 con 0x7f7b9415a3b0 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b98f59700 1 -- 172.21.15.97:0/2014590919 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90011d20 con 0x7f7b9415a3b0 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/2014590919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9415a3b0 msgr2=0x7f7b940b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/2014590919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9415a3b0 0x7f7b940b3320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009870 tx=0x7f7b90009b80 comp rx=0 tx=0).stop 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/2014590919 shutdown_connections 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/2014590919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b9415a3b0 0x7f7b940b3320 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:18.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/2014590919 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b940cf360 0x7f7b940b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:18.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/2014590919 >> 172.21.15.97:0/2014590919 conn(0x7f7b940b1740 msgr2=0x7f7b940b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:18.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/2014590919 shutdown_connections 2023-11-24T06:09:18.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.878+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/2014590919 wait complete. 2023-11-24T06:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.880+0000 7f7b9c1bf700 1 Processor -- start 2023-11-24T06:09:18.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.880+0000 7f7b9c1bf700 1 -- start start 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.880+0000 7f7b9c1bf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b940b3da0 0x7f7b94148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.880+0000 7f7b9c1bf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b940cf360 0x7f7b94158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.880+0000 7f7b9c1bf700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b9415a3b0 0x7f7b94156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.880+0000 7f7b9c1bf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94157f50 con 0x7f7b940b3da0 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.880+0000 7f7b9c1bf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94155ef0 con 0x7f7b940cf360 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.880+0000 7f7b9c1bf700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b941572d0 con 0x7f7b9415a3b0 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b940b3da0 0x7f7b94148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b940b3da0 0x7f7b94148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45798/0 (socket says 172.21.15.97:45798) 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 -- 172.21.15.97:0/927710903 learned_addr learned my addr 172.21.15.97:0/927710903 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b9a75c700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b9415a3b0 0x7f7b94156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:18.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 -- 172.21.15.97:0/927710903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b9415a3b0 msgr2=0x7f7b94156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b9415a3b0 0x7f7b94156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b9975a700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b940cf360 0x7f7b94158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 -- 172.21.15.97:0/927710903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b940cf360 msgr2=0x7f7b94158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b940cf360 0x7f7b94158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 -- 172.21.15.97:0/927710903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900094a0 con 0x7f7b940b3da0 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b99f5b700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b940b3da0 0x7f7b94148900 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b900090b0 tx=0x7f7b90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b8a7fc700 1 -- 172.21.15.97:0/927710903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b900045f0 con 0x7f7b940b3da0 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b8a7fc700 1 -- 172.21.15.97:0/927710903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9001f070 con 0x7f7b940b3da0 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b941575b0 con 0x7f7b940b3da0 2023-11-24T06:09:18.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b8a7fc700 1 -- 172.21.15.97:0/927710903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90003ac0 con 0x7f7b940b3da0 2023-11-24T06:09:18.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.881+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b94157b70 con 0x7f7b940b3da0 2023-11-24T06:09:18.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.882+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b940751e0 con 0x7f7b940b3da0 2023-11-24T06:09:18.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.882+0000 7f7b8a7fc700 1 -- 172.21.15.97:0/927710903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b90003c20 con 0x7f7b940b3da0 2023-11-24T06:09:18.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.883+0000 7f7b8a7fc700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b7806aed0 0x7f7b7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:18.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.883+0000 7f7b8a7fc700 1 -- 172.21.15.97:0/927710903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b90021070 con 0x7f7b940b3da0 2023-11-24T06:09:18.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.883+0000 7f7b9975a700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b7806aed0 0x7f7b7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:18.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.883+0000 7f7b9975a700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b7806aed0 0x7f7b7806d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b9406ede0 tx=0x7f7b84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:18.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:18.887+0000 7f7b8a7fc700 1 -- 172.21.15.97:0/927710903 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b90056160 con 0x7f7b940b3da0 2023-11-24T06:09:19.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.121+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7b940b3100 con 0x7f7b940b3da0 2023-11-24T06:09:19.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.122+0000 7f7b8a7fc700 1 -- 172.21.15.97:0/927710903 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7b9002b070 con 0x7f7b940b3da0 2023-11-24T06:09:19.126 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b7806aed0 msgr2=0x7f7b7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b7806aed0 0x7f7b7806d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b9406ede0 tx=0x7f7b84008040 comp rx=0 tx=0).stop 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b940b3da0 msgr2=0x7f7b94148900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b940b3da0 0x7f7b94148900 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b900090b0 tx=0x7f7b90004750 comp rx=0 tx=0).stop 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 shutdown_connections 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b7806aed0 0x7f7b7806d390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b9415a3b0 0x7f7b94156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b940b3da0 0x7f7b94148900 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:19.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 --2- 172.21.15.97:0/927710903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b940cf360 0x7f7b94158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 >> 172.21.15.97:0/927710903 conn(0x7f7b940b1740 msgr2=0x7f7b940cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 shutdown_connections 2023-11-24T06:09:19.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:19.124+0000 7f7b9c1bf700 1 -- 172.21.15.97:0/927710903 wait complete. 2023-11-24T06:09:19.140 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:09:19.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:09:19.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-24T06:09:19.140 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T06:09:19.140 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:09:19.140 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:09:19.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:09:19.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:09:19.141 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:09:19.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.141 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:19.159 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42178sec preferred_lft 42178sec 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c9d4:801c:12b3:b866/64 scope link noprefixroute 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether a2:28:0d:2d:82:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:09:19.160 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.161 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::7c25:b1ff:fe16:36f1/64 scope link 2023-11-24T06:09:19.161 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.161 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:09:19.161 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether a2:28:0d:2d:82:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:09:19.161 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a028:dff:fe2d:822b/64 scope link tentative 2023-11-24T06:09:19.161 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.161 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:09:19.224 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-24T06:09:19.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.226 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-24T06:09:19.284 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-24T06:09:19.284 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:09:19.284 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:09:19.284 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:09:19.284 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:09:19.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.285 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:19.285 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:19.285 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:09:19.285 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:09:19.285 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:09:19.285 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:19.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.389 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:09:19.405 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:09:19.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.406 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2023-11-24T06:09:19.462 INFO:teuthology.orchestra.run.smithi097.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T06:09:19.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.462 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:19.463 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:19.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:09:19.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T06:09:19.463 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:19.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.556 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:09:19.592 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:09:19.592 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:09:19.592 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 26:35:56:db:48:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2435:56ff:fedb:4834/64 scope link 2023-11-24T06:09:19.593 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:09:19.691 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 26:35:56:db:48:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::2435:56ff:fedb:4834/64 scope link 2023-11-24T06:09:19.692 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:19.705 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T06:09:19.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.706 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:19.707 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:19.707 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T06:09:19.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:09:19.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T06:09:19.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T06:09:19.709 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T06:09:19.709 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:19.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:19.965 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:09:19.965 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:09:19.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.1 up 2023-11-24T06:09:19.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:09:19.966 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:09:20.073 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:09:20.073 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T06:09:20.096 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:09:20.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:09:20.096 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:09:20.153 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-24T06:09:20.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:20.153 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:20.238 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:20.238 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:20.264 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:20.264 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:20.264 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:09:20.264 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 2@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:09:20.264 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:09:20.264 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:09:20.264 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:09:20.264 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:09:20.339 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 2@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:09:20.340 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:09:20.341 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:09:20.342 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:09:20.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:20.342 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:09:20.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:20.368 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:20.441 INFO:teuthology.orchestra.run.smithi097.stdout:client4484 2023-11-24T06:09:20.441 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:09:20.441 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:20.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:20.500 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:20.572 INFO:teuthology.orchestra.run.smithi097.stdout:client4484 2023-11-24T06:09:20.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:20.572 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:20.644 INFO:teuthology.orchestra.run.smithi097.stdout:client4484 2023-11-24T06:09:20.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:20.644 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.client4484/status') 2023-11-24T06:09:20.723 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4484 (3)172.21.15.97:0/1828861593 2023-11-24T06:09:20.723 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2023-11-24T06:09:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2023-11-24T06:09:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2023-11-24T06:09:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:70 bytes copied, 2.2539e-05 s, 3.1 MB/s 2023-11-24T06:09:20.724 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-24T06:09:20.724 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:09:20.724 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:09:20.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:09:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.880+0000 7fb65020d700 1 Processor -- start 2023-11-24T06:09:20.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.881+0000 7fb65020d700 1 -- start start 2023-11-24T06:09:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb65020d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6480b0ba0 0x7fb6480af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb65020d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb64815ee90 0x7fb64815b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:20.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb65020d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6480b2bd0 con 0x7fb6480b0f80 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb65020d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6480b1fc0 con 0x7fb6480b0ba0 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb65020d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480b22a0 con 0x7fb64815ee90 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64dfa9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6480b0ba0 0x7fb6480af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64dfa9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6480b0ba0 0x7fb6480af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45810/0 (socket says 172.21.15.97:45810) 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64dfa9700 1 -- 172.21.15.97:0/1542003711 learned_addr learned my addr 172.21.15.97:0/1542003711 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64e7aa700 1 --2- 172.21.15.97:0/1542003711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb64815ee90 0x7fb64815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64dfa9700 1 -- 172.21.15.97:0/1542003711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb64815ee90 msgr2=0x7fb64815b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64dfa9700 1 --2- 172.21.15.97:0/1542003711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb64815ee90 0x7fb64815b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64dfa9700 1 -- 172.21.15.97:0/1542003711 >> v1:172.21.15.183:6789/0 conn(0x7fb6480b0f80 legacy=0x7fb6480cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64dfa9700 1 -- 172.21.15.97:0/1542003711 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480b2580 con 0x7fb6480b0ba0 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64e7aa700 1 --2- 172.21.15.97:0/1542003711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb64815ee90 0x7fb64815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.882+0000 7fb64dfa9700 1 --2- 172.21.15.97:0/1542003711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480b0ba0 0x7fb6480af5c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098f0 tx=0x7fb638009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af2b2a05b921df7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:20.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb64cfa7700 1 -- 172.21.15.97:0/1542003711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638004230 con 0x7fb6480b0ba0 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb64cfa7700 1 -- 172.21.15.97:0/1542003711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb638004390 con 0x7fb6480b0ba0 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb64cfa7700 1 -- 172.21.15.97:0/1542003711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380036a0 con 0x7fb6480b0ba0 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb65020d700 1 -- 172.21.15.97:0/1542003711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480b0ba0 msgr2=0x7fb6480af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb65020d700 1 --2- 172.21.15.97:0/1542003711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480b0ba0 0x7fb6480af5c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098f0 tx=0x7fb638009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb65020d700 1 -- 172.21.15.97:0/1542003711 shutdown_connections 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb65020d700 1 --2- 172.21.15.97:0/1542003711 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb64815ee90 0x7fb64815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb65020d700 1 --2- 172.21.15.97:0/1542003711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480b0ba0 0x7fb6480af5c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb65020d700 1 -- 172.21.15.97:0/1542003711 >> 172.21.15.97:0/1542003711 conn(0x7fb6480b57b0 msgr2=0x7fb6480b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb65020d700 1 -- 172.21.15.97:0/1542003711 shutdown_connections 2023-11-24T06:09:20.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.883+0000 7fb65020d700 1 -- 172.21.15.97:0/1542003711 wait complete. 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.885+0000 7fb65020d700 1 Processor -- start 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.885+0000 7fb65020d700 1 -- start start 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb65020d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6480b0f80 0x7fb648159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb65020d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb64815ee90 0x7fb6480ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb65020d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6480ae3d0 0x7fb648061a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb65020d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb648159e80 con 0x7fb64815ee90 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb65020d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480d0760 con 0x7fb6480ae3d0 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb65020d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480cfb50 con 0x7fb6480b0f80 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb64815ee90 0x7fb6480ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb64815ee90 0x7fb6480ade90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45818/0 (socket says 172.21.15.97:45818) 2023-11-24T06:09:20.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 -- 172.21.15.97:0/210593610 learned_addr learned my addr 172.21.15.97:0/210593610 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 -- 172.21.15.97:0/210593610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6480b0f80 msgr2=0x7fb648159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64dfa9700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6480b0f80 0x7fb648159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64e7aa700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6480ae3d0 0x7fb648061a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6480b0f80 0x7fb648159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 -- 172.21.15.97:0/210593610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6480ae3d0 msgr2=0x7fb648061a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6480ae3d0 0x7fb648061a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 -- 172.21.15.97:0/210593610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638009550 con 0x7fb64815ee90 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb64d7a8700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb64815ee90 0x7fb6480ade90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb64400e910 tx=0x7fb64400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb63e7fc700 1 -- 172.21.15.97:0/210593610 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644004060 con 0x7fb64815ee90 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb63e7fc700 1 -- 172.21.15.97:0/210593610 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6440041e0 con 0x7fb64815ee90 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6480cfe30 con 0x7fb64815ee90 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb63e7fc700 1 -- 172.21.15.97:0/210593610 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644011470 con 0x7fb64815ee90 2023-11-24T06:09:20.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.886+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6480d03f0 con 0x7fb64815ee90 2023-11-24T06:09:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.887+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb648067f60 con 0x7fb64815ee90 2023-11-24T06:09:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.888+0000 7fb63e7fc700 1 -- 172.21.15.97:0/210593610 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6440115d0 con 0x7fb64815ee90 2023-11-24T06:09:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.888+0000 7fb63e7fc700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb62c06aed0 0x7fb62c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.888+0000 7fb63e7fc700 1 -- 172.21.15.97:0/210593610 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb64408bab0 con 0x7fb64815ee90 2023-11-24T06:09:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.888+0000 7fb64dfa9700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb62c06aed0 0x7fb62c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:20.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.889+0000 7fb64dfa9700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb62c06aed0 0x7fb62c06d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6480d1310 tx=0x7fb638008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:20.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:20.893+0000 7fb63e7fc700 1 -- 172.21.15.97:0/210593610 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb644056b60 con 0x7fb64815ee90 2023-11-24T06:09:21.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.126+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb6480d08a0 con 0x7fb64815ee90 2023-11-24T06:09:21.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.127+0000 7fb63e7fc700 1 -- 172.21.15.97:0/210593610 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb64405a180 con 0x7fb64815ee90 2023-11-24T06:09:21.131 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.128+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb62c06aed0 msgr2=0x7fb62c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb62c06aed0 0x7fb62c06d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb6480d1310 tx=0x7fb638008030 comp rx=0 tx=0).stop 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb64815ee90 msgr2=0x7fb6480ade90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb64815ee90 0x7fb6480ade90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb64400e910 tx=0x7fb64400ecd0 comp rx=0 tx=0).stop 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 shutdown_connections 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6480ae3d0 0x7fb648061a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb62c06aed0 0x7fb62c06d390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6480b0f80 0x7fb648159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 --2- 172.21.15.97:0/210593610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb64815ee90 0x7fb6480ade90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 >> 172.21.15.97:0/210593610 conn(0x7fb6480b57b0 msgr2=0x7fb6480cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:21.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 shutdown_connections 2023-11-24T06:09:21.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:21.129+0000 7fb65020d700 1 -- 172.21.15.97:0/210593610 wait complete. 2023-11-24T06:09:21.144 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:09:21.145 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:09:21.145 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-11-24T06:09:21.145 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:09:21.145 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:09:21.145 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:09:21.145 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:09:21.145 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:09:21.145 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:09:21.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.145 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-11-24T06:09:21.166 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:09:21.166 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:09:21.167 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:21.168 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:09:21.168 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42168sec preferred_lft 42168sec 2023-11-24T06:09:21.168 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::5382:2bb9:1085:5950/64 scope link noprefixroute 2023-11-24T06:09:21.168 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:21.168 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:09:21.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.168 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:21.168 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:21.168 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:09:21.169 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip addr flush dev ceph-brx 2023-11-24T06:09:21.169 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set ceph-brx up 2023-11-24T06:09:21.169 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:09:21.169 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:21.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.335 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:09:21.363 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-11-24T06:09:21.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.365 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-11-24T06:09:21.440 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-11-24T06:09:21.441 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:09:21.441 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:09:21.441 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:09:21.441 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:09:21.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.441 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:21.441 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:21.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:09:21.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:09:21.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:09:21.442 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:21.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.682 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:09:21.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.699 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list-id 2023-11-24T06:09:21.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.756 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:21.756 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:21.756 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:09:21.756 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T06:09:21.756 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:21.850 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T06:09:21.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:21.850 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:21.850 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:21.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T06:09:21.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:09:21.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T06:09:21.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T06:09:21.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T06:09:21.851 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:22.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:22.058 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:22.058 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:22.058 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2023-11-24T06:09:22.058 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:09:22.058 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:22.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:09:22.171 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:09:22.188 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:09:22.188 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:09:22.188 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:09:22.246 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:09:22.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:22.246 DEBUG:teuthology.orchestra.run.smithi183:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:22.353 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:22.354 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:22.458 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:22.458 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:22.458 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:09:22.458 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:09:22.458 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:09:22.458 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:09:22.458 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:09:22.459 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:09:22.491 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:09:22.492 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:09:22.495 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:09:22.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:22.495 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:09:22.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:22.523 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:22.595 INFO:teuthology.orchestra.run.smithi183.stdout:client4624 2023-11-24T06:09:22.595 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:09:22.595 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:22.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:22.653 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:22.725 INFO:teuthology.orchestra.run.smithi183.stdout:client4624 2023-11-24T06:09:22.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:22.725 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:22.797 INFO:teuthology.orchestra.run.smithi183.stdout:client4624 2023-11-24T06:09:22.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:22.797 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.client4624/status') 2023-11-24T06:09:22.876 INFO:teuthology.orchestra.run.smithi183.stdout:instance: client.4624 (3)192.168.0.1:0/3132805943 2023-11-24T06:09:22.877 INFO:teuthology.orchestra.run.smithi183.stdout:blocklisted: false 2023-11-24T06:09:22.877 INFO:teuthology.orchestra.run.smithi183.stderr:0+1 records in 2023-11-24T06:09:22.877 INFO:teuthology.orchestra.run.smithi183.stderr:0+1 records out 2023-11-24T06:09:22.877 INFO:teuthology.orchestra.run.smithi183.stderr:69 bytes copied, 2.2487e-05 s, 3.1 MB/s 2023-11-24T06:09:22.878 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-24T06:09:22.878 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:09:22.878 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:09:22.878 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:09:23.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.008+0000 7fe609abc700 1 Processor -- start 2023-11-24T06:09:23.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.008+0000 7fe609abc700 1 -- start start 2023-11-24T06:09:23.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe609abc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6040b0f80 0x7fe60415a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe609abc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040cd770 0x7fe60415a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe609abc700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6040b2bd0 con 0x7fe6040b0ba0 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe609abc700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6040b1fc0 con 0x7fe6040b0f80 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe609abc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6040b22a0 con 0x7fe6040cd770 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe602ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6040b0f80 0x7fe60415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe602ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6040b0f80 0x7fe60415a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45842/0 (socket says 172.21.15.97:45842) 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe6037fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe6040b0ba0 0x7fe6040af5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38292/0 (socket says 172.21.15.97:38292) 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.009+0000 7fe602ffd700 1 -- 172.21.15.97:0/1241798521 learned_addr learned my addr 172.21.15.97:0/1241798521 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe602ffd700 1 -- 172.21.15.97:0/1241798521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040cd770 msgr2=0x7fe60415a980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:09:23.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe602ffd700 1 --2- 172.21.15.97:0/1241798521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040cd770 0x7fe60415a980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe602ffd700 1 -- 172.21.15.97:0/1241798521 >> v1:172.21.15.97:6789/0 conn(0x7fe6040b0ba0 legacy=0x7fe6040af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe602ffd700 1 -- 172.21.15.97:0/1241798521 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040b2580 con 0x7fe6040b0f80 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe6027fc700 1 -- 172.21.15.97:0/1241798521 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461241918 0 0) 0x7fe6040b2bd0 con 0x7fe6040b0ba0 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe602ffd700 1 --2- 172.21.15.97:0/1241798521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0f80 0x7fe60415a440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40098f0 tx=0x7fe5f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e989f8bc1c2ddc08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe6027fc700 1 -- 172.21.15.97:0/1241798521 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400f940 con 0x7fe6040b0f80 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe6027fc700 1 -- 172.21.15.97:0/1241798521 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f400faa0 con 0x7fe6040b0f80 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe6027fc700 1 -- 172.21.15.97:0/1241798521 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400fe30 con 0x7fe6040b0f80 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe609abc700 1 -- 172.21.15.97:0/1241798521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0f80 msgr2=0x7fe60415a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.010+0000 7fe609abc700 1 --2- 172.21.15.97:0/1241798521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0f80 0x7fe60415a440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40098f0 tx=0x7fe5f4009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.011+0000 7fe609abc700 1 -- 172.21.15.97:0/1241798521 shutdown_connections 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.011+0000 7fe609abc700 1 --2- 172.21.15.97:0/1241798521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040cd770 0x7fe60415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.011+0000 7fe609abc700 1 --2- 172.21.15.97:0/1241798521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0f80 0x7fe60415a440 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.011+0000 7fe609abc700 1 -- 172.21.15.97:0/1241798521 >> 172.21.15.97:0/1241798521 conn(0x7fe6040b57b0 msgr2=0x7fe6040b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.011+0000 7fe609abc700 1 -- 172.21.15.97:0/1241798521 shutdown_connections 2023-11-24T06:09:23.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.011+0000 7fe609abc700 1 -- 172.21.15.97:0/1241798521 wait complete. 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.012+0000 7fe609abc700 1 Processor -- start 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.012+0000 7fe609abc700 1 -- start start 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe609abc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0ba0 0x7fe604158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe609abc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040b0f80 0x7fe6040ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe609abc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6040cd770 0x7fe6040ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe609abc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041585e0 con 0x7fe6040b0ba0 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe609abc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60415a900 con 0x7fe6040cd770 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe609abc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041f10b0 con 0x7fe6040b0f80 2023-11-24T06:09:23.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0ba0 0x7fe604158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0ba0 0x7fe604158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45846/0 (socket says 172.21.15.97:45846) 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 -- 172.21.15.97:0/2207374407 learned_addr learned my addr 172.21.15.97:0/2207374407 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe602ffd700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040b0f80 0x7fe6040ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe603fff700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6040cd770 0x7fe6040ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 -- 172.21.15.97:0/2207374407 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040b0f80 msgr2=0x7fe6040ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040b0f80 0x7fe6040ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 -- 172.21.15.97:0/2207374407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6040cd770 msgr2=0x7fe6040ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6040cd770 0x7fe6040ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 -- 172.21.15.97:0/2207374407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f4009550 con 0x7fe6040b0ba0 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe6037fe700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0ba0 0x7fe604158bb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0011e0 tx=0x7fe5ec00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:23.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.013+0000 7fe5fbfff700 1 -- 172.21.15.97:0/2207374407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec004c80 con 0x7fe6040b0ba0 2023-11-24T06:09:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.014+0000 7fe5fbfff700 1 -- 172.21.15.97:0/2207374407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec004e00 con 0x7fe6040b0ba0 2023-11-24T06:09:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.014+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6041f0d80 con 0x7fe6040b0ba0 2023-11-24T06:09:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.014+0000 7fe5fbfff700 1 -- 172.21.15.97:0/2207374407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec007310 con 0x7fe6040b0ba0 2023-11-24T06:09:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.014+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60415a2b0 con 0x7fe6040b0ba0 2023-11-24T06:09:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.014+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6040b1d70 con 0x7fe6040b0ba0 2023-11-24T06:09:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.015+0000 7fe5fbfff700 1 -- 172.21.15.97:0/2207374407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe5ec007470 con 0x7fe6040b0ba0 2023-11-24T06:09:23.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.015+0000 7fe5fbfff700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe5e406aed0 0x7fe5e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:23.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.015+0000 7fe5fbfff700 1 -- 172.21.15.97:0/2207374407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5ec0893b0 con 0x7fe6040b0ba0 2023-11-24T06:09:23.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.015+0000 7fe602ffd700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe5e406aed0 0x7fe5e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:23.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.016+0000 7fe602ffd700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe5e406aed0 0x7fe5e406d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4019040 tx=0x7fe5f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:23.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.020+0000 7fe5fbfff700 1 -- 172.21.15.97:0/2207374407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5ec054330 con 0x7fe6040b0ba0 2023-11-24T06:09:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.255+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe604067f60 con 0x7fe6040b0ba0 2023-11-24T06:09:23.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.255+0000 7fe5fbfff700 1 -- 172.21.15.97:0/2207374407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe5ec057950 con 0x7fe6040b0ba0 2023-11-24T06:09:23.258 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:09:23.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe5e406aed0 msgr2=0x7fe5e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:23.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe5e406aed0 0x7fe5e406d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4019040 tx=0x7fe5f4008030 comp rx=0 tx=0).stop 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0ba0 msgr2=0x7fe604158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0ba0 0x7fe604158bb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0011e0 tx=0x7fe5ec00ca30 comp rx=0 tx=0).stop 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 shutdown_connections 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6040cd770 0x7fe6040ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6040b0ba0 0x7fe604158bb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe5e406aed0 0x7fe5e406d390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 --2- 172.21.15.97:0/2207374407 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe6040b0f80 0x7fe6040ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 >> 172.21.15.97:0/2207374407 conn(0x7fe6040b57b0 msgr2=0x7fe60406e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.257+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 shutdown_connections 2023-11-24T06:09:23.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:23.258+0000 7fe609abc700 1 -- 172.21.15.97:0/2207374407 wait complete. 2023-11-24T06:09:23.272 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:09:23.272 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:09:23.272 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-11-24T06:09:23.272 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T06:09:23.272 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:09:23.273 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:09:23.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:09:23.273 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:09:23.273 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:09:23.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.273 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-11-24T06:09:23.290 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:09:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:09:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:09:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:09:23.290 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42165sec preferred_lft 42165sec 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::5382:2bb9:1085:5950/64 scope link noprefixroute 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.291 INFO:teuthology.orchestra.run.smithi183.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether fe:4f:a7:e6:1c:7f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::b8cd:8aff:fe67:b24d/64 scope link tentative 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether fe:4f:a7:e6:1c:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::fc4f:a7ff:fee6:1c7f/64 scope link tentative 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.293 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:09:23.355 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-11-24T06:09:23.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.357 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-11-24T06:09:23.416 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-11-24T06:09:23.416 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:09:23.416 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:09:23.416 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:09:23.416 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:09:23.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.417 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:23.417 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:23.417 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:09:23.417 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:09:23.417 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:09:23.417 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:23.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.521 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:09:23.538 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:09:23.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.538 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list-id 2023-11-24T06:09:23.594 INFO:teuthology.orchestra.run.smithi183.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T06:09:23.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.595 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:23.595 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:23.595 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:09:23.595 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T06:09:23.595 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:23.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.688 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:09:23.730 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:09:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:09:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:09:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:09:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.730 INFO:teuthology.orchestra.run.smithi183.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:09:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 76:03:af:ff:86:7d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:09:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:09:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::7403:afff:feff:867d/64 scope link tentative 2023-11-24T06:09:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.743 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:09:23.819 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:09:23.819 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:09:23.819 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:09:23.819 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.819 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:09:23.819 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.820 INFO:teuthology.orchestra.run.smithi183.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:09:23.820 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 76:03:af:ff:86:7d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:09:23.820 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:09:23.820 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.820 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::7403:afff:feff:867d/64 scope link tentative 2023-11-24T06:09:23.820 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:09:23.827 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T06:09:23.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:23.828 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:23.828 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:23.828 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T06:09:23.828 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:09:23.828 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T06:09:23.828 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T06:09:23.828 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T06:09:23.828 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:24.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:24.019 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:24.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:24.071 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:09:24.071 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:09:24.071 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.1 up 2023-11-24T06:09:24.072 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:09:24.072 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:09:24.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:09:24.181 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T06:09:24.201 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:09:24.201 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:09:24.201 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:09:24.257 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-24T06:09:24.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:24.257 DEBUG:teuthology.orchestra.run.smithi183:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:24.342 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:24.342 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:24.365 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:24.366 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:24.366 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:09:24.366 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 3@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:09:24.366 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:09:24.366 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:09:24.366 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:09:24.366 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:09:24.398 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:09:24.398 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:09:24.398 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:09:24.398 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 3@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:09:24.398 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:09:24.398 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:09:24.399 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:09:24.399 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:09:24.399 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:09:24.399 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:09:24.399 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:09:24.401 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:09:24.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:24.401 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:09:24.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:24.430 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:24.503 INFO:teuthology.orchestra.run.smithi183.stdout:client4490 2023-11-24T06:09:24.503 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:09:24.503 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:24.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:24.560 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:24.632 INFO:teuthology.orchestra.run.smithi183.stdout:client4490 2023-11-24T06:09:24.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:24.632 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:09:24.704 INFO:teuthology.orchestra.run.smithi183.stdout:client4490 2023-11-24T06:09:24.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:24.704 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.client4490/status') 2023-11-24T06:09:24.784 INFO:teuthology.orchestra.run.smithi183.stdout:instance: client.4490 (3)192.168.0.2:0/3409319829 2023-11-24T06:09:24.784 INFO:teuthology.orchestra.run.smithi183.stdout:blocklisted: false 2023-11-24T06:09:24.784 INFO:teuthology.orchestra.run.smithi183.stderr:0+1 records in 2023-11-24T06:09:24.784 INFO:teuthology.orchestra.run.smithi183.stderr:0+1 records out 2023-11-24T06:09:24.785 INFO:teuthology.orchestra.run.smithi183.stderr:69 bytes copied, 2.1699e-05 s, 3.2 MB/s 2023-11-24T06:09:24.785 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T06:09:24.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:24.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.929+0000 7f7350fa3700 1 Processor -- start 2023-11-24T06:09:24.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.930+0000 7f7350fa3700 1 -- start start 2023-11-24T06:09:24.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.930+0000 7f7350fa3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c1446a0 0x7f734c0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:24.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.930+0000 7f7350fa3700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f734c0adf00 0x7f734c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:24.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7350fa3700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c0af950 con 0x7f734c06c540 2023-11-24T06:09:24.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7350fa3700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c144a80 con 0x7f734c0adf00 2023-11-24T06:09:24.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7350fa3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c06c9b0 con 0x7f734c1446a0 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7349d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f734c0adf00 0x7f734c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7349d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f734c0adf00 0x7f734c06c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45856/0 (socket says 172.21.15.97:45856) 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7349d9b700 1 -- 172.21.15.97:0/855832925 learned_addr learned my addr 172.21.15.97:0/855832925 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f734a59c700 1 --2- 172.21.15.97:0/855832925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c1446a0 0x7f734c0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7349d9b700 1 -- 172.21.15.97:0/855832925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c1446a0 msgr2=0x7f734c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7349d9b700 1 --2- 172.21.15.97:0/855832925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c1446a0 0x7f734c0b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7349d9b700 1 -- 172.21.15.97:0/855832925 >> v1:172.21.15.183:6789/0 conn(0x7f734c06c540 legacy=0x7f734c06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7349d9b700 1 -- 172.21.15.97:0/855832925 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c0afc30 con 0x7f734c0adf00 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f734a59c700 1 --2- 172.21.15.97:0/855832925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c1446a0 0x7f734c0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f7349d9b700 1 --2- 172.21.15.97:0/855832925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c0adf00 0x7f734c06c000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f733c009920 tx=0x7f733c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc54bda70d8eb9c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f734959a700 1 -- 172.21.15.97:0/855832925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00ba20 con 0x7f734c0adf00 2023-11-24T06:09:24.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f734959a700 1 -- 172.21.15.97:0/855832925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c00bb80 con 0x7f734c0adf00 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.931+0000 7f734959a700 1 -- 172.21.15.97:0/855832925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c009e70 con 0x7f734c0adf00 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.932+0000 7f7350fa3700 1 -- 172.21.15.97:0/855832925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c0adf00 msgr2=0x7f734c06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.932+0000 7f7350fa3700 1 --2- 172.21.15.97:0/855832925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c0adf00 0x7f734c06c000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f733c009920 tx=0x7f733c009c30 comp rx=0 tx=0).stop 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.932+0000 7f7350fa3700 1 -- 172.21.15.97:0/855832925 shutdown_connections 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.932+0000 7f7350fa3700 1 --2- 172.21.15.97:0/855832925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c1446a0 0x7f734c0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.932+0000 7f7350fa3700 1 --2- 172.21.15.97:0/855832925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c0adf00 0x7f734c06c000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.932+0000 7f7350fa3700 1 -- 172.21.15.97:0/855832925 >> 172.21.15.97:0/855832925 conn(0x7f734c071b90 msgr2=0x7f734c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.932+0000 7f7350fa3700 1 -- 172.21.15.97:0/855832925 shutdown_connections 2023-11-24T06:09:24.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.932+0000 7f7350fa3700 1 -- 172.21.15.97:0/855832925 wait complete. 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.933+0000 7f7350fa3700 1 Processor -- start 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.933+0000 7f7350fa3700 1 -- start start 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f7350fa3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c06c540 0x7f734c1f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f7350fa3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f734c0adf00 0x7f734c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f7350fa3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c1446a0 0x7f734c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f7350fa3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0d03d0 con 0x7f734c1446a0 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f7350fa3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0cf330 con 0x7f734c0adf00 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f7350fa3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0cf610 con 0x7f734c06c540 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f734ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c1446a0 0x7f734c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:24.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f734ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c1446a0 0x7f734c0744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45864/0 (socket says 172.21.15.97:45864) 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f734ad9d700 1 -- 172.21.15.97:0/3282497613 learned_addr learned my addr 172.21.15.97:0/3282497613 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f734a59c700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c06c540 0x7f734c1f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.934+0000 7f7349d9b700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f734c0adf00 0x7f734c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f734ad9d700 1 -- 172.21.15.97:0/3282497613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c06c540 msgr2=0x7f734c1f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f734ad9d700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c06c540 0x7f734c1f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f734ad9d700 1 -- 172.21.15.97:0/3282497613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f734c0adf00 msgr2=0x7f734c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f734ad9d700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f734c0adf00 0x7f734c073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f734ad9d700 1 -- 172.21.15.97:0/3282497613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c009550 con 0x7f734c1446a0 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f734ad9d700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c1446a0 0x7f734c0744d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f734400cc50 tx=0x7f734400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f733affd700 1 -- 172.21.15.97:0/3282497613 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734400ad80 con 0x7f734c1446a0 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f733affd700 1 -- 172.21.15.97:0/3282497613 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734400ce50 con 0x7f734c1446a0 2023-11-24T06:09:24.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c0cf8f0 con 0x7f734c1446a0 2023-11-24T06:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f733affd700 1 -- 172.21.15.97:0/3282497613 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344003c40 con 0x7f734c1446a0 2023-11-24T06:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.935+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c0cfeb0 con 0x7f734c1446a0 2023-11-24T06:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.936+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c0753f0 con 0x7f734c1446a0 2023-11-24T06:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.936+0000 7f733affd700 1 -- 172.21.15.97:0/3282497613 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73440050d0 con 0x7f734c1446a0 2023-11-24T06:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.937+0000 7f733affd700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f732c06aed0 0x7f732c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.937+0000 7f733affd700 1 -- 172.21.15.97:0/3282497613 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73440896f0 con 0x7f734c1446a0 2023-11-24T06:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.937+0000 7f734a59c700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f732c06aed0 0x7f732c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:24.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.937+0000 7f734a59c700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f732c06aed0 0x7f732c06d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f734000c900 tx=0x7f7340008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:24.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:24.941+0000 7f733affd700 1 -- 172.21.15.97:0/3282497613 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f73440547a0 con 0x7f734c1446a0 2023-11-24T06:09:25.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.130+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f734c0d0040 con 0x7f734c1446a0 2023-11-24T06:09:25.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.131+0000 7f733affd700 1 -- 172.21.15.97:0/3282497613 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f7344057dc0 con 0x7f734c1446a0 2023-11-24T06:09:25.134 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:25.135 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:25.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f732c06aed0 msgr2=0x7f732c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f732c06aed0 0x7f732c06d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f734000c900 tx=0x7f7340008040 comp rx=0 tx=0).stop 2023-11-24T06:09:25.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c1446a0 msgr2=0x7f734c0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c1446a0 0x7f734c0744d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f734400cc50 tx=0x7f734400a340 comp rx=0 tx=0).stop 2023-11-24T06:09:25.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 shutdown_connections 2023-11-24T06:09:25.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f734c1446a0 0x7f734c0744d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f732c06aed0 0x7f732c06d390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f734c06c540 0x7f734c1f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 --2- 172.21.15.97:0/3282497613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f734c0adf00 0x7f734c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 >> 172.21.15.97:0/3282497613 conn(0x7f734c071b90 msgr2=0x7f734c0b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:25.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 shutdown_connections 2023-11-24T06:09:25.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.133+0000 7f7350fa3700 1 -- 172.21.15.97:0/3282497613 wait complete. 2023-11-24T06:09:25.148 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:25.149 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.269+0000 7ffa3a4f6700 1 Processor -- start 2023-11-24T06:09:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.269+0000 7ffa3a4f6700 1 -- start start 2023-11-24T06:09:25.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa3a4f6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa340cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa3a4f6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa340add40 0x7ffa34073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa3a4f6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa340b5ee0 con 0x7ffa340744d0 2023-11-24T06:09:25.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa3a4f6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3406bfc0 con 0x7ffa340add40 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa3a4f6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa34074960 con 0x7ffa3415be70 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa337fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa340add40 0x7ffa34073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa337fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa340add40 0x7ffa34073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45870/0 (socket says 172.21.15.97:45870) 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa337fe700 1 -- 172.21.15.97:0/3483764979 learned_addr learned my addr 172.21.15.97:0/3483764979 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa33fff700 1 --2- 172.21.15.97:0/3483764979 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa340cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa337fe700 1 -- 172.21.15.97:0/3483764979 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 msgr2=0x7ffa340cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa337fe700 1 --2- 172.21.15.97:0/3483764979 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa340cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa337fe700 1 -- 172.21.15.97:0/3483764979 >> v1:172.21.15.183:6789/0 conn(0x7ffa340744d0 legacy=0x7ffa340af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa32ffd700 1 -- 172.21.15.97:0/3483764979 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487233648 0 0) 0x7ffa340b5ee0 con 0x7ffa340744d0 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa337fe700 1 -- 172.21.15.97:0/3483764979 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3406ba00 con 0x7ffa340add40 2023-11-24T06:09:25.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa337fe700 1 --2- 172.21.15.97:0/3483764979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340add40 0x7ffa34073f90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa280098f0 tx=0x7ffa28009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1a0970c65f6472 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa32ffd700 1 -- 172.21.15.97:0/3483764979 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28011830 con 0x7ffa340add40 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa32ffd700 1 -- 172.21.15.97:0/3483764979 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa28011990 con 0x7ffa340add40 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.270+0000 7ffa32ffd700 1 -- 172.21.15.97:0/3483764979 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28011d20 con 0x7ffa340add40 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.271+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3483764979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340add40 msgr2=0x7ffa34073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.271+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3483764979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340add40 0x7ffa34073f90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa280098f0 tx=0x7ffa28009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.271+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3483764979 shutdown_connections 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.271+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3483764979 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa340cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.271+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3483764979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340add40 0x7ffa34073f90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.271+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3483764979 >> 172.21.15.97:0/3483764979 conn(0x7ffa340d0bf0 msgr2=0x7ffa340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:25.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.271+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3483764979 shutdown_connections 2023-11-24T06:09:25.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.271+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3483764979 wait complete. 2023-11-24T06:09:25.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.272+0000 7ffa3a4f6700 1 Processor -- start 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.272+0000 7ffa3a4f6700 1 -- start start 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa3a4f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340744d0 0x7ffa341f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa3a4f6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa340add40 0x7ffa3409cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa3a4f6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa3409b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa3a4f6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3415c3d0 con 0x7ffa340744d0 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa3a4f6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa341f1ae0 con 0x7ffa340add40 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa3a4f6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3409bd80 con 0x7ffa3415be70 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa33fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340744d0 0x7ffa341f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa33fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340744d0 0x7ffa341f15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45874/0 (socket says 172.21.15.97:45874) 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa33fff700 1 -- 172.21.15.97:0/3853990360 learned_addr learned my addr 172.21.15.97:0/3853990360 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:25.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa337fe700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa340add40 0x7ffa3409cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa38a93700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa3409b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa33fff700 1 -- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 msgr2=0x7ffa3409b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa33fff700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa3409b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa33fff700 1 -- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa340add40 msgr2=0x7ffa3409cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa33fff700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa340add40 0x7ffa3409cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa33fff700 1 -- 172.21.15.97:0/3853990360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28009550 con 0x7ffa340744d0 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa337fe700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa340add40 0x7ffa3409cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.273+0000 7ffa38a93700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa3409b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.274+0000 7ffa33fff700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340744d0 0x7ffa341f15a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400e870 tx=0x7ffa2400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.274+0000 7ffa30ff9700 1 -- 172.21.15.97:0/3853990360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24009590 con 0x7ffa340744d0 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.274+0000 7ffa30ff9700 1 -- 172.21.15.97:0/3853990360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2400ee70 con 0x7ffa340744d0 2023-11-24T06:09:25.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.274+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3415c5e0 con 0x7ffa340744d0 2023-11-24T06:09:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.274+0000 7ffa30ff9700 1 -- 172.21.15.97:0/3853990360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400ca50 con 0x7ffa340744d0 2023-11-24T06:09:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.274+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3415cba0 con 0x7ffa340744d0 2023-11-24T06:09:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.275+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3415a470 con 0x7ffa340744d0 2023-11-24T06:09:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.275+0000 7ffa30ff9700 1 -- 172.21.15.97:0/3853990360 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa2400cbb0 con 0x7ffa340744d0 2023-11-24T06:09:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.276+0000 7ffa30ff9700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffa2006aed0 0x7ffa2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.276+0000 7ffa30ff9700 1 -- 172.21.15.97:0/3853990360 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa24089550 con 0x7ffa340744d0 2023-11-24T06:09:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.276+0000 7ffa337fe700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffa2006aed0 0x7ffa2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.276+0000 7ffa337fe700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffa2006aed0 0x7ffa2006d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffa28000c00 tx=0x7ffa28008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.280+0000 7ffa30ff9700 1 -- 172.21.15.97:0/3853990360 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa24054600 con 0x7ffa340744d0 2023-11-24T06:09:25.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.468+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa34068fb0 con 0x7ffa340744d0 2023-11-24T06:09:25.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.468+0000 7ffa30ff9700 1 -- 172.21.15.97:0/3853990360 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa2400ce60 con 0x7ffa340744d0 2023-11-24T06:09:25.471 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffa2006aed0 msgr2=0x7ffa2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffa2006aed0 0x7ffa2006d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffa28000c00 tx=0x7ffa28008030 comp rx=0 tx=0).stop 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340744d0 msgr2=0x7ffa341f15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340744d0 0x7ffa341f15a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400e870 tx=0x7ffa2400ec30 comp rx=0 tx=0).stop 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 shutdown_connections 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffa2006aed0 0x7ffa2006d390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffa3415be70 0x7ffa3409b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa340744d0 0x7ffa341f15a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 --2- 172.21.15.97:0/3853990360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa340add40 0x7ffa3409cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.470+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 >> 172.21.15.97:0/3853990360 conn(0x7ffa340d0bf0 msgr2=0x7ffa3415a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:25.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.471+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 shutdown_connections 2023-11-24T06:09:25.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.471+0000 7ffa3a4f6700 1 -- 172.21.15.97:0/3853990360 wait complete. 2023-11-24T06:09:25.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.606+0000 7f9bb2a18700 1 Processor -- start 2023-11-24T06:09:25.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.606+0000 7f9bb2a18700 1 -- start start 2023-11-24T06:09:25.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.606+0000 7f9bb2a18700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9bac1587f0 0x7f9bac06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.606+0000 7f9bb2a18700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bac15b370 0x7f9bac15b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.606+0000 7f9bb2a18700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac06b530 con 0x7f9bac0b3c60 2023-11-24T06:09:25.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.606+0000 7f9bb2a18700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bac0ae5e0 con 0x7f9bac15b370 2023-11-24T06:09:25.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.606+0000 7f9bb2a18700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bac15be30 con 0x7f9bac1587f0 2023-11-24T06:09:25.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9bab7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bac15b370 0x7f9bac15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9bb0fb5700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9bac0b3c60 0x7f9bac0af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38338/0 (socket says 172.21.15.97:38338) 2023-11-24T06:09:25.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9bb0fb5700 1 -- 172.21.15.97:0/3026720437 learned_addr learned my addr 172.21.15.97:0/3026720437 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9babfff700 1 --2- 172.21.15.97:0/3026720437 >> v2:172.21.15.183:3300/0 conn(0x7f9bac1587f0 0x7f9bac06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9bab7fe700 1 -- 172.21.15.97:0/3026720437 >> v2:172.21.15.183:3300/0 conn(0x7f9bac1587f0 msgr2=0x7f9bac06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9bab7fe700 1 --2- 172.21.15.97:0/3026720437 >> v2:172.21.15.183:3300/0 conn(0x7f9bac1587f0 0x7f9bac06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9bab7fe700 1 -- 172.21.15.97:0/3026720437 >> v1:172.21.15.97:6789/0 conn(0x7f9bac0b3c60 legacy=0x7f9bac0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9bab7fe700 1 -- 172.21.15.97:0/3026720437 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac15c110 con 0x7f9bac15b370 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9baaffd700 1 -- 172.21.15.97:0/3026720437 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405495180 0 0) 0x7f9bac06b530 con 0x7f9bac0b3c60 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9bab7fe700 1 --2- 172.21.15.97:0/3026720437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac15b370 0x7f9bac15b800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009840 tx=0x7f9b9c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f95e80c5715e85a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9baaffd700 1 -- 172.21.15.97:0/3026720437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c011830 con 0x7f9bac15b370 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9baaffd700 1 -- 172.21.15.97:0/3026720437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c011990 con 0x7f9bac15b370 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.607+0000 7f9baaffd700 1 -- 172.21.15.97:0/3026720437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c011d20 con 0x7f9bac15b370 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.608+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3026720437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac15b370 msgr2=0x7f9bac15b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.608+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3026720437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac15b370 0x7f9bac15b800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009840 tx=0x7f9b9c009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:25.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.608+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3026720437 shutdown_connections 2023-11-24T06:09:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.608+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3026720437 >> v2:172.21.15.183:3300/0 conn(0x7f9bac1587f0 0x7f9bac06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.608+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3026720437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac15b370 0x7f9bac15b800 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.608+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3026720437 >> 172.21.15.97:0/3026720437 conn(0x7f9bac0d0bf0 msgr2=0x7f9bac0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.608+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3026720437 shutdown_connections 2023-11-24T06:09:25.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.608+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3026720437 wait complete. 2023-11-24T06:09:25.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.609+0000 7f9bb2a18700 1 Processor -- start 2023-11-24T06:09:25.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.609+0000 7f9bb2a18700 1 -- start start 2023-11-24T06:09:25.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bb2a18700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bac0b3c60 0x7f9bac1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bb2a18700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac1587f0 0x7f9bac159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bb2a18700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bac15b370 0x7f9bac15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bb2a18700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0b0b00 con 0x7f9bac1587f0 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bb2a18700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0b0de0 con 0x7f9bac15b370 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bb2a18700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0b10c0 con 0x7f9bac0b3c60 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac1587f0 0x7f9bac159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac1587f0 0x7f9bac159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45890/0 (socket says 172.21.15.97:45890) 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 -- 172.21.15.97:0/3087273060 learned_addr learned my addr 172.21.15.97:0/3087273060 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bb0fb5700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bac15b370 0x7f9bac15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9babfff700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bac0b3c60 0x7f9bac1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 -- 172.21.15.97:0/3087273060 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bac0b3c60 msgr2=0x7f9bac1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bac0b3c60 0x7f9bac1516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 -- 172.21.15.97:0/3087273060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bac15b370 msgr2=0x7f9bac15a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bac15b370 0x7f9bac15a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 -- 172.21.15.97:0/3087273060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0094a0 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.610+0000 7f9bab7fe700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac1587f0 0x7f9bac159c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0090b0 tx=0x7f9b9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.611+0000 7f9ba8ff9700 1 -- 172.21.15.97:0/3087273060 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c01a830 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.611+0000 7f9ba8ff9700 1 -- 172.21.15.97:0/3087273060 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c01a990 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.611+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac0b13a0 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.611+0000 7f9ba8ff9700 1 -- 172.21.15.97:0/3087273060 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c01ad20 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.611+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac151c30 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.612+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac068fb0 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.612+0000 7f9ba8ff9700 1 -- 172.21.15.97:0/3087273060 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b9c02d450 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.612+0000 7f9ba8ff9700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b8c06aed0 0x7f9b8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.612+0000 7f9ba8ff9700 1 -- 172.21.15.97:0/3087273060 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b9c08a960 con 0x7f9bac1587f0 2023-11-24T06:09:25.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.613+0000 7f9babfff700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b8c06aed0 0x7f9b8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.613+0000 7f9babfff700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b8c06aed0 0x7f9b8c06d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b94006b20 tx=0x7f9b94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.617+0000 7f9ba8ff9700 1 -- 172.21.15.97:0/3087273060 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b9c055aa0 con 0x7f9bac1587f0 2023-11-24T06:09:25.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.808+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9bac06c0f0 con 0x7f9bac1587f0 2023-11-24T06:09:25.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.808+0000 7f9ba8ff9700 1 -- 172.21.15.97:0/3087273060 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b9c0150a0 con 0x7f9bac1587f0 2023-11-24T06:09:25.811 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b8c06aed0 msgr2=0x7f9b8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b8c06aed0 0x7f9b8c06d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b94006b20 tx=0x7f9b94008040 comp rx=0 tx=0).stop 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac1587f0 msgr2=0x7f9bac159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac1587f0 0x7f9bac159c20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0090b0 tx=0x7f9b9c004750 comp rx=0 tx=0).stop 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 shutdown_connections 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bac15b370 0x7f9bac15a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b8c06aed0 0x7f9b8c06d390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bac0b3c60 0x7f9bac1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 --2- 172.21.15.97:0/3087273060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bac1587f0 0x7f9bac159c20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 >> 172.21.15.97:0/3087273060 conn(0x7f9bac0d0bf0 msgr2=0x7f9bac1568a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 shutdown_connections 2023-11-24T06:09:25.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.811+0000 7f9bb2a18700 1 -- 172.21.15.97:0/3087273060 wait complete. 2023-11-24T06:09:25.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:25.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.945+0000 7fcb32369700 1 Processor -- start 2023-11-24T06:09:25.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.945+0000 7fcb32369700 1 -- start start 2023-11-24T06:09:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.946+0000 7fcb32369700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15a620 0x7fcb2c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.946+0000 7fcb32369700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c0b2f20 con 0x7fcb2c06ecf0 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.946+0000 7fcb32369700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c0ce690 con 0x7fcb2c0cc540 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.946+0000 7fcb32369700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c15aa00 con 0x7fcb2c15a620 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.946+0000 7fcb30906700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcb2c06ecf0 0x7fcb2c0b3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38350/0 (socket says 172.21.15.97:38350) 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.946+0000 7fcb30906700 1 -- 172.21.15.97:0/585685207 learned_addr learned my addr 172.21.15.97:0/585685207 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344475310 0 0) 0x7fcb2c0b2f20 con 0x7fcb2c06ecf0 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2bfff700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15a620 0x7fcb2c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb140046c0 con 0x7fcb2c06ecf0 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2903425177 0 0) 0x7fcb140046c0 con 0x7fcb2c06ecf0 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15a620 msgr2=0x7fcb2c0ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15a620 0x7fcb2c0ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 >> v1:172.21.15.183:6789/0 conn(0x7fcb2c0cc540 legacy=0x7fcb2c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c0ce960 con 0x7fcb2c06ecf0 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fcb240019f0 con 0x7fcb2c06ecf0 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 >> v1:172.21.15.97:6789/0 conn(0x7fcb2c06ecf0 legacy=0x7fcb2c0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb14004b40 0x7fcb14004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15a620 0x7fcb14005960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb14005ea0 0x7fcb14006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c15aa00 con 0x7fcb14005ea0 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb140035b0 con 0x7fcb14004b40 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb14003770 con 0x7fcb2c15a620 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb30906700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb14005ea0 0x7fcb14006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2bfff700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb14004b40 0x7fcb14004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb2b7fe700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15a620 0x7fcb14005960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb30906700 1 -- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15a620 msgr2=0x7fcb14005960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:25.953 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb30906700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15a620 0x7fcb14005960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb30906700 1 -- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb14004b40 msgr2=0x7fcb14004f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb30906700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb14004b40 0x7fcb14004f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.947+0000 7fcb30906700 1 -- 172.21.15.97:0/585685207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb140046c0 con 0x7fcb14005ea0 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb30906700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb14005ea0 0x7fcb14006320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0b5720 tx=0x7fcb14006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba98900181cd3186 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400a2f0 con 0x7fcb14005ea0 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24012620 con 0x7fcb14005ea0 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb2affd700 1 -- 172.21.15.97:0/585685207 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240129d0 con 0x7fcb14005ea0 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb32369700 1 -- 172.21.15.97:0/585685207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb14005ea0 msgr2=0x7fcb14006320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb32369700 1 --2- 172.21.15.97:0/585685207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb14005ea0 0x7fcb14006320 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0b5720 tx=0x7fcb14006990 comp rx=0 tx=0).stop 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb30906700 1 -- 172.21.15.97:0/585685207 reap_dead start 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb32369700 1 -- 172.21.15.97:0/585685207 shutdown_connections 2023-11-24T06:09:25.954 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb32369700 1 -- 172.21.15.97:0/585685207 >> 172.21.15.97:0/585685207 conn(0x7fcb2c0b2050 msgr2=0x7fcb2c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:25.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb32369700 1 -- 172.21.15.97:0/585685207 shutdown_connections 2023-11-24T06:09:25.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.948+0000 7fcb32369700 1 -- 172.21.15.97:0/585685207 wait complete. 2023-11-24T06:09:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.949+0000 7fcb32369700 1 Processor -- start 2023-11-24T06:09:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb32369700 1 -- start start 2023-11-24T06:09:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb32369700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c14cce0 0x7fcb2c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb32369700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15abd0 0x7fcb2c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb32369700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb2c14ef50 0x7fcb2c14f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb32369700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c14f940 con 0x7fcb2c14cce0 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb32369700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c14d0c0 con 0x7fcb2c14ef50 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb32369700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c15b140 con 0x7fcb2c15abd0 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c14cce0 0x7fcb2c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c14cce0 0x7fcb2c15c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45906/0 (socket says 172.21.15.97:45906) 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.950+0000 7fcb2bfff700 1 -- 172.21.15.97:0/3567034772 learned_addr learned my addr 172.21.15.97:0/3567034772 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb2b7fe700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15abd0 0x7fcb2c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb30906700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb2c14ef50 0x7fcb2c14f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb2bfff700 1 -- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15abd0 msgr2=0x7fcb2c14e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb2bfff700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15abd0 0x7fcb2c14e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb2bfff700 1 -- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb2c14ef50 msgr2=0x7fcb2c14f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:25.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb2bfff700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb2c14ef50 0x7fcb2c14f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb2bfff700 1 -- 172.21.15.97:0/3567034772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c15b420 con 0x7fcb2c14cce0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb30906700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb2c14ef50 0x7fcb2c14f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb2b7fe700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15abd0 0x7fcb2c14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb2bfff700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c14cce0 0x7fcb2c15c1e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb20011ad0 tx=0x7fcb20011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb28ff9700 1 -- 172.21.15.97:0/3567034772 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2000fa00 con 0x7fcb2c14cce0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb28ff9700 1 -- 172.21.15.97:0/3567034772 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2000fb60 con 0x7fcb2c14cce0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c15b700 con 0x7fcb2c14cce0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb28ff9700 1 -- 172.21.15.97:0/3567034772 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20014440 con 0x7fcb2c14cce0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.951+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c15bcc0 con 0x7fcb2c14cce0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.952+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c068d90 con 0x7fcb2c14cce0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.952+0000 7fcb28ff9700 1 -- 172.21.15.97:0/3567034772 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb200145a0 con 0x7fcb2c14cce0 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.953+0000 7fcb28ff9700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcb1806aed0 0x7fcb1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:25.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.953+0000 7fcb28ff9700 1 -- 172.21.15.97:0/3567034772 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb20089d80 con 0x7fcb2c14cce0 2023-11-24T06:09:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.953+0000 7fcb2b7fe700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcb1806aed0 0x7fcb1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.953+0000 7fcb2b7fe700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcb1806aed0 0x7fcb1806d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0b53e0 tx=0x7fcb1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:25.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:25.957+0000 7fcb28ff9700 1 -- 172.21.15.97:0/3567034772 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb20054ef0 con 0x7fcb2c14cce0 2023-11-24T06:09:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.145+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb2c068fb0 con 0x7fcb2c14cce0 2023-11-24T06:09:26.148 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.145+0000 7fcb28ff9700 1 -- 172.21.15.97:0/3567034772 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb20058510 con 0x7fcb2c14cce0 2023-11-24T06:09:26.148 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:26.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcb1806aed0 msgr2=0x7fcb1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcb1806aed0 0x7fcb1806d390 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0b53e0 tx=0x7fcb1c009040 comp rx=0 tx=0).stop 2023-11-24T06:09:26.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c14cce0 msgr2=0x7fcb2c15c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c14cce0 0x7fcb2c15c1e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcb20011ad0 tx=0x7fcb20011e90 comp rx=0 tx=0).stop 2023-11-24T06:09:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 shutdown_connections 2023-11-24T06:09:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb2c14ef50 0x7fcb2c14f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb2c14cce0 0x7fcb2c15c1e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcb1806aed0 0x7fcb1806d390 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 --2- 172.21.15.97:0/3567034772 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb2c15abd0 0x7fcb2c14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 >> 172.21.15.97:0/3567034772 conn(0x7fcb2c0b2050 msgr2=0x7fcb2c0b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 shutdown_connections 2023-11-24T06:09:26.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.147+0000 7fcb32369700 1 -- 172.21.15.97:0/3567034772 wait complete. 2023-11-24T06:09:26.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.279+0000 7fa29747d700 1 Processor -- start 2023-11-24T06:09:26.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.280+0000 7fa29747d700 1 -- start start 2023-11-24T06:09:26.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.280+0000 7fa29747d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa29015f560 0x7fa2900d0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.280+0000 7fa29747d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2900d0740 0x7fa2900d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.280+0000 7fa29747d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2900cd060 con 0x7fa2900d2020 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.280+0000 7fa29747d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa29015f940 con 0x7fa29015f560 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.280+0000 7fa29747d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900cdec0 con 0x7fa2900d0740 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa294a18700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2900d0740 0x7fa2900d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa295219700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa29015f560 0x7fa2900d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa294a18700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2900d0740 0x7fa2900d1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:32950/0 (socket says 172.21.15.97:32950) 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa295a1a700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fa2900d2020 0x7fa2900ce1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:47692/0 (socket says 172.21.15.97:47692) 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa295219700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa29015f560 0x7fa2900d0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:40074/0 (socket says 172.21.15.97:40074) 2023-11-24T06:09:26.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa294a18700 1 -- 172.21.15.97:0/1864117395 learned_addr learned my addr 172.21.15.97:0/1864117395 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa294a18700 1 -- 172.21.15.97:0/1864117395 >> v2:172.21.15.183:3300/0 conn(0x7fa29015f560 msgr2=0x7fa2900d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa294a18700 1 --2- 172.21.15.97:0/1864117395 >> v2:172.21.15.183:3300/0 conn(0x7fa29015f560 0x7fa2900d0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa294a18700 1 -- 172.21.15.97:0/1864117395 >> v1:172.21.15.183:6789/0 conn(0x7fa2900d2020 legacy=0x7fa2900ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa294a18700 1 -- 172.21.15.97:0/1864117395 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2900d24b0 con 0x7fa2900d0740 2023-11-24T06:09:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa294a18700 1 --2- 172.21.15.97:0/1864117395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2900d0740 0x7fa2900d1ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0098f0 tx=0x7fa28c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65405996b273991b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa283fff700 1 -- 172.21.15.97:0/1864117395 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c011830 con 0x7fa2900d0740 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa283fff700 1 -- 172.21.15.97:0/1864117395 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c011990 con 0x7fa2900d0740 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.281+0000 7fa283fff700 1 -- 172.21.15.97:0/1864117395 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c011d20 con 0x7fa2900d0740 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.282+0000 7fa29747d700 1 -- 172.21.15.97:0/1864117395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2900d0740 msgr2=0x7fa2900d1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.282+0000 7fa29747d700 1 --2- 172.21.15.97:0/1864117395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2900d0740 0x7fa2900d1ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0098f0 tx=0x7fa28c009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.282+0000 7fa29747d700 1 -- 172.21.15.97:0/1864117395 shutdown_connections 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.282+0000 7fa29747d700 1 --2- 172.21.15.97:0/1864117395 >> v2:172.21.15.183:3300/0 conn(0x7fa29015f560 0x7fa2900d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.282+0000 7fa29747d700 1 --2- 172.21.15.97:0/1864117395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2900d0740 0x7fa2900d1ae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.282+0000 7fa29747d700 1 -- 172.21.15.97:0/1864117395 >> 172.21.15.97:0/1864117395 conn(0x7fa29009aa00 msgr2=0x7fa29009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.282+0000 7fa29747d700 1 -- 172.21.15.97:0/1864117395 shutdown_connections 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.282+0000 7fa29747d700 1 -- 172.21.15.97:0/1864117395 wait complete. 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.283+0000 7fa29747d700 1 Processor -- start 2023-11-24T06:09:26.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.283+0000 7fa29747d700 1 -- start start 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa29747d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2900d0740 0x7fa2900b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa29747d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2900d2020 0x7fa2900b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa29747d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa29015f560 0x7fa29014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa29747d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900b6dd0 con 0x7fa2900d0740 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa29747d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900b70b0 con 0x7fa2900d2020 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa29747d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900b7390 con 0x7fa29015f560 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295219700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2900d0740 0x7fa2900b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295219700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2900d0740 0x7fa2900b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45920/0 (socket says 172.21.15.97:45920) 2023-11-24T06:09:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295219700 1 -- 172.21.15.97:0/649158107 learned_addr learned my addr 172.21.15.97:0/649158107 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa294a18700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2900d2020 0x7fa2900b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295a1a700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa29015f560 0x7fa29014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295219700 1 -- 172.21.15.97:0/649158107 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa29015f560 msgr2=0x7fa29014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295219700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa29015f560 0x7fa29014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295219700 1 -- 172.21.15.97:0/649158107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2900d2020 msgr2=0x7fa2900b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295219700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2900d2020 0x7fa2900b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.284+0000 7fa295219700 1 -- 172.21.15.97:0/649158107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c009550 con 0x7fa2900d0740 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.285+0000 7fa295219700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2900d0740 0x7fa2900b4c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e870 tx=0x7fa28400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.285+0000 7fa281ffb700 1 -- 172.21.15.97:0/649158107 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284009590 con 0x7fa2900d0740 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.285+0000 7fa281ffb700 1 -- 172.21.15.97:0/649158107 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa284013680 con 0x7fa2900d0740 2023-11-24T06:09:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.285+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2900b7670 con 0x7fa2900d0740 2023-11-24T06:09:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.285+0000 7fa281ffb700 1 -- 172.21.15.97:0/649158107 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840139f0 con 0x7fa2900d0740 2023-11-24T06:09:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.285+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2900b6a30 con 0x7fa2900d0740 2023-11-24T06:09:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.286+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa290068f10 con 0x7fa2900d0740 2023-11-24T06:09:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.286+0000 7fa281ffb700 1 -- 172.21.15.97:0/649158107 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa284013b50 con 0x7fa2900d0740 2023-11-24T06:09:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.287+0000 7fa281ffb700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa27406aed0 0x7fa27406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.287+0000 7fa281ffb700 1 -- 172.21.15.97:0/649158107 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa284089430 con 0x7fa2900d0740 2023-11-24T06:09:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.287+0000 7fa294a18700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa27406aed0 0x7fa27406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.287+0000 7fa294a18700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa27406aed0 0x7fa27406d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa290072760 tx=0x7fa28c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.291+0000 7fa281ffb700 1 -- 172.21.15.97:0/649158107 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2840544e0 con 0x7fa2900d0740 2023-11-24T06:09:26.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.484+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2900cbf50 con 0x7fa2900d0740 2023-11-24T06:09:26.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.485+0000 7fa281ffb700 1 -- 172.21.15.97:0/649158107 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7fa284057b00 con 0x7fa2900d0740 2023-11-24T06:09:26.488 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:26.488 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa27406aed0 msgr2=0x7fa27406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa27406aed0 0x7fa27406d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa290072760 tx=0x7fa28c008030 comp rx=0 tx=0).stop 2023-11-24T06:09:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2900d0740 msgr2=0x7fa2900b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2900d0740 0x7fa2900b4c40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa28400e870 tx=0x7fa28400eb80 comp rx=0 tx=0).stop 2023-11-24T06:09:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 shutdown_connections 2023-11-24T06:09:26.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa27406aed0 0x7fa27406d390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa29015f560 0x7fa29014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2900d0740 0x7fa2900b4c40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 --2- 172.21.15.97:0/649158107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2900d2020 0x7fa2900b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 >> 172.21.15.97:0/649158107 conn(0x7fa29009aa00 msgr2=0x7fa290070610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 shutdown_connections 2023-11-24T06:09:26.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.487+0000 7fa29747d700 1 -- 172.21.15.97:0/649158107 wait complete. 2023-11-24T06:09:26.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:26.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:26.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.623+0000 7ff93a9dd700 1 Processor -- start 2023-11-24T06:09:26.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.623+0000 7ff93a9dd700 1 -- start start 2023-11-24T06:09:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.623+0000 7ff93a9dd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff93415e160 0x7ff9340b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.623+0000 7ff93a9dd700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340b3c60 con 0x7ff9340afc90 2023-11-24T06:09:26.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.623+0000 7ff93a9dd700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340b58c0 con 0x7ff9340746a0 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.623+0000 7ff93a9dd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340b4500 con 0x7ff93415e160 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff9337fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff9340afc90 0x7ff934073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38380/0 (socket says 172.21.15.97:38380) 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff9337fe700 1 -- 172.21.15.97:0/1326735832 learned_addr learned my addr 172.21.15.97:0/1326735832 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff933fff700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff93415e160 0x7ff9340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778686158 0 0) 0x7ff9340b3c60 con 0x7ff9340afc90 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff91c0035b0 con 0x7ff9340afc90 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517126264 0 0) 0x7ff91c0035b0 con 0x7ff9340afc90 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff93415e160 msgr2=0x7ff9340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff93415e160 0x7ff9340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 >> v1:172.21.15.183:6789/0 conn(0x7ff9340746a0 legacy=0x7ff93406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93406bdc0 con 0x7ff9340afc90 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7ff928001670 con 0x7ff9340afc90 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 >> v1:172.21.15.97:6789/0 conn(0x7ff9340afc90 legacy=0x7ff934073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff93415e160 0x7ff91c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff91c003f70 0x7ff91c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff91c004d90 0x7ff91c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340b4500 con 0x7ff91c004d90 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c00ba70 con 0x7ff91c003f70 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c00bc30 con 0x7ff93415e160 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff9337fe700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff91c003f70 0x7ff91c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff938f7a700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff91c004d90 0x7ff91c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.624+0000 7ff933fff700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff93415e160 0x7ff91c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff938f7a700 1 -- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff93415e160 msgr2=0x7ff91c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff938f7a700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff93415e160 0x7ff91c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff938f7a700 1 -- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff91c003f70 msgr2=0x7ff91c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff938f7a700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff91c003f70 0x7ff91c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff938f7a700 1 -- 172.21.15.97:0/1326735832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0035b0 con 0x7ff91c004d90 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff938f7a700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff91c004d90 0x7ff91c005210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c260 tx=0x7ff91c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5c7826d837738a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c001e70 con 0x7ff91c004d90 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c00ce20 con 0x7ff91c004d90 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff932ffd700 1 -- 172.21.15.97:0/1326735832 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c012660 con 0x7ff91c004d90 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff93a9dd700 1 -- 172.21.15.97:0/1326735832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff91c004d90 msgr2=0x7ff91c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff93a9dd700 1 --2- 172.21.15.97:0/1326735832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff91c004d90 0x7ff91c005210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c260 tx=0x7ff91c005880 comp rx=0 tx=0).stop 2023-11-24T06:09:26.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.625+0000 7ff938f7a700 1 -- 172.21.15.97:0/1326735832 reap_dead start 2023-11-24T06:09:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.626+0000 7ff93a9dd700 1 -- 172.21.15.97:0/1326735832 shutdown_connections 2023-11-24T06:09:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.626+0000 7ff93a9dd700 1 -- 172.21.15.97:0/1326735832 >> 172.21.15.97:0/1326735832 conn(0x7ff9340d0bf0 msgr2=0x7ff9340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.626+0000 7ff93a9dd700 1 -- 172.21.15.97:0/1326735832 shutdown_connections 2023-11-24T06:09:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.626+0000 7ff93a9dd700 1 -- 172.21.15.97:0/1326735832 wait complete. 2023-11-24T06:09:26.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.627+0000 7ff93a9dd700 1 Processor -- start 2023-11-24T06:09:26.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.627+0000 7ff93a9dd700 1 -- start start 2023-11-24T06:09:26.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff93a9dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff934158ba0 0x7ff9341f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff93a9dd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9341f4ae0 0x7ff9341f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff93a9dd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9340c6fe0 0x7ff9340c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff93a9dd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c6bc0 con 0x7ff934158ba0 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff93a9dd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9341f48e0 con 0x7ff9340c6fe0 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff93a9dd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c5f40 con 0x7ff9341f4ae0 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff934158ba0 0x7ff9341f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff934158ba0 0x7ff9341f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45940/0 (socket says 172.21.15.97:45940) 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 -- 172.21.15.97:0/633174454 learned_addr learned my addr 172.21.15.97:0/633174454 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff9337fe700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9341f4ae0 0x7ff9341f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff938f7a700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9340c6fe0 0x7ff9340c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 -- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9341f4ae0 msgr2=0x7ff9341f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9341f4ae0 0x7ff9341f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 -- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9340c6fe0 msgr2=0x7ff9340c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9340c6fe0 0x7ff9340c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 -- 172.21.15.97:0/633174454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9340c6220 con 0x7ff934158ba0 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff9337fe700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9341f4ae0 0x7ff9341f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff933fff700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff934158ba0 0x7ff9341f43a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff924011ad0 tx=0x7ff924011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff930ff9700 1 -- 172.21.15.97:0/633174454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400f940 con 0x7ff934158ba0 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.628+0000 7ff930ff9700 1 -- 172.21.15.97:0/633174454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92400faa0 con 0x7ff934158ba0 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.629+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9340c6500 con 0x7ff934158ba0 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.629+0000 7ff930ff9700 1 -- 172.21.15.97:0/633174454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400fe50 con 0x7ff934158ba0 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.629+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff934158f80 con 0x7ff934158ba0 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.630+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff934068fb0 con 0x7ff934158ba0 2023-11-24T06:09:26.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.630+0000 7ff930ff9700 1 -- 172.21.15.97:0/633174454 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff924014440 con 0x7ff934158ba0 2023-11-24T06:09:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.630+0000 7ff930ff9700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff92006aed0 0x7ff92006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.630+0000 7ff930ff9700 1 -- 172.21.15.97:0/633174454 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff924089d00 con 0x7ff934158ba0 2023-11-24T06:09:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.630+0000 7ff9337fe700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff92006aed0 0x7ff92006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.631+0000 7ff9337fe700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff92006aed0 0x7ff92006d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff928001050 tx=0x7ff928009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.635+0000 7ff930ff9700 1 -- 172.21.15.97:0/633174454 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff924054e70 con 0x7ff934158ba0 2023-11-24T06:09:26.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.825+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff93406c140 con 0x7ff934158ba0 2023-11-24T06:09:26.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.825+0000 7ff930ff9700 1 -- 172.21.15.97:0/633174454 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff924058490 con 0x7ff934158ba0 2023-11-24T06:09:26.828 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:26.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff92006aed0 msgr2=0x7ff92006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff92006aed0 0x7ff92006d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff928001050 tx=0x7ff928009040 comp rx=0 tx=0).stop 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff934158ba0 msgr2=0x7ff9341f43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff934158ba0 0x7ff9341f43a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff924011ad0 tx=0x7ff924011de0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 shutdown_connections 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff9340c6fe0 0x7ff9340c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff934158ba0 0x7ff9341f43a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff92006aed0 0x7ff92006d390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 --2- 172.21.15.97:0/633174454 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9341f4ae0 0x7ff9341f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 >> 172.21.15.97:0/633174454 conn(0x7ff9340d0bf0 msgr2=0x7ff93415d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 shutdown_connections 2023-11-24T06:09:26.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.827+0000 7ff93a9dd700 1 -- 172.21.15.97:0/633174454 wait complete. 2023-11-24T06:09:26.842 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:26.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.963+0000 7f746d15f700 1 Processor -- start 2023-11-24T06:09:26.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.963+0000 7f746d15f700 1 -- start start 2023-11-24T06:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f746d15f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f746815e160 0x7f74680b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f746d15f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680afc90 0x7f7468073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f746d15f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f74680744d0 0x7f746806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f746d15f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74680b4500 con 0x7f746815e160 2023-11-24T06:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f746d15f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7468074960 con 0x7f74680744d0 2023-11-24T06:09:26.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f746d15f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74680b3c60 con 0x7f74680afc90 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f746815e160 0x7f74680b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f746815e160 0x7f74680b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45948/0 (socket says 172.21.15.97:45948) 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 -- 172.21.15.97:0/2863712037 learned_addr learned my addr 172.21.15.97:0/2863712037 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 -- 172.21.15.97:0/2863712037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680afc90 msgr2=0x7f7468073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 --2- 172.21.15.97:0/2863712037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680afc90 0x7f7468073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 -- 172.21.15.97:0/2863712037 >> v2:172.21.15.183:3300/0 conn(0x7f74680744d0 msgr2=0x7f746806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 --2- 172.21.15.97:0/2863712037 >> v2:172.21.15.183:3300/0 conn(0x7f74680744d0 0x7f746806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 -- 172.21.15.97:0/2863712037 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74680b58c0 con 0x7f746815e160 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7466d9d700 1 --2- 172.21.15.97:0/2863712037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f74680b5da0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74680d1cd0 tx=0x7f745c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebc50815fa44a5c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7465d9b700 1 -- 172.21.15.97:0/2863712037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00b870 con 0x7f746815e160 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7465d9b700 1 -- 172.21.15.97:0/2863712037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c00b9d0 con 0x7f746815e160 2023-11-24T06:09:26.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.964+0000 7f7465d9b700 1 -- 172.21.15.97:0/2863712037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00bd60 con 0x7f746815e160 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 -- 172.21.15.97:0/2863712037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 msgr2=0x7f74680b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 --2- 172.21.15.97:0/2863712037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f74680b5da0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74680d1cd0 tx=0x7f745c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 -- 172.21.15.97:0/2863712037 shutdown_connections 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 --2- 172.21.15.97:0/2863712037 >> v2:172.21.15.183:3300/0 conn(0x7f74680744d0 0x7f746806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 --2- 172.21.15.97:0/2863712037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f74680b5da0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 --2- 172.21.15.97:0/2863712037 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680afc90 0x7f7468073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 -- 172.21.15.97:0/2863712037 >> 172.21.15.97:0/2863712037 conn(0x7f74680d0bf0 msgr2=0x7f74680d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 -- 172.21.15.97:0/2863712037 shutdown_connections 2023-11-24T06:09:26.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.965+0000 7f746d15f700 1 -- 172.21.15.97:0/2863712037 wait complete. 2023-11-24T06:09:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.966+0000 7f746d15f700 1 Processor -- start 2023-11-24T06:09:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746d15f700 1 -- start start 2023-11-24T06:09:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746d15f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680744d0 0x7f7468161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746d15f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74680afc90 0x7f7468151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746d15f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f746815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746d15f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746815fd70 con 0x7f746815e160 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746d15f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74681f8960 con 0x7f74680afc90 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746d15f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74681f7ab0 con 0x7f74680744d0 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f746815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f746815a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45958/0 (socket says 172.21.15.97:45958) 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746759e700 1 -- 172.21.15.97:0/218920199 learned_addr learned my addr 172.21.15.97:0/218920199 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f7466d9d700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680744d0 0x7f7468161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.967+0000 7f746659c700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74680afc90 0x7f7468151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f746759e700 1 -- 172.21.15.97:0/218920199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680744d0 msgr2=0x7f7468161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f746759e700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680744d0 0x7f7468161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f746759e700 1 -- 172.21.15.97:0/218920199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74680afc90 msgr2=0x7f7468151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f746759e700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74680afc90 0x7f7468151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f746759e700 1 -- 172.21.15.97:0/218920199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c009550 con 0x7f746815e160 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f746759e700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f746815a120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f746000b970 tx=0x7f746000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f744f7fe700 1 -- 172.21.15.97:0/218920199 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000d860 con 0x7f746815e160 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f744f7fe700 1 -- 172.21.15.97:0/218920199 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746000d9e0 con 0x7f746815e160 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f744f7fe700 1 -- 172.21.15.97:0/218920199 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000dd90 con 0x7f746815e160 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74681f7d50 con 0x7f746815e160 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.968+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74681f8310 con 0x7f746815e160 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.969+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74681e9010 con 0x7f746815e160 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.970+0000 7f744f7fe700 1 -- 172.21.15.97:0/218920199 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f746000e430 con 0x7f746815e160 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.970+0000 7f744f7fe700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f745406aed0 0x7f745406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.970+0000 7f7466d9d700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f745406aed0 0x7f745406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.970+0000 7f744f7fe700 1 -- 172.21.15.97:0/218920199 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7460006e40 con 0x7f746815e160 2023-11-24T06:09:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.970+0000 7f7466d9d700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f745406aed0 0x7f745406d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f745c014040 tx=0x7f745c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:26.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:26.975+0000 7f744f7fe700 1 -- 172.21.15.97:0/218920199 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74681e9010 con 0x7f746815e160 2023-11-24T06:09:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.163+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74681e9010 con 0x7f746815e160 2023-11-24T06:09:27.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.164+0000 7f744f7fe700 1 -- 172.21.15.97:0/218920199 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f746001c020 con 0x7f746815e160 2023-11-24T06:09:27.167 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f745406aed0 msgr2=0x7f745406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f745406aed0 0x7f745406d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f745c014040 tx=0x7f745c008030 comp rx=0 tx=0).stop 2023-11-24T06:09:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 msgr2=0x7f746815a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f746815a120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f746000b970 tx=0x7f746000bc80 comp rx=0 tx=0).stop 2023-11-24T06:09:27.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 shutdown_connections 2023-11-24T06:09:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f746815e160 0x7f746815a120 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f745406aed0 0x7f745406d390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74680744d0 0x7f7468161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 --2- 172.21.15.97:0/218920199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74680afc90 0x7f7468151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 >> 172.21.15.97:0/218920199 conn(0x7f74680d0bf0 msgr2=0x7f74680720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:27.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 shutdown_connections 2023-11-24T06:09:27.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.166+0000 7f746d15f700 1 -- 172.21.15.97:0/218920199 wait complete. 2023-11-24T06:09:27.181 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:27.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.303+0000 7f8748658700 1 Processor -- start 2023-11-24T06:09:27.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.303+0000 7f8748658700 1 -- start start 2023-11-24T06:09:27.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f8748658700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f874015e320 0x7f87400d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f8748658700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f8740167d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f8748658700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400b6fb0 con 0x7f87400b93c0 2023-11-24T06:09:27.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f8748658700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400d4da0 con 0x7f874015e320 2023-11-24T06:09:27.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f8748658700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400d2940 con 0x7f874006b700 2023-11-24T06:09:27.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f8746bf5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f8740167d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f87463f4700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f874015e320 0x7f87400d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f8745bf3700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f87400b93c0 0x7f874006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:47738/0 (socket says 172.21.15.97:47738) 2023-11-24T06:09:27.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f8745bf3700 1 -- 172.21.15.97:0/877591798 learned_addr learned my addr 172.21.15.97:0/877591798 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:27.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.304+0000 7f87463f4700 1 --2- 172.21.15.97:0/877591798 >> v2:172.21.15.183:3300/0 conn(0x7f874015e320 0x7f87400d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:40122/0 (socket says 172.21.15.97:40122) 2023-11-24T06:09:27.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.305+0000 7f8746bf5700 1 -- 172.21.15.97:0/877591798 >> v2:172.21.15.183:3300/0 conn(0x7f874015e320 msgr2=0x7f87400d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:27.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.305+0000 7f8746bf5700 1 --2- 172.21.15.97:0/877591798 >> v2:172.21.15.183:3300/0 conn(0x7f874015e320 0x7f87400d2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.305+0000 7f8746bf5700 1 -- 172.21.15.97:0/877591798 >> v1:172.21.15.183:6789/0 conn(0x7f87400b93c0 legacy=0x7f874006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.305+0000 7f8746bf5700 1 -- 172.21.15.97:0/877591798 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400d2c10 con 0x7f874006b700 2023-11-24T06:09:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.305+0000 7f8746bf5700 1 --2- 172.21.15.97:0/877591798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f8740167d70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8738009d90 tx=0x7f873800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3433c05692629559 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.305+0000 7f87453f2700 1 -- 172.21.15.97:0/877591798 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800e070 con 0x7f874006b700 2023-11-24T06:09:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.305+0000 7f87453f2700 1 -- 172.21.15.97:0/877591798 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873800bb20 con 0x7f874006b700 2023-11-24T06:09:27.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.305+0000 7f87453f2700 1 -- 172.21.15.97:0/877591798 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738012430 con 0x7f874006b700 2023-11-24T06:09:27.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.306+0000 7f8748658700 1 -- 172.21.15.97:0/877591798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 msgr2=0x7f8740167d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.306+0000 7f8748658700 1 --2- 172.21.15.97:0/877591798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f8740167d70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8738009d90 tx=0x7f873800b330 comp rx=0 tx=0).stop 2023-11-24T06:09:27.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.306+0000 7f8748658700 1 -- 172.21.15.97:0/877591798 shutdown_connections 2023-11-24T06:09:27.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.306+0000 7f8748658700 1 --2- 172.21.15.97:0/877591798 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f8740167d70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.306+0000 7f8748658700 1 --2- 172.21.15.97:0/877591798 >> v2:172.21.15.183:3300/0 conn(0x7f874015e320 0x7f87400d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.306+0000 7f8748658700 1 -- 172.21.15.97:0/877591798 >> 172.21.15.97:0/877591798 conn(0x7f87400d3e50 msgr2=0x7f87400d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:27.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.306+0000 7f8748658700 1 -- 172.21.15.97:0/877591798 shutdown_connections 2023-11-24T06:09:27.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.306+0000 7f8748658700 1 -- 172.21.15.97:0/877591798 wait complete. 2023-11-24T06:09:27.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.307+0000 7f8748658700 1 Processor -- start 2023-11-24T06:09:27.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.307+0000 7f8748658700 1 -- start start 2023-11-24T06:09:27.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8748658700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f87401e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8748658700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87400b93c0 0x7f87400cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8748658700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874015e320 0x7f874015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8748658700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401e6fd0 con 0x7f874015e320 2023-11-24T06:09:27.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8748658700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401e72b0 con 0x7f87400b93c0 2023-11-24T06:09:27.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8748658700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87401e7590 con 0x7f874006b700 2023-11-24T06:09:27.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874015e320 0x7f874015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874015e320 0x7f874015cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45972/0 (socket says 172.21.15.97:45972) 2023-11-24T06:09:27.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 -- 172.21.15.97:0/989215431 learned_addr learned my addr 172.21.15.97:0/989215431 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:27.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8745bf3700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87400b93c0 0x7f87400cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f87463f4700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f87401e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 -- 172.21.15.97:0/989215431 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 msgr2=0x7f87401e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f87401e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 -- 172.21.15.97:0/989215431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87400b93c0 msgr2=0x7f87400cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87400b93c0 0x7f87400cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 -- 172.21.15.97:0/989215431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738009a70 con 0x7f874015e320 2023-11-24T06:09:27.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8746bf5700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874015e320 0x7f874015cee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8738014040 tx=0x7f8738004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.308+0000 7f8736ffd700 1 -- 172.21.15.97:0/989215431 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738012430 con 0x7f874015e320 2023-11-24T06:09:27.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.309+0000 7f8736ffd700 1 -- 172.21.15.97:0/989215431 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8738020070 con 0x7f874015e320 2023-11-24T06:09:27.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.309+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87401e7870 con 0x7f874015e320 2023-11-24T06:09:27.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.309+0000 7f8736ffd700 1 -- 172.21.15.97:0/989215431 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738003ea0 con 0x7f874015e320 2023-11-24T06:09:27.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.309+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87401e6cc0 con 0x7f874015e320 2023-11-24T06:09:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.310+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87400cfc10 con 0x7f874015e320 2023-11-24T06:09:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.310+0000 7f8736ffd700 1 -- 172.21.15.97:0/989215431 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8738004280 con 0x7f874015e320 2023-11-24T06:09:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.310+0000 7f8736ffd700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f872406aed0 0x7f872406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.311+0000 7f87463f4700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f872406aed0 0x7f872406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.311+0000 7f8736ffd700 1 -- 172.21.15.97:0/989215431 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8738022070 con 0x7f874015e320 2023-11-24T06:09:27.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.311+0000 7f87463f4700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f872406aed0 0x7f872406d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8730009ad0 tx=0x7f8730008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.315+0000 7f8736ffd700 1 -- 172.21.15.97:0/989215431 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8738056410 con 0x7f874015e320 2023-11-24T06:09:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.504+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8740068fb0 con 0x7f874015e320 2023-11-24T06:09:27.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.505+0000 7f8736ffd700 1 -- 172.21.15.97:0/989215431 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8738059a30 con 0x7f874015e320 2023-11-24T06:09:27.508 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:27.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f872406aed0 msgr2=0x7f872406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f872406aed0 0x7f872406d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8730009ad0 tx=0x7f8730008040 comp rx=0 tx=0).stop 2023-11-24T06:09:27.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874015e320 msgr2=0x7f874015cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874015e320 0x7f874015cee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8738014040 tx=0x7f8738004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 shutdown_connections 2023-11-24T06:09:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f874015e320 0x7f874015cee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f872406aed0 0x7f872406d390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f874006b700 0x7f87401e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 --2- 172.21.15.97:0/989215431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87400b93c0 0x7f87400cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 >> 172.21.15.97:0/989215431 conn(0x7f87400d3e50 msgr2=0x7f87400d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 shutdown_connections 2023-11-24T06:09:27.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.507+0000 7f8748658700 1 -- 172.21.15.97:0/989215431 wait complete. 2023-11-24T06:09:27.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.639+0000 7f6fc8b75700 1 Processor -- start 2023-11-24T06:09:27.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.639+0000 7f6fc8b75700 1 -- start start 2023-11-24T06:09:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc8b75700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6fc415bdd0 0x7f6fc41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc8b75700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc40d1c70 0x7f6fc40d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc8b75700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc40b22d0 con 0x7f6fc40b3370 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc8b75700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc40b2f50 con 0x7f6fc415bdd0 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc8b75700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40d2640 con 0x7f6fc40d1c70 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc2d9d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc40d1c70 0x7f6fc40d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc259c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6fc415bdd0 0x7f6fc41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc259c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6fc415bdd0 0x7f6fc41604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:40136/0 (socket says 172.21.15.97:40136) 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc2d9d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc40d1c70 0x7f6fc40d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:33016/0 (socket says 172.21.15.97:33016) 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc1d9b700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f6fc40b3370 0x7f6fc40b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:47752/0 (socket says 172.21.15.97:47752) 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.640+0000 7f6fc259c700 1 -- 172.21.15.97:0/3443381 learned_addr learned my addr 172.21.15.97:0/3443381 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc2d9d700 1 -- 172.21.15.97:0/3443381 >> v2:172.21.15.183:3300/0 conn(0x7f6fc415bdd0 msgr2=0x7f6fc41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc2d9d700 1 --2- 172.21.15.97:0/3443381 >> v2:172.21.15.183:3300/0 conn(0x7f6fc415bdd0 0x7f6fc41604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc2d9d700 1 -- 172.21.15.97:0/3443381 >> v1:172.21.15.183:6789/0 conn(0x7f6fc40b3370 legacy=0x7f6fc40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc2d9d700 1 -- 172.21.15.97:0/3443381 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40b25b0 con 0x7f6fc40d1c70 2023-11-24T06:09:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc259c700 1 --2- 172.21.15.97:0/3443381 >> v2:172.21.15.183:3300/0 conn(0x7f6fc415bdd0 0x7f6fc41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc2d9d700 1 --2- 172.21.15.97:0/3443381 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc40d1c70 0x7f6fc40d2100 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009d90 tx=0x7f6fbc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9e43e7c07f7564e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc159a700 1 -- 172.21.15.97:0/3443381 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00e070 con 0x7f6fc40d1c70 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc159a700 1 -- 172.21.15.97:0/3443381 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc00bb20 con 0x7f6fc40d1c70 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc159a700 1 -- 172.21.15.97:0/3443381 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc012430 con 0x7f6fc40d1c70 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc8b75700 1 -- 172.21.15.97:0/3443381 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc40d1c70 msgr2=0x7f6fc40d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/3443381 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc40d1c70 0x7f6fc40d2100 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009d90 tx=0x7f6fbc00b330 comp rx=0 tx=0).stop 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc8b75700 1 -- 172.21.15.97:0/3443381 shutdown_connections 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/3443381 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc40d1c70 0x7f6fc40d2100 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.641+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/3443381 >> v2:172.21.15.183:3300/0 conn(0x7f6fc415bdd0 0x7f6fc41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.642+0000 7f6fc8b75700 1 -- 172.21.15.97:0/3443381 >> 172.21.15.97:0/3443381 conn(0x7f6fc40731a0 msgr2=0x7f6fc409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.642+0000 7f6fc8b75700 1 -- 172.21.15.97:0/3443381 shutdown_connections 2023-11-24T06:09:27.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.642+0000 7f6fc8b75700 1 -- 172.21.15.97:0/3443381 wait complete. 2023-11-24T06:09:27.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.643+0000 7f6fc8b75700 1 Processor -- start 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.643+0000 7f6fc8b75700 1 -- start start 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc8b75700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc40b3370 0x7f6fc40cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc8b75700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc40d1c70 0x7f6fc41e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc8b75700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc415bdd0 0x7f6fc40c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc8b75700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40b5ea0 con 0x7f6fc40b3370 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc8b75700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40cb840 con 0x7f6fc40d1c70 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc8b75700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40b5290 con 0x7f6fc415bdd0 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc40b3370 0x7f6fc40cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc40b3370 0x7f6fc40cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45982/0 (socket says 172.21.15.97:45982) 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 -- 172.21.15.97:0/1410367343 learned_addr learned my addr 172.21.15.97:0/1410367343 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:27.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc1d9b700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc40d1c70 0x7f6fc41e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc2d9d700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc415bdd0 0x7f6fc40c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 -- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc415bdd0 msgr2=0x7f6fc40c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc415bdd0 0x7f6fc40c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 -- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc40d1c70 msgr2=0x7f6fc41e7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc40d1c70 0x7f6fc41e7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 -- 172.21.15.97:0/1410367343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4009550 con 0x7f6fc40b3370 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc2d9d700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc415bdd0 0x7f6fc40c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc259c700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc40b3370 0x7f6fc40cb300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400e9b0 tx=0x7f6fb400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fb2ffd700 1 -- 172.21.15.97:0/1410367343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400c850 con 0x7f6fc40b3370 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fb2ffd700 1 -- 172.21.15.97:0/1410367343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb400c9b0 con 0x7f6fc40b3370 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc009a70 con 0x7f6fc40b3370 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fb2ffd700 1 -- 172.21.15.97:0/1410367343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400cd20 con 0x7f6fc40b3370 2023-11-24T06:09:27.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.644+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc40b5850 con 0x7f6fc40b3370 2023-11-24T06:09:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.645+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc4067f60 con 0x7f6fc40b3370 2023-11-24T06:09:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.646+0000 7f6fb2ffd700 1 -- 172.21.15.97:0/1410367343 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fb4013470 con 0x7f6fc40b3370 2023-11-24T06:09:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.646+0000 7f6fb2ffd700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6fa406aed0 0x7f6fa406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.646+0000 7f6fb2ffd700 1 -- 172.21.15.97:0/1410367343 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fb4089610 con 0x7f6fc40b3370 2023-11-24T06:09:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.646+0000 7f6fc1d9b700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6fa406aed0 0x7f6fa406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.646+0000 7f6fc1d9b700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6fa406aed0 0x7f6fa406d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6fc41f5470 tx=0x7f6fb8007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.651+0000 7f6fb2ffd700 1 -- 172.21.15.97:0/1410367343 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fb40546c0 con 0x7f6fc40b3370 2023-11-24T06:09:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.843+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fc40b7c60 con 0x7f6fc40b3370 2023-11-24T06:09:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.844+0000 7f6fb2ffd700 1 -- 172.21.15.97:0/1410367343 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f6fb4057ce0 con 0x7f6fc40b3370 2023-11-24T06:09:27.848 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:27.848 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:27.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6fa406aed0 msgr2=0x7f6fa406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6fa406aed0 0x7f6fa406d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6fc41f5470 tx=0x7f6fb8007f10 comp rx=0 tx=0).stop 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc40b3370 msgr2=0x7f6fc40cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc40b3370 0x7f6fc40cb300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400e9b0 tx=0x7f6fb400ecc0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 shutdown_connections 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6fa406aed0 0x7f6fa406d390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6fc415bdd0 0x7f6fc40c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc40b3370 0x7f6fc40cb300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 --2- 172.21.15.97:0/1410367343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fc40d1c70 0x7f6fc41e7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 >> 172.21.15.97:0/1410367343 conn(0x7f6fc40731a0 msgr2=0x7f6fc406ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 shutdown_connections 2023-11-24T06:09:27.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.846+0000 7f6fc8b75700 1 -- 172.21.15.97:0/1410367343 wait complete. 2023-11-24T06:09:27.861 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:27.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:27.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.981+0000 7f4e8cacf700 1 Processor -- start 2023-11-24T06:09:27.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.981+0000 7f4e8cacf700 1 -- start start 2023-11-24T06:09:27.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.982+0000 7f4e8cacf700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4e8806c710 0x7f4e8815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.982+0000 7f4e8cacf700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e8815f2d0 0x7f4e8806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.982+0000 7f4e8cacf700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e880b46d0 con 0x7f4e88143b80 2023-11-24T06:09:27.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.982+0000 7f4e8cacf700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8815faf0 con 0x7f4e8806c710 2023-11-24T06:09:27.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.982+0000 7f4e8cacf700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880b3de0 con 0x7f4e8815f2d0 2023-11-24T06:09:27.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e86d9d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e8815f2d0 0x7f4e8806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e85d9b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4e8806c710 0x7f4e8815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e8659c700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f4e88143b80 0x7f4e8806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:47764/0 (socket says 172.21.15.97:47764) 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e85d9b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4e8806c710 0x7f4e8815ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:40152/0 (socket says 172.21.15.97:40152) 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e8659c700 1 -- 172.21.15.97:0/2543042895 learned_addr learned my addr 172.21.15.97:0/2543042895 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e86d9d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e8815f2d0 0x7f4e8806ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:33030/0 (socket says 172.21.15.97:33030) 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e86d9d700 1 -- 172.21.15.97:0/2543042895 >> v2:172.21.15.183:3300/0 conn(0x7f4e8806c710 msgr2=0x7f4e8815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e86d9d700 1 --2- 172.21.15.97:0/2543042895 >> v2:172.21.15.183:3300/0 conn(0x7f4e8806c710 0x7f4e8815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e86d9d700 1 -- 172.21.15.97:0/2543042895 >> v1:172.21.15.183:6789/0 conn(0x7f4e88143b80 legacy=0x7f4e8806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e86d9d700 1 -- 172.21.15.97:0/2543042895 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880b4080 con 0x7f4e8815f2d0 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e86d9d700 1 --2- 172.21.15.97:0/2543042895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e8815f2d0 0x7f4e8806ee00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009d90 tx=0x7f4e8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67adbfe8e16f66f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e8559a700 1 -- 172.21.15.97:0/2543042895 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8000e070 con 0x7f4e8815f2d0 2023-11-24T06:09:27.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e8559a700 1 -- 172.21.15.97:0/2543042895 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8000bb20 con 0x7f4e8815f2d0 2023-11-24T06:09:27.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.983+0000 7f4e8559a700 1 -- 172.21.15.97:0/2543042895 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e80012430 con 0x7f4e8815f2d0 2023-11-24T06:09:27.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.984+0000 7f4e8cacf700 1 -- 172.21.15.97:0/2543042895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e8815f2d0 msgr2=0x7f4e8806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.984+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/2543042895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e8815f2d0 0x7f4e8806ee00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009d90 tx=0x7f4e8000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.984+0000 7f4e8cacf700 1 -- 172.21.15.97:0/2543042895 shutdown_connections 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.984+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/2543042895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e8815f2d0 0x7f4e8806ee00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.984+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/2543042895 >> v2:172.21.15.183:3300/0 conn(0x7f4e8806c710 0x7f4e8815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.984+0000 7f4e8cacf700 1 -- 172.21.15.97:0/2543042895 >> 172.21.15.97:0/2543042895 conn(0x7f4e88071ac0 msgr2=0x7f4e88071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.984+0000 7f4e8cacf700 1 -- 172.21.15.97:0/2543042895 shutdown_connections 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.984+0000 7f4e8cacf700 1 -- 172.21.15.97:0/2543042895 wait complete. 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.985+0000 7f4e8cacf700 1 Processor -- start 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.985+0000 7f4e8cacf700 1 -- start start 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8cacf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8806c710 0x7f4e881f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8cacf700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e88143b80 0x7f4e880a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8cacf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8815f2d0 0x7f4e881f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8cacf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880aa3b0 con 0x7f4e8806c710 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8cacf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880aaa60 con 0x7f4e8815f2d0 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8cacf700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e880aad40 con 0x7f4e88143b80 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8806c710 0x7f4e881f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8806c710 0x7f4e881f4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45998/0 (socket says 172.21.15.97:45998) 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e85d9b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e88143b80 0x7f4e880a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8659c700 1 -- 172.21.15.97:0/1643789405 learned_addr learned my addr 172.21.15.97:0/1643789405 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e86d9d700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8815f2d0 0x7f4e881f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.986+0000 7f4e8659c700 1 -- 172.21.15.97:0/1643789405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e88143b80 msgr2=0x7f4e880a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e8659c700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e88143b80 0x7f4e880a9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e8659c700 1 -- 172.21.15.97:0/1643789405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8815f2d0 msgr2=0x7f4e881f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e8659c700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8815f2d0 0x7f4e881f70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:27.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e8659c700 1 -- 172.21.15.97:0/1643789405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e80009a70 con 0x7f4e8806c710 2023-11-24T06:09:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e8659c700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8806c710 0x7f4e881f4410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c520 tx=0x7f4e7c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e76ffd700 1 -- 172.21.15.97:0/1643789405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c005d70 con 0x7f4e8806c710 2023-11-24T06:09:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e76ffd700 1 -- 172.21.15.97:0/1643789405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0070d0 con 0x7f4e8806c710 2023-11-24T06:09:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8815f8c0 con 0x7f4e8806c710 2023-11-24T06:09:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e76ffd700 1 -- 172.21.15.97:0/1643789405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c007440 con 0x7f4e8806c710 2023-11-24T06:09:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.987+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e880ab300 con 0x7f4e8806c710 2023-11-24T06:09:27.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.988+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e88068f10 con 0x7f4e8806c710 2023-11-24T06:09:27.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.988+0000 7f4e76ffd700 1 -- 172.21.15.97:0/1643789405 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e7c0075a0 con 0x7f4e8806c710 2023-11-24T06:09:27.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.988+0000 7f4e76ffd700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4e6806aed0 0x7f4e6806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:27.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.989+0000 7f4e76ffd700 1 -- 172.21.15.97:0/1643789405 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e7c0888e0 con 0x7f4e8806c710 2023-11-24T06:09:27.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.989+0000 7f4e85d9b700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4e6806aed0 0x7f4e6806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:27.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.989+0000 7f4e85d9b700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4e6806aed0 0x7f4e6806d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e880b42a0 tx=0x7f4e78009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:27.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:27.993+0000 7f4e76ffd700 1 -- 172.21.15.97:0/1643789405 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e7c0539b0 con 0x7f4e8806c710 2023-11-24T06:09:28.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.184+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e880b52d0 con 0x7f4e8806c710 2023-11-24T06:09:28.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.184+0000 7f4e76ffd700 1 -- 172.21.15.97:0/1643789405 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e7c017020 con 0x7f4e8806c710 2023-11-24T06:09:28.187 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:28.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4e6806aed0 msgr2=0x7f4e6806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4e6806aed0 0x7f4e6806d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e880b42a0 tx=0x7f4e78009210 comp rx=0 tx=0).stop 2023-11-24T06:09:28.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8806c710 msgr2=0x7f4e881f4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8806c710 0x7f4e881f4410 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c520 tx=0x7f4e7c00c830 comp rx=0 tx=0).stop 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 shutdown_connections 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8815f2d0 0x7f4e881f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e8806c710 0x7f4e881f4410 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4e6806aed0 0x7f4e6806d390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 --2- 172.21.15.97:0/1643789405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4e88143b80 0x7f4e880a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 >> 172.21.15.97:0/1643789405 conn(0x7f4e88071ac0 msgr2=0x7f4e880b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 shutdown_connections 2023-11-24T06:09:28.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.186+0000 7f4e8cacf700 1 -- 172.21.15.97:0/1643789405 wait complete. 2023-11-24T06:09:28.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:28.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.321+0000 7f9df5e55700 1 Processor -- start 2023-11-24T06:09:28.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.321+0000 7f9df5e55700 1 -- start start 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9df5e55700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9df0155170 0x7f9df00b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9df5e55700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9df015bcb0 0x7f9df00728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9df5e55700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df00ce680 con 0x7f9df00b5d90 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9df5e55700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df015c230 con 0x7f9df0155170 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9df5e55700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df00ce940 con 0x7f9df015bcb0 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9def7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9df0155170 0x7f9df00b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9def7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9df0155170 0x7f9df00b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46004/0 (socket says 172.21.15.97:46004) 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9def7fe700 1 -- 172.21.15.97:0/168537596 learned_addr learned my addr 172.21.15.97:0/168537596 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9deeffd700 1 --2- 172.21.15.97:0/168537596 >> v2:172.21.15.183:3300/0 conn(0x7f9df015bcb0 0x7f9df00728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9def7fe700 1 -- 172.21.15.97:0/168537596 >> v2:172.21.15.183:3300/0 conn(0x7f9df015bcb0 msgr2=0x7f9df00728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:28.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9def7fe700 1 --2- 172.21.15.97:0/168537596 >> v2:172.21.15.183:3300/0 conn(0x7f9df015bcb0 0x7f9df00728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9def7fe700 1 -- 172.21.15.97:0/168537596 >> v1:172.21.15.183:6789/0 conn(0x7f9df00b5d90 legacy=0x7f9df00cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9def7fe700 1 -- 172.21.15.97:0/168537596 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df00ccbf0 con 0x7f9df0155170 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.322+0000 7f9def7fe700 1 --2- 172.21.15.97:0/168537596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b30d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9df00b5350 tx=0x7f9de8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13de63c08b54ad2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9dee7fc700 1 -- 172.21.15.97:0/168537596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800b870 con 0x7f9df0155170 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9dee7fc700 1 -- 172.21.15.97:0/168537596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de800b9d0 con 0x7f9df0155170 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9dee7fc700 1 -- 172.21.15.97:0/168537596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de800bd60 con 0x7f9df0155170 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9df5e55700 1 -- 172.21.15.97:0/168537596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 msgr2=0x7f9df00b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9df5e55700 1 --2- 172.21.15.97:0/168537596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b30d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9df00b5350 tx=0x7f9de8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9df5e55700 1 -- 172.21.15.97:0/168537596 shutdown_connections 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9df5e55700 1 --2- 172.21.15.97:0/168537596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b30d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9df5e55700 1 --2- 172.21.15.97:0/168537596 >> v2:172.21.15.183:3300/0 conn(0x7f9df015bcb0 0x7f9df00728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9df5e55700 1 -- 172.21.15.97:0/168537596 >> 172.21.15.97:0/168537596 conn(0x7f9df006cec0 msgr2=0x7f9df006d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:28.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9df5e55700 1 -- 172.21.15.97:0/168537596 shutdown_connections 2023-11-24T06:09:28.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.323+0000 7f9df5e55700 1 -- 172.21.15.97:0/168537596 wait complete. 2023-11-24T06:09:28.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.324+0000 7f9df5e55700 1 Processor -- start 2023-11-24T06:09:28.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9df5e55700 1 -- start start 2023-11-24T06:09:28.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9df5e55700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00b5d90 0x7f9df0159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9df5e55700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9df5e55700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df015bcb0 0x7f9df00b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9df5e55700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df014bcb0 con 0x7f9df0155170 2023-11-24T06:09:28.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9df5e55700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df014b0a0 con 0x7f9df00b5d90 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9df5e55700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df014b380 con 0x7f9df015bcb0 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9deeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9deeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46012/0 (socket says 172.21.15.97:46012) 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9deeffd700 1 -- 172.21.15.97:0/1657752197 learned_addr learned my addr 172.21.15.97:0/1657752197 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.325+0000 7f9def7fe700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00b5d90 0x7f9df0159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9deffff700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df015bcb0 0x7f9df00b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9deeffd700 1 -- 172.21.15.97:0/1657752197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df015bcb0 msgr2=0x7f9df00b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9deeffd700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df015bcb0 0x7f9df00b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9deeffd700 1 -- 172.21.15.97:0/1657752197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00b5d90 msgr2=0x7f9df0159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9deeffd700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00b5d90 0x7f9df0159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9deeffd700 1 -- 172.21.15.97:0/1657752197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de8009550 con 0x7f9df0155170 2023-11-24T06:09:28.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9deeffd700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b3370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800e760 tx=0x7f9dd800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9de7fff700 1 -- 172.21.15.97:0/1657752197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8005c20 con 0x7f9df0155170 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9de7fff700 1 -- 172.21.15.97:0/1657752197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd8005da0 con 0x7f9df0155170 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9de7fff700 1 -- 172.21.15.97:0/1657752197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8006260 con 0x7f9df0155170 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df014b660 con 0x7f9df0155170 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.326+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df0159d50 con 0x7f9df0155170 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.327+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df0062400 con 0x7f9df0155170 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.327+0000 7f9de7fff700 1 -- 172.21.15.97:0/1657752197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9dd80063c0 con 0x7f9df0155170 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.328+0000 7f9de7fff700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd006aed0 0x7f9dd006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.328+0000 7f9de7fff700 1 -- 172.21.15.97:0/1657752197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dd8088e80 con 0x7f9df0155170 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.328+0000 7f9def7fe700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd006aed0 0x7f9dd006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.328+0000 7f9def7fe700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd006aed0 0x7f9dd006d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000c00 tx=0x7f9de8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:28.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.332+0000 7f9de7fff700 1 -- 172.21.15.97:0/1657752197 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9dd8053f30 con 0x7f9df0155170 2023-11-24T06:09:28.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.522+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9df0068fb0 con 0x7f9df0155170 2023-11-24T06:09:28.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.522+0000 7f9de7fff700 1 -- 172.21.15.97:0/1657752197 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9dd8020020 con 0x7f9df0155170 2023-11-24T06:09:28.525 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:28.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd006aed0 msgr2=0x7f9dd006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd006aed0 0x7f9dd006d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000c00 tx=0x7f9de8008030 comp rx=0 tx=0).stop 2023-11-24T06:09:28.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 msgr2=0x7f9df00b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b3370 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800e760 tx=0x7f9dd800ea70 comp rx=0 tx=0).stop 2023-11-24T06:09:28.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 shutdown_connections 2023-11-24T06:09:28.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd006aed0 0x7f9dd006d390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df015bcb0 0x7f9df00b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00b5d90 0x7f9df0159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 --2- 172.21.15.97:0/1657752197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df0155170 0x7f9df00b3370 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 >> 172.21.15.97:0/1657752197 conn(0x7f9df006cec0 msgr2=0x7f9df015c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:28.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 shutdown_connections 2023-11-24T06:09:28.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.524+0000 7f9df5e55700 1 -- 172.21.15.97:0/1657752197 wait complete. 2023-11-24T06:09:28.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:28.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.659+0000 7f0e27d9a700 1 Processor -- start 2023-11-24T06:09:28.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.660+0000 7f0e27d9a700 1 -- start start 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.660+0000 7f0e27d9a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0e2015bf50 0x7f0e200ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.660+0000 7f0e27d9a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0e200c6c30 0x7f0e200cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.660+0000 7f0e27d9a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e200d0140 con 0x7f0e200ce830 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.660+0000 7f0e27d9a700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2015c330 con 0x7f0e200c6c30 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.660+0000 7f0e27d9a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e200cf3c0 con 0x7f0e2015bf50 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e26337700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0e200c6c30 0x7f0e200cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e25335700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0e200ce830 0x7f0e200cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38470/0 (socket says 172.21.15.97:38470) 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e25335700 1 -- 172.21.15.97:0/265881545 learned_addr learned my addr 172.21.15.97:0/265881545 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e25b36700 1 --2- 172.21.15.97:0/265881545 >> v2:172.21.15.183:3300/0 conn(0x7f0e2015bf50 0x7f0e200ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e26337700 1 -- 172.21.15.97:0/265881545 >> v2:172.21.15.183:3300/0 conn(0x7f0e2015bf50 msgr2=0x7f0e200ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e26337700 1 --2- 172.21.15.97:0/265881545 >> v2:172.21.15.183:3300/0 conn(0x7f0e2015bf50 0x7f0e200ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e26337700 1 -- 172.21.15.97:0/265881545 >> v1:172.21.15.97:6789/0 conn(0x7f0e200ce830 legacy=0x7f0e200cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e24b34700 1 -- 172.21.15.97:0/265881545 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919053531 0 0) 0x7f0e200d0140 con 0x7f0e200ce830 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e26337700 1 -- 172.21.15.97:0/265881545 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200d03e0 con 0x7f0e200c6c30 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e26337700 1 --2- 172.21.15.97:0/265881545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200c6c30 0x7f0e200cc6c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e200715e0 tx=0x7f0e1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0ef1ddd817d7c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e24b34700 1 -- 172.21.15.97:0/265881545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1800e070 con 0x7f0e200c6c30 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e24b34700 1 -- 172.21.15.97:0/265881545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1800bbd0 con 0x7f0e200c6c30 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.661+0000 7f0e24b34700 1 -- 172.21.15.97:0/265881545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18012430 con 0x7f0e200c6c30 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.662+0000 7f0e27d9a700 1 -- 172.21.15.97:0/265881545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200c6c30 msgr2=0x7f0e200cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.662+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/265881545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200c6c30 0x7f0e200cc6c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0e200715e0 tx=0x7f0e1800b330 comp rx=0 tx=0).stop 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.662+0000 7f0e27d9a700 1 -- 172.21.15.97:0/265881545 shutdown_connections 2023-11-24T06:09:28.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.662+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/265881545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e200c6c30 0x7f0e200cc6c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.662+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/265881545 >> v2:172.21.15.183:3300/0 conn(0x7f0e2015bf50 0x7f0e200ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.662+0000 7f0e27d9a700 1 -- 172.21.15.97:0/265881545 >> 172.21.15.97:0/265881545 conn(0x7f0e200b3780 msgr2=0x7f0e200b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:28.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.662+0000 7f0e27d9a700 1 -- 172.21.15.97:0/265881545 shutdown_connections 2023-11-24T06:09:28.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.662+0000 7f0e27d9a700 1 -- 172.21.15.97:0/265881545 wait complete. 2023-11-24T06:09:28.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.663+0000 7f0e27d9a700 1 Processor -- start 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.663+0000 7f0e27d9a700 1 -- start start 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e27d9a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e200c6c30 0x7f0e200b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e27d9a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0e200ce830 0x7f0e200c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e27d9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015bf50 0x7f0e2009e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e27d9a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e200cab50 con 0x7f0e2015bf50 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e27d9a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20147310 con 0x7f0e200c6c30 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e27d9a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e201f16d0 con 0x7f0e200ce830 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015bf50 0x7f0e2009e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015bf50 0x7f0e2009e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46026/0 (socket says 172.21.15.97:46026) 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 -- 172.21.15.97:0/1926214920 learned_addr learned my addr 172.21.15.97:0/1926214920 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:28.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e25b36700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e200c6c30 0x7f0e200b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 -- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0e200ce830 msgr2=0x7f0e200c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e25335700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0e200ce830 0x7f0e200c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0e200ce830 0x7f0e200c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 -- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e200c6c30 msgr2=0x7f0e200b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e200c6c30 0x7f0e200b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 -- 172.21.15.97:0/1926214920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e18009a70 con 0x7f0e2015bf50 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e25b36700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e200c6c30 0x7f0e200b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e26337700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015bf50 0x7f0e2009e240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0e200723f0 tx=0x7f0e18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.664+0000 7f0e167fc700 1 -- 172.21.15.97:0/1926214920 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18012430 con 0x7f0e2015bf50 2023-11-24T06:09:28.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.665+0000 7f0e167fc700 1 -- 172.21.15.97:0/1926214920 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e18020070 con 0x7f0e2015bf50 2023-11-24T06:09:28.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.665+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e200b2ce0 con 0x7f0e2015bf50 2023-11-24T06:09:28.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.665+0000 7f0e167fc700 1 -- 172.21.15.97:0/1926214920 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18004280 con 0x7f0e2015bf50 2023-11-24T06:09:28.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.665+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e200ca870 con 0x7f0e2015bf50 2023-11-24T06:09:28.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.666+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e20068fb0 con 0x7f0e2015bf50 2023-11-24T06:09:28.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.666+0000 7f0e167fc700 1 -- 172.21.15.97:0/1926214920 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e180043e0 con 0x7f0e2015bf50 2023-11-24T06:09:28.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.666+0000 7f0e167fc700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0e0406aed0 0x7f0e0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:28.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.666+0000 7f0e167fc700 1 -- 172.21.15.97:0/1926214920 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e18022070 con 0x7f0e2015bf50 2023-11-24T06:09:28.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.666+0000 7f0e25b36700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0e0406aed0 0x7f0e0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:28.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.667+0000 7f0e25b36700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0e0406aed0 0x7f0e0406d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009820 tx=0x7f0e10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:28.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.671+0000 7f0e167fc700 1 -- 172.21.15.97:0/1926214920 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e18056480 con 0x7f0e2015bf50 2023-11-24T06:09:28.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.865+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e20147450 con 0x7f0e2015bf50 2023-11-24T06:09:28.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.866+0000 7f0e167fc700 1 -- 172.21.15.97:0/1926214920 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e20147450 con 0x7f0e2015bf50 2023-11-24T06:09:28.869 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.867+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0e0406aed0 msgr2=0x7f0e0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.867+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0e0406aed0 0x7f0e0406d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e10009820 tx=0x7f0e10008040 comp rx=0 tx=0).stop 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.867+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015bf50 msgr2=0x7f0e2009e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.867+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015bf50 0x7f0e2009e240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0e200723f0 tx=0x7f0e18004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.868+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 shutdown_connections 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.868+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e2015bf50 0x7f0e2009e240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.868+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e200c6c30 0x7f0e200b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.868+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0e0406aed0 0x7f0e0406d390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.868+0000 7f0e27d9a700 1 --2- 172.21.15.97:0/1926214920 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0e200ce830 0x7f0e200c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.868+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 >> 172.21.15.97:0/1926214920 conn(0x7f0e200b3780 msgr2=0x7f0e20061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:28.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.868+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 shutdown_connections 2023-11-24T06:09:28.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:28.868+0000 7f0e27d9a700 1 -- 172.21.15.97:0/1926214920 wait complete. 2023-11-24T06:09:28.883 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.000+0000 7f1a14754700 1 Processor -- start 2023-11-24T06:09:29.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.000+0000 7f1a14754700 1 -- start start 2023-11-24T06:09:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a14754700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a0c06c710 0x7f1a0c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a14754700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c0ae430 con 0x7f1a0c15e080 2023-11-24T06:09:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a14754700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c0ae710 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a14754700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c0af0b0 con 0x7f1a0c06c710 2023-11-24T06:09:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a11cef700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a0c06c710 0x7f1a0c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a124f0700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1a0c15e080 0x7f1a0c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38482/0 (socket says 172.21.15.97:38482) 2023-11-24T06:09:29.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a124f0700 1 -- 172.21.15.97:0/493492611 learned_addr learned my addr 172.21.15.97:0/493492611 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a11cef700 1 -- 172.21.15.97:0/493492611 >> v1:172.21.15.183:6789/0 conn(0x7f1a0c0af4d0 legacy=0x7f1a0c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a11cef700 1 -- 172.21.15.97:0/493492611 >> v1:172.21.15.97:6789/0 conn(0x7f1a0c15e080 legacy=0x7f1a0c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a11cef700 1 -- 172.21.15.97:0/493492611 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0ae9f0 con 0x7f1a0c06c710 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a114ee700 1 -- 172.21.15.97:0/493492611 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3130075304 0 0) 0x7f1a0c0ae430 con 0x7f1a0c15e080 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a11cef700 1 --2- 172.21.15.97:0/493492611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c06c710 0x7f1a0c0b5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009840 tx=0x7f19fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798e9e2a1e2b3cb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a114ee700 1 -- 172.21.15.97:0/493492611 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc004230 con 0x7f1a0c06c710 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a114ee700 1 -- 172.21.15.97:0/493492611 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc004390 con 0x7f1a0c06c710 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.001+0000 7f1a114ee700 1 -- 172.21.15.97:0/493492611 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0036a0 con 0x7f1a0c06c710 2023-11-24T06:09:29.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.002+0000 7f1a14754700 1 -- 172.21.15.97:0/493492611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c06c710 msgr2=0x7f1a0c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.002+0000 7f1a14754700 1 --2- 172.21.15.97:0/493492611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c06c710 0x7f1a0c0b5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009840 tx=0x7f19fc009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.002+0000 7f1a14754700 1 -- 172.21.15.97:0/493492611 shutdown_connections 2023-11-24T06:09:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.002+0000 7f1a14754700 1 --2- 172.21.15.97:0/493492611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c06c710 0x7f1a0c0b5050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.002+0000 7f1a14754700 1 -- 172.21.15.97:0/493492611 >> 172.21.15.97:0/493492611 conn(0x7f1a0c071b90 msgr2=0x7f1a0c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.002+0000 7f1a14754700 1 -- 172.21.15.97:0/493492611 shutdown_connections 2023-11-24T06:09:29.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.002+0000 7f1a14754700 1 -- 172.21.15.97:0/493492611 wait complete. 2023-11-24T06:09:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.003+0000 7f1a14754700 1 Processor -- start 2023-11-24T06:09:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.003+0000 7f1a14754700 1 -- start start 2023-11-24T06:09:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a14754700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c06c710 0x7f1a0c160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a14754700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c0af4d0 0x7f1a0c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a14754700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c15e080 0x7f1a0c1f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a14754700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c156380 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a14754700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c161460 con 0x7f1a0c06c710 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a14754700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c156570 con 0x7f1a0c15e080 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a11cef700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c0af4d0 0x7f1a0c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a11cef700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c0af4d0 0x7f1a0c153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46038/0 (socket says 172.21.15.97:46038) 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a11cef700 1 -- 172.21.15.97:0/3248399757 learned_addr learned my addr 172.21.15.97:0/3248399757 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a124f0700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c06c710 0x7f1a0c160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a12cf1700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c15e080 0x7f1a0c1f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a11cef700 1 -- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c15e080 msgr2=0x7f1a0c1f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a11cef700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c15e080 0x7f1a0c1f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a11cef700 1 -- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c06c710 msgr2=0x7f1a0c160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a11cef700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c06c710 0x7f1a0c160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a11cef700 1 -- 172.21.15.97:0/3248399757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19fc0094a0 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.004+0000 7f1a124f0700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c06c710 0x7f1a0c160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.005+0000 7f1a11cef700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c0af4d0 0x7f1a0c153ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0090b0 tx=0x7f19fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.005+0000 7f1a02ffd700 1 -- 172.21.15.97:0/3248399757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc011830 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.005+0000 7f1a02ffd700 1 -- 172.21.15.97:0/3248399757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc01e070 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.005+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c156850 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.005+0000 7f1a02ffd700 1 -- 172.21.15.97:0/3248399757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc011bb0 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.005+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c156e10 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.006+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c0753f0 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.006+0000 7f1a02ffd700 1 -- 172.21.15.97:0/3248399757 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f19fc011d10 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.007+0000 7f1a02ffd700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19f006aed0 0x7f19f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.007+0000 7f1a02ffd700 1 -- 172.21.15.97:0/3248399757 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19fc08a970 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.007+0000 7f1a124f0700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19f006aed0 0x7f19f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.007+0000 7f1a124f0700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19f006aed0 0x7f19f006d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1a08000f60 tx=0x7f1a08007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.011+0000 7f1a02ffd700 1 -- 172.21.15.97:0/3248399757 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19fc055ab0 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.202+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a0c067f60 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.203+0000 7f1a02ffd700 1 -- 172.21.15.97:0/3248399757 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f19fc020020 con 0x7f1a0c0af4d0 2023-11-24T06:09:29.206 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:29.206 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19f006aed0 msgr2=0x7f19f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19f006aed0 0x7f19f006d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1a08000f60 tx=0x7f1a08007d30 comp rx=0 tx=0).stop 2023-11-24T06:09:29.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c0af4d0 msgr2=0x7f1a0c153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c0af4d0 0x7f1a0c153ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0090b0 tx=0x7f19fc004750 comp rx=0 tx=0).stop 2023-11-24T06:09:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 shutdown_connections 2023-11-24T06:09:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19f006aed0 0x7f19f006d390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a0c15e080 0x7f1a0c1f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a0c06c710 0x7f1a0c160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 --2- 172.21.15.97:0/3248399757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a0c0af4d0 0x7f1a0c153ef0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 >> 172.21.15.97:0/3248399757 conn(0x7f1a0c071b90 msgr2=0x7f1a0c06bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 shutdown_connections 2023-11-24T06:09:29.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.205+0000 7f1a14754700 1 -- 172.21.15.97:0/3248399757 wait complete. 2023-11-24T06:09:29.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:29.221 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:29.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.341+0000 7fc6edba4700 1 Processor -- start 2023-11-24T06:09:29.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.341+0000 7fc6edba4700 1 -- start start 2023-11-24T06:09:29.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6edba4700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc6e815e160 0x7fc6e80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6edba4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80afc90 0x7fc6e8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6edba4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6e80744d0 0x7fc6e806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6edba4700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e80b4500 con 0x7fc6e80744d0 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6edba4700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e8074960 con 0x7fc6e815e160 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6edba4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80b3c60 con 0x7fc6e80afc90 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6e7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6e80744d0 0x7fc6e806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6e6ffd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80afc90 0x7fc6e8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6e77fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc6e815e160 0x7fc6e80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6e7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6e80744d0 0x7fc6e806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46048/0 (socket says 172.21.15.97:46048) 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.342+0000 7fc6e7fff700 1 -- 172.21.15.97:0/3611512446 learned_addr learned my addr 172.21.15.97:0/3611512446 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e7fff700 1 -- 172.21.15.97:0/3611512446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80afc90 msgr2=0x7fc6e8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:29.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e7fff700 1 --2- 172.21.15.97:0/3611512446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80afc90 0x7fc6e8073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e7fff700 1 -- 172.21.15.97:0/3611512446 >> v2:172.21.15.183:3300/0 conn(0x7fc6e815e160 msgr2=0x7fc6e80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e7fff700 1 --2- 172.21.15.97:0/3611512446 >> v2:172.21.15.183:3300/0 conn(0x7fc6e815e160 0x7fc6e80b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e7fff700 1 -- 172.21.15.97:0/3611512446 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80b58c0 con 0x7fc6e80744d0 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e7fff700 1 --2- 172.21.15.97:0/3611512446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80744d0 0x7fc6e806d930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc6e809efa0 tx=0x7fc6e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6e78b873b9158a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e67fc700 1 -- 172.21.15.97:0/3611512446 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000e070 con 0x7fc6e80744d0 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e67fc700 1 -- 172.21.15.97:0/3611512446 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e000b910 con 0x7fc6e80744d0 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6e67fc700 1 -- 172.21.15.97:0/3611512446 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e000bcc0 con 0x7fc6e80744d0 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6edba4700 1 -- 172.21.15.97:0/3611512446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80744d0 msgr2=0x7fc6e806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6edba4700 1 --2- 172.21.15.97:0/3611512446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80744d0 0x7fc6e806d930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc6e809efa0 tx=0x7fc6e000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6edba4700 1 -- 172.21.15.97:0/3611512446 shutdown_connections 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.343+0000 7fc6edba4700 1 --2- 172.21.15.97:0/3611512446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80744d0 0x7fc6e806d930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.344+0000 7fc6edba4700 1 --2- 172.21.15.97:0/3611512446 >> v2:172.21.15.183:3300/0 conn(0x7fc6e815e160 0x7fc6e80b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.344+0000 7fc6edba4700 1 --2- 172.21.15.97:0/3611512446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80afc90 0x7fc6e8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.344+0000 7fc6edba4700 1 -- 172.21.15.97:0/3611512446 >> 172.21.15.97:0/3611512446 conn(0x7fc6e80d0bf0 msgr2=0x7fc6e80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.344+0000 7fc6edba4700 1 -- 172.21.15.97:0/3611512446 shutdown_connections 2023-11-24T06:09:29.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.344+0000 7fc6edba4700 1 -- 172.21.15.97:0/3611512446 wait complete. 2023-11-24T06:09:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.345+0000 7fc6edba4700 1 Processor -- start 2023-11-24T06:09:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.345+0000 7fc6edba4700 1 -- start start 2023-11-24T06:09:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6edba4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80744d0 0x7fc6e814b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6edba4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80afc90 0x7fc6e8160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6edba4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6e815e160 0x7fc6e80b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6edba4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81566e0 con 0x7fc6e80afc90 2023-11-24T06:09:29.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6edba4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8153160 con 0x7fc6e815e160 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6edba4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8160cd0 con 0x7fc6e80744d0 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80afc90 0x7fc6e8160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80afc90 0x7fc6e8160360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46052/0 (socket says 172.21.15.97:46052) 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 -- 172.21.15.97:0/2323502764 learned_addr learned my addr 172.21.15.97:0/2323502764 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e77fe700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80744d0 0x7fc6e814b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e7fff700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6e815e160 0x7fc6e80b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 -- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80744d0 msgr2=0x7fc6e814b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80744d0 0x7fc6e814b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 -- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6e815e160 msgr2=0x7fc6e80b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6e815e160 0x7fc6e80b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 -- 172.21.15.97:0/2323502764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009b30 con 0x7fc6e80afc90 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e7fff700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6e815e160 0x7fc6e80b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:29.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.346+0000 7fc6e6ffd700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80afc90 0x7fc6e8160360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800e9e0 tx=0x7fc6d800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.347+0000 7fc6cffff700 1 -- 172.21.15.97:0/2323502764 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8009710 con 0x7fc6e80afc90 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.347+0000 7fc6cffff700 1 -- 172.21.15.97:0/2323502764 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d800c850 con 0x7fc6e80afc90 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.347+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e8160fb0 con 0x7fc6e80afc90 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.347+0000 7fc6cffff700 1 -- 172.21.15.97:0/2323502764 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d800cbc0 con 0x7fc6e80afc90 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.347+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e81533b0 con 0x7fc6e80afc90 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.348+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e80b3e70 con 0x7fc6e80afc90 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.348+0000 7fc6cffff700 1 -- 172.21.15.97:0/2323502764 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc6d800cd20 con 0x7fc6e80afc90 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.348+0000 7fc6cffff700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc6d406aed0 0x7fc6d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.348+0000 7fc6e77fe700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc6d406aed0 0x7fc6d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.348+0000 7fc6cffff700 1 -- 172.21.15.97:0/2323502764 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6d80896b0 con 0x7fc6e80afc90 2023-11-24T06:09:29.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.349+0000 7fc6e77fe700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc6d406aed0 0x7fc6d406d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009ac0 tx=0x7fc6dc009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.353+0000 7fc6cffff700 1 -- 172.21.15.97:0/2323502764 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6d80546d0 con 0x7fc6e80afc90 2023-11-24T06:09:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.544+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6e8068fb0 con 0x7fc6e80afc90 2023-11-24T06:09:29.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.544+0000 7fc6cffff700 1 -- 172.21.15.97:0/2323502764 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6d8057cf0 con 0x7fc6e80afc90 2023-11-24T06:09:29.547 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc6d406aed0 msgr2=0x7fc6d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc6d406aed0 0x7fc6d406d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009ac0 tx=0x7fc6dc009210 comp rx=0 tx=0).stop 2023-11-24T06:09:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80afc90 msgr2=0x7fc6e8160360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80afc90 0x7fc6e8160360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800e9e0 tx=0x7fc6d800ecf0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 shutdown_connections 2023-11-24T06:09:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6e815e160 0x7fc6e80b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc6d406aed0 0x7fc6d406d390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc6e80744d0 0x7fc6e814b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 --2- 172.21.15.97:0/2323502764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e80afc90 0x7fc6e8160360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 >> 172.21.15.97:0/2323502764 conn(0x7fc6e80d0bf0 msgr2=0x7fc6e806dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 shutdown_connections 2023-11-24T06:09:29.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.546+0000 7fc6edba4700 1 -- 172.21.15.97:0/2323502764 wait complete. 2023-11-24T06:09:29.561 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:29.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.682+0000 7f00430d6700 1 Processor -- start 2023-11-24T06:09:29.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.682+0000 7f00430d6700 1 -- start start 2023-11-24T06:09:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f00430d6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c0c6c30 0x7f003c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f00430d6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f003c0cef40 0x7f003c0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f00430d6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c0d0140 con 0x7f003c15bf50 2023-11-24T06:09:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f00430d6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003c15c330 con 0x7f003c0cef40 2023-11-24T06:09:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f00430d6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0cf3d0 con 0x7f003c0c6c30 2023-11-24T06:09:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f0040e72700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f003c15bf50 0x7f003c0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38510/0 (socket says 172.21.15.97:38510) 2023-11-24T06:09:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f0040e72700 1 -- 172.21.15.97:0/387474088 learned_addr learned my addr 172.21.15.97:0/387474088 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:29.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f0041673700 1 --2- 172.21.15.97:0/387474088 >> v2:172.21.15.97:3300/0 conn(0x7f003c0cef40 0x7f003c0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f003bfff700 1 --2- 172.21.15.97:0/387474088 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c0c6c30 0x7f003c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f0041673700 1 -- 172.21.15.97:0/387474088 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c0c6c30 msgr2=0x7f003c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f0041673700 1 --2- 172.21.15.97:0/387474088 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c0c6c30 0x7f003c0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f0041673700 1 -- 172.21.15.97:0/387474088 >> v1:172.21.15.97:6789/0 conn(0x7f003c15bf50 legacy=0x7f003c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f003b7fe700 1 -- 172.21.15.97:0/387474088 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380886759 0 0) 0x7f003c0d0140 con 0x7f003c15bf50 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f0041673700 1 -- 172.21.15.97:0/387474088 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0d03e0 con 0x7f003c0cef40 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f0041673700 1 --2- 172.21.15.97:0/387474088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c0cc6c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0034009d90 tx=0x7f003400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce13f371c791f9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f003b7fe700 1 -- 172.21.15.97:0/387474088 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003400e070 con 0x7f003c0cef40 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f003b7fe700 1 -- 172.21.15.97:0/387474088 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003400bbd0 con 0x7f003c0cef40 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.683+0000 7f003b7fe700 1 -- 172.21.15.97:0/387474088 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034012430 con 0x7f003c0cef40 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.684+0000 7f00430d6700 1 -- 172.21.15.97:0/387474088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 msgr2=0x7f003c0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.684+0000 7f00430d6700 1 --2- 172.21.15.97:0/387474088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c0cc6c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0034009d90 tx=0x7f003400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.684+0000 7f00430d6700 1 -- 172.21.15.97:0/387474088 shutdown_connections 2023-11-24T06:09:29.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.684+0000 7f00430d6700 1 --2- 172.21.15.97:0/387474088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c0cc6c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.684+0000 7f00430d6700 1 --2- 172.21.15.97:0/387474088 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c0c6c30 0x7f003c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.684+0000 7f00430d6700 1 -- 172.21.15.97:0/387474088 >> 172.21.15.97:0/387474088 conn(0x7f003c0b3780 msgr2=0x7f003c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.684+0000 7f00430d6700 1 -- 172.21.15.97:0/387474088 shutdown_connections 2023-11-24T06:09:29.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.684+0000 7f00430d6700 1 -- 172.21.15.97:0/387474088 wait complete. 2023-11-24T06:09:29.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.685+0000 7f00430d6700 1 Processor -- start 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.685+0000 7f00430d6700 1 -- start start 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f00430d6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003c0c6c30 0x7f003c1f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f00430d6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c1f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f00430d6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c15bf50 0x7f003c0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f00430d6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0ce770 con 0x7f003c0cef40 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f00430d6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1f3060 con 0x7f003c0c6c30 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f00430d6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1f3270 con 0x7f003c15bf50 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f003bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c1f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f003bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c1f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46066/0 (socket says 172.21.15.97:46066) 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f003bfff700 1 -- 172.21.15.97:0/4079773354 learned_addr learned my addr 172.21.15.97:0/4079773354 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f0040e72700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003c0c6c30 0x7f003c1f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f0041673700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c15bf50 0x7f003c0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f003bfff700 1 -- 172.21.15.97:0/4079773354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c15bf50 msgr2=0x7f003c0c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f003bfff700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c15bf50 0x7f003c0c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f003bfff700 1 -- 172.21.15.97:0/4079773354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003c0c6c30 msgr2=0x7f003c1f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f003bfff700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003c0c6c30 0x7f003c1f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.686+0000 7f003bfff700 1 -- 172.21.15.97:0/4079773354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034009a70 con 0x7f003c0cef40 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.687+0000 7f003bfff700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c1f4310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f002800e810 tx=0x7f002800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.687+0000 7f00397fa700 1 -- 172.21.15.97:0/4079773354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028013610 con 0x7f003c0cef40 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.687+0000 7f00397fa700 1 -- 172.21.15.97:0/4079773354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0028013770 con 0x7f003c0cef40 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.687+0000 7f00397fa700 1 -- 172.21.15.97:0/4079773354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028013b20 con 0x7f003c0cef40 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.687+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c1f3550 con 0x7f003c0cef40 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.687+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c1f3b10 con 0x7f003c0cef40 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.688+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c068fb0 con 0x7f003c0cef40 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.688+0000 7f00397fa700 1 -- 172.21.15.97:0/4079773354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0028013c80 con 0x7f003c0cef40 2023-11-24T06:09:29.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.689+0000 7f00397fa700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002006aed0 0x7f002006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:29.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.689+0000 7f00397fa700 1 -- 172.21.15.97:0/4079773354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0028089230 con 0x7f003c0cef40 2023-11-24T06:09:29.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.689+0000 7f0040e72700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002006aed0 0x7f002006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:29.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.689+0000 7f0040e72700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002006aed0 0x7f002006d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0030000f60 tx=0x7f0030007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:29.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.693+0000 7f00397fa700 1 -- 172.21.15.97:0/4079773354 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00280542f0 con 0x7f003c0cef40 2023-11-24T06:09:29.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.884+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f003c158a50 con 0x7f003c0cef40 2023-11-24T06:09:29.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.884+0000 7f00397fa700 1 -- 172.21.15.97:0/4079773354 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0028057910 con 0x7f003c0cef40 2023-11-24T06:09:29.887 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:29.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002006aed0 msgr2=0x7f002006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002006aed0 0x7f002006d390 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0030000f60 tx=0x7f0030007c90 comp rx=0 tx=0).stop 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 msgr2=0x7f003c1f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c1f4310 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f002800e810 tx=0x7f002800ebd0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 shutdown_connections 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002006aed0 0x7f002006d390 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f003c15bf50 0x7f003c0c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003c0c6c30 0x7f003c1f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 --2- 172.21.15.97:0/4079773354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003c0cef40 0x7f003c1f4310 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 >> 172.21.15.97:0/4079773354 conn(0x7f003c0b3780 msgr2=0x7f003c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 shutdown_connections 2023-11-24T06:09:29.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:29.886+0000 7f00430d6700 1 -- 172.21.15.97:0/4079773354 wait complete. 2023-11-24T06:09:29.901 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:30.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.021+0000 7f2173cf9700 1 Processor -- start 2023-11-24T06:09:30.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.021+0000 7f2173cf9700 1 -- start start 2023-11-24T06:09:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2173cf9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 0x7f216c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2173cf9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f216c0afc90 0x7f216c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2173cf9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0b3c60 con 0x7f216c0744d0 2023-11-24T06:09:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2173cf9700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0b4500 con 0x7f216c0afc90 2023-11-24T06:09:30.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2173cf9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c074960 con 0x7f216c15e160 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2171294700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f216c0afc90 0x7f216c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2172296700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f216c0744d0 0x7f216c06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38528/0 (socket says 172.21.15.97:38528) 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2172296700 1 -- 172.21.15.97:0/4233010607 learned_addr learned my addr 172.21.15.97:0/4233010607 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2171294700 1 -- 172.21.15.97:0/4233010607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 msgr2=0x7f216c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2171294700 1 --2- 172.21.15.97:0/4233010607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 0x7f216c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2171294700 1 -- 172.21.15.97:0/4233010607 >> v1:172.21.15.97:6789/0 conn(0x7f216c0744d0 legacy=0x7f216c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2171294700 1 -- 172.21.15.97:0/4233010607 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0b58c0 con 0x7f216c0afc90 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2171294700 1 --2- 172.21.15.97:0/4233010607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0afc90 0x7f216c073f90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f215c009840 tx=0x7f215c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb7362f0db36b1f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2170a93700 1 -- 172.21.15.97:0/4233010607 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011830 con 0x7f216c0afc90 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2170a93700 1 -- 172.21.15.97:0/4233010607 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c011990 con 0x7f216c0afc90 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.022+0000 7f2170a93700 1 -- 172.21.15.97:0/4233010607 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011d20 con 0x7f216c0afc90 2023-11-24T06:09:30.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.023+0000 7f2173cf9700 1 -- 172.21.15.97:0/4233010607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0afc90 msgr2=0x7f216c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.023+0000 7f2173cf9700 1 --2- 172.21.15.97:0/4233010607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0afc90 0x7f216c073f90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f215c009840 tx=0x7f215c009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.023+0000 7f2173cf9700 1 -- 172.21.15.97:0/4233010607 shutdown_connections 2023-11-24T06:09:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.023+0000 7f2173cf9700 1 --2- 172.21.15.97:0/4233010607 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 0x7f216c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.023+0000 7f2173cf9700 1 --2- 172.21.15.97:0/4233010607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0afc90 0x7f216c073f90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.023+0000 7f2173cf9700 1 -- 172.21.15.97:0/4233010607 >> 172.21.15.97:0/4233010607 conn(0x7f216c0d0bf0 msgr2=0x7f216c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.023+0000 7f2173cf9700 1 -- 172.21.15.97:0/4233010607 shutdown_connections 2023-11-24T06:09:30.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.023+0000 7f2173cf9700 1 -- 172.21.15.97:0/4233010607 wait complete. 2023-11-24T06:09:30.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.024+0000 7f2173cf9700 1 Processor -- start 2023-11-24T06:09:30.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.024+0000 7f2173cf9700 1 -- start start 2023-11-24T06:09:30.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2173cf9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0744d0 0x7f216c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2173cf9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f216c0afc90 0x7f216c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2173cf9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 0x7f216c15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2173cf9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c15f4a0 con 0x7f216c0744d0 2023-11-24T06:09:30.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2173cf9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c15e540 con 0x7f216c0afc90 2023-11-24T06:09:30.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2173cf9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c15e820 con 0x7f216c15e160 2023-11-24T06:09:30.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0744d0 0x7f216c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0744d0 0x7f216c154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46078/0 (socket says 172.21.15.97:46078) 2023-11-24T06:09:30.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 -- 172.21.15.97:0/2771139127 learned_addr learned my addr 172.21.15.97:0/2771139127 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:30.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2172296700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 0x7f216c15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171294700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f216c0afc90 0x7f216c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 -- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 msgr2=0x7f216c15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 0x7f216c15fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 -- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f216c0afc90 msgr2=0x7f216c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f216c0afc90 0x7f216c15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 -- 172.21.15.97:0/2771139127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c0094a0 con 0x7f216c0744d0 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171294700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f216c0afc90 0x7f216c15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2172296700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 0x7f216c15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.025+0000 7f2171a95700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0744d0 0x7f216c154c60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f216800b4a0 tx=0x7f216800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.026+0000 7f21627fc700 1 -- 172.21.15.97:0/2771139127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168011830 con 0x7f216c0744d0 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.026+0000 7f21627fc700 1 -- 172.21.15.97:0/2771139127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2168011990 con 0x7f216c0744d0 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.026+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c15eb00 con 0x7f216c0744d0 2023-11-24T06:09:30.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.026+0000 7f21627fc700 1 -- 172.21.15.97:0/2771139127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168011d40 con 0x7f216c0744d0 2023-11-24T06:09:30.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.026+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c15f0c0 con 0x7f216c0744d0 2023-11-24T06:09:30.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.027+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c15f710 con 0x7f216c0744d0 2023-11-24T06:09:30.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.027+0000 7f21627fc700 1 -- 172.21.15.97:0/2771139127 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f216801a400 con 0x7f216c0744d0 2023-11-24T06:09:30.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.027+0000 7f21627fc700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f215006aed0 0x7f215006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.028+0000 7f21627fc700 1 -- 172.21.15.97:0/2771139127 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2168089670 con 0x7f216c0744d0 2023-11-24T06:09:30.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.028+0000 7f2171294700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f215006aed0 0x7f215006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.028+0000 7f2171294700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f215006aed0 0x7f215006d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f215c000c00 tx=0x7f215c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.032+0000 7f21627fc700 1 -- 172.21.15.97:0/2771139127 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2168054690 con 0x7f216c0744d0 2023-11-24T06:09:30.223 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.220+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f216c068fb0 con 0x7f216c0744d0 2023-11-24T06:09:30.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.220+0000 7f21627fc700 1 -- 172.21.15.97:0/2771139127 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2168057cb0 con 0x7f216c0744d0 2023-11-24T06:09:30.225 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:30.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f215006aed0 msgr2=0x7f215006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f215006aed0 0x7f215006d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f215c000c00 tx=0x7f215c008030 comp rx=0 tx=0).stop 2023-11-24T06:09:30.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0744d0 msgr2=0x7f216c154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0744d0 0x7f216c154c60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f216800b4a0 tx=0x7f216800b860 comp rx=0 tx=0).stop 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 shutdown_connections 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f215006aed0 0x7f215006d390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f216c15e160 0x7f216c15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f216c0744d0 0x7f216c154c60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 --2- 172.21.15.97:0/2771139127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f216c0afc90 0x7f216c15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.222+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 >> 172.21.15.97:0/2771139127 conn(0x7f216c0d0bf0 msgr2=0x7f216c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.223+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 shutdown_connections 2023-11-24T06:09:30.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.223+0000 7f2173cf9700 1 -- 172.21.15.97:0/2771139127 wait complete. 2023-11-24T06:09:30.307 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:09:30.307 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:09:30.307 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:09:30.307 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:09:30.307 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:09:30.307 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:09:30.308 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:09:30.308 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:09:30.308 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:09:30.308 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T06:09:30.308 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T06:09:30.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:30.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.426+0000 7f391b516700 1 Processor -- start 2023-11-24T06:09:30.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.426+0000 7f391b516700 1 -- start start 2023-11-24T06:09:30.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f391b516700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39140af4d0 0x7f39140b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f391b516700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140ae430 con 0x7f391406c710 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f391b516700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140ae710 con 0x7f391415e080 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f391b516700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39140af0b0 con 0x7f39140af4d0 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f3919ab3700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39140af4d0 0x7f39140b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f3918ab1700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f391406c710 0x7f391406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38540/0 (socket says 172.21.15.97:38540) 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f3918ab1700 1 -- 172.21.15.97:0/2220788842 learned_addr learned my addr 172.21.15.97:0/2220788842 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f3919ab3700 1 -- 172.21.15.97:0/2220788842 >> v1:172.21.15.183:6789/0 conn(0x7f391415e080 legacy=0x7f391406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f3919ab3700 1 -- 172.21.15.97:0/2220788842 >> v1:172.21.15.97:6789/0 conn(0x7f391406c710 legacy=0x7f391406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f390bfff700 1 -- 172.21.15.97:0/2220788842 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745508073 0 0) 0x7f39140ae430 con 0x7f391406c710 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.427+0000 7f3919ab3700 1 -- 172.21.15.97:0/2220788842 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140ae9f0 con 0x7f39140af4d0 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f3919ab3700 1 --2- 172.21.15.97:0/2220788842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39140af4d0 0x7f39140b5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f390c009d90 tx=0x7f390c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b36b3d97bd8f8ff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f390bfff700 1 -- 172.21.15.97:0/2220788842 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c00e070 con 0x7f39140af4d0 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f390bfff700 1 -- 172.21.15.97:0/2220788842 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c00bbd0 con 0x7f39140af4d0 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f390bfff700 1 -- 172.21.15.97:0/2220788842 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c012430 con 0x7f39140af4d0 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f391b516700 1 -- 172.21.15.97:0/2220788842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39140af4d0 msgr2=0x7f39140b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f391b516700 1 --2- 172.21.15.97:0/2220788842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39140af4d0 0x7f39140b5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f390c009d90 tx=0x7f390c00b330 comp rx=0 tx=0).stop 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f391b516700 1 -- 172.21.15.97:0/2220788842 shutdown_connections 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f391b516700 1 --2- 172.21.15.97:0/2220788842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39140af4d0 0x7f39140b5050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f391b516700 1 -- 172.21.15.97:0/2220788842 >> 172.21.15.97:0/2220788842 conn(0x7f3914071b90 msgr2=0x7f3914071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f391b516700 1 -- 172.21.15.97:0/2220788842 shutdown_connections 2023-11-24T06:09:30.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.428+0000 7f391b516700 1 -- 172.21.15.97:0/2220788842 wait complete. 2023-11-24T06:09:30.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.429+0000 7f391b516700 1 Processor -- start 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f391b516700 1 -- start start 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f391b516700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391406c710 0x7f3914157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f391b516700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39140af4d0 0x7f3914149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f391b516700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f391415e080 0x7f39140ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f391b516700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39141589a0 con 0x7f391415e080 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f391b516700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3914159d70 con 0x7f391406c710 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f391b516700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3914157790 con 0x7f39140af4d0 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f3919ab3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f391415e080 0x7f39140ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f3919ab3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f391415e080 0x7f39140ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46096/0 (socket says 172.21.15.97:46096) 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.430+0000 7f3919ab3700 1 -- 172.21.15.97:0/4241997676 learned_addr learned my addr 172.21.15.97:0/4241997676 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f39192b2700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391406c710 0x7f3914157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3918ab1700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39140af4d0 0x7f3914149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3919ab3700 1 -- 172.21.15.97:0/4241997676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39140af4d0 msgr2=0x7f3914149480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3919ab3700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39140af4d0 0x7f3914149480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3919ab3700 1 -- 172.21.15.97:0/4241997676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391406c710 msgr2=0x7f3914157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3919ab3700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391406c710 0x7f3914157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3919ab3700 1 -- 172.21.15.97:0/4241997676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c009a70 con 0x7f391415e080 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3919ab3700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f391415e080 0x7f39140ceb90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f390c014040 tx=0x7f390c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3909ffb700 1 -- 172.21.15.97:0/4241997676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c01b810 con 0x7f391415e080 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3909ffb700 1 -- 172.21.15.97:0/4241997676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c020070 con 0x7f391415e080 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391415a560 con 0x7f391415e080 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f3909ffb700 1 -- 172.21.15.97:0/4241997676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c01bbb0 con 0x7f391415e080 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.431+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391415a280 con 0x7f391415e080 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.432+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39140753f0 con 0x7f391415e080 2023-11-24T06:09:30.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.433+0000 7f3909ffb700 1 -- 172.21.15.97:0/4241997676 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f390c01bd10 con 0x7f391415e080 2023-11-24T06:09:30.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.433+0000 7f3909ffb700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f38f806aed0 0x7f38f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.433+0000 7f39192b2700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f38f806aed0 0x7f38f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.433+0000 7f3909ffb700 1 -- 172.21.15.97:0/4241997676 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f390c022070 con 0x7f391415e080 2023-11-24T06:09:30.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.434+0000 7f39192b2700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f38f806aed0 0x7f38f806d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3904000f60 tx=0x7f3904009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.438+0000 7f3909ffb700 1 -- 172.21.15.97:0/4241997676 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f390c05a4e0 con 0x7f391415e080 2023-11-24T06:09:30.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.634+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3914067f60 con 0x7f391415e080 2023-11-24T06:09:30.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.635+0000 7f3909ffb700 1 -- 172.21.15.97:0/4241997676 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f390c05a070 con 0x7f391415e080 2023-11-24T06:09:30.638 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:30.638 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f38f806aed0 msgr2=0x7f38f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f38f806aed0 0x7f38f806d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3904000f60 tx=0x7f3904009040 comp rx=0 tx=0).stop 2023-11-24T06:09:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f391415e080 msgr2=0x7f39140ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f391415e080 0x7f39140ceb90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f390c014040 tx=0x7f390c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 shutdown_connections 2023-11-24T06:09:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f391415e080 0x7f39140ceb90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f391406c710 0x7f3914157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f38f806aed0 0x7f38f806d390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 --2- 172.21.15.97:0/4241997676 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39140af4d0 0x7f3914149480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 >> 172.21.15.97:0/4241997676 conn(0x7f3914071b90 msgr2=0x7f39140b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 shutdown_connections 2023-11-24T06:09:30.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.637+0000 7f391b516700 1 -- 172.21.15.97:0/4241997676 wait complete. 2023-11-24T06:09:30.652 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:30.653 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.774+0000 7f8be205c700 1 Processor -- start 2023-11-24T06:09:30.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.774+0000 7f8be205c700 1 -- start start 2023-11-24T06:09:30.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8be205c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8bdc15e320 0x7f8bdc0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8be205c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bdc0b6fb0 con 0x7f8bdc06b700 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8be205c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bdc0d2940 con 0x7f8bdc0b93c0 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8be205c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc0d4da0 con 0x7f8bdc15e320 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8bdb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8bdc15e320 0x7f8bdc0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8bdbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8bdc06b700 0x7f8bdc167d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38556/0 (socket says 172.21.15.97:38556) 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8bdbfff700 1 -- 172.21.15.97:0/3154983513 learned_addr learned my addr 172.21.15.97:0/3154983513 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8bdb7fe700 1 -- 172.21.15.97:0/3154983513 >> v1:172.21.15.183:6789/0 conn(0x7f8bdc0b93c0 legacy=0x7f8bdc06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8bdb7fe700 1 -- 172.21.15.97:0/3154983513 >> v1:172.21.15.97:6789/0 conn(0x7f8bdc06b700 legacy=0x7f8bdc167d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8bdb7fe700 1 -- 172.21.15.97:0/3154983513 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc0d2c10 con 0x7f8bdc15e320 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.775+0000 7f8bda7fc700 1 -- 172.21.15.97:0/3154983513 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571144147 0 0) 0x7f8bdc0b6fb0 con 0x7f8bdc06b700 2023-11-24T06:09:30.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8bdb7fe700 1 --2- 172.21.15.97:0/3154983513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0d2400 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009840 tx=0x7f8bc4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fefde1b5a16b37 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8bda7fc700 1 -- 172.21.15.97:0/3154983513 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4011830 con 0x7f8bdc15e320 2023-11-24T06:09:30.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8bda7fc700 1 -- 172.21.15.97:0/3154983513 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc4011990 con 0x7f8bdc15e320 2023-11-24T06:09:30.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8bda7fc700 1 -- 172.21.15.97:0/3154983513 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4011d20 con 0x7f8bdc15e320 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8be205c700 1 -- 172.21.15.97:0/3154983513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 msgr2=0x7f8bdc0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8be205c700 1 --2- 172.21.15.97:0/3154983513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0d2400 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009840 tx=0x7f8bc4009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8be205c700 1 -- 172.21.15.97:0/3154983513 shutdown_connections 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8be205c700 1 --2- 172.21.15.97:0/3154983513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0d2400 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8be205c700 1 -- 172.21.15.97:0/3154983513 >> 172.21.15.97:0/3154983513 conn(0x7f8bdc0d3e50 msgr2=0x7f8bdc0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8be205c700 1 -- 172.21.15.97:0/3154983513 shutdown_connections 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.776+0000 7f8be205c700 1 -- 172.21.15.97:0/3154983513 wait complete. 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.778+0000 7f8be205c700 1 Processor -- start 2023-11-24T06:09:30.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.778+0000 7f8be205c700 1 -- start start 2023-11-24T06:09:30.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8be205c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bdc06b700 0x7f8bdc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8be205c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bdc0b93c0 0x7f8bdc0cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8be205c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8be205c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc14dbe0 con 0x7f8bdc15e320 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8be205c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc14dec0 con 0x7f8bdc06b700 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8be205c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc14e1a0 con 0x7f8bdc0b93c0 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46110/0 (socket says 172.21.15.97:46110) 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdbfff700 1 -- 172.21.15.97:0/2968698414 learned_addr learned my addr 172.21.15.97:0/2968698414 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdb7fe700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bdc06b700 0x7f8bdc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdaffd700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bdc0b93c0 0x7f8bdc0cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdbfff700 1 -- 172.21.15.97:0/2968698414 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bdc0b93c0 msgr2=0x7f8bdc0cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdbfff700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bdc0b93c0 0x7f8bdc0cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdbfff700 1 -- 172.21.15.97:0/2968698414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bdc06b700 msgr2=0x7f8bdc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdbfff700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bdc06b700 0x7f8bdc0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.779+0000 7f8bdbfff700 1 -- 172.21.15.97:0/2968698414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40094a0 con 0x7f8bdc15e320 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.780+0000 7f8bdbfff700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0ca2b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400cc10 tx=0x7f8bd400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.780+0000 7f8bd3fff700 1 -- 172.21.15.97:0/2968698414 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400aaa0 con 0x7f8bdc15e320 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.780+0000 7f8bd3fff700 1 -- 172.21.15.97:0/2968698414 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd400ac20 con 0x7f8bdc15e320 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.780+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc14e480 con 0x7f8bdc15e320 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.780+0000 7f8bd3fff700 1 -- 172.21.15.97:0/2968698414 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40070d0 con 0x7f8bdc15e320 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.780+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc0cbe10 con 0x7f8bdc15e320 2023-11-24T06:09:30.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.781+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc068fb0 con 0x7f8bdc15e320 2023-11-24T06:09:30.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.781+0000 7f8bd3fff700 1 -- 172.21.15.97:0/2968698414 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8bd4007230 con 0x7f8bdc15e320 2023-11-24T06:09:30.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.781+0000 7f8bd3fff700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8bbc06aed0 0x7f8bbc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:30.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.782+0000 7f8bd3fff700 1 -- 172.21.15.97:0/2968698414 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bd4089080 con 0x7f8bdc15e320 2023-11-24T06:09:30.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.782+0000 7f8bdb7fe700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8bbc06aed0 0x7f8bbc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:30.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.782+0000 7f8bdb7fe700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8bbc06aed0 0x7f8bbc06d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40090b0 tx=0x7f8bc401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:30.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.786+0000 7f8bd3fff700 1 -- 172.21.15.97:0/2968698414 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8bd40540b0 con 0x7f8bdc15e320 2023-11-24T06:09:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.974+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bdc0b6c80 con 0x7f8bdc15e320 2023-11-24T06:09:30.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.974+0000 7f8bd3fff700 1 -- 172.21.15.97:0/2968698414 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bd40576d0 con 0x7f8bdc15e320 2023-11-24T06:09:30.977 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:30.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8bbc06aed0 msgr2=0x7f8bbc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8bbc06aed0 0x7f8bbc06d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40090b0 tx=0x7f8bc401c040 comp rx=0 tx=0).stop 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 msgr2=0x7f8bdc0ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0ca2b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400cc10 tx=0x7f8bd400cf20 comp rx=0 tx=0).stop 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 shutdown_connections 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bdc15e320 0x7f8bdc0ca2b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bdc06b700 0x7f8bdc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8bbc06aed0 0x7f8bbc06d390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 --2- 172.21.15.97:0/2968698414 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bdc0b93c0 0x7f8bdc0cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 >> 172.21.15.97:0/2968698414 conn(0x7f8bdc0d3e50 msgr2=0x7f8bdc0b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.976+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 shutdown_connections 2023-11-24T06:09:30.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:30.977+0000 7f8be205c700 1 -- 172.21.15.97:0/2968698414 wait complete. 2023-11-24T06:09:30.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:31.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.111+0000 7fa425c9c700 1 Processor -- start 2023-11-24T06:09:31.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.111+0000 7fa425c9c700 1 -- start start 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.112+0000 7fa425c9c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 0x7fa4200b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.112+0000 7fa425c9c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4200afc90 0x7fa42006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.112+0000 7fa425c9c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200b3c60 con 0x7fa42015e160 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.112+0000 7fa425c9c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4200b4500 con 0x7fa4200afc90 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.112+0000 7fa425c9c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200b58c0 con 0x7fa4200746a0 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.112+0000 7fa41ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4200afc90 0x7fa42006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.112+0000 7fa41ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa4200afc90 0x7fa42006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46118/0 (socket says 172.21.15.97:46118) 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.112+0000 7fa41ffff700 1 -- 172.21.15.97:0/2671318303 learned_addr learned my addr 172.21.15.97:0/2671318303 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41effd700 1 --2- 172.21.15.97:0/2671318303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 0x7fa4200b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41ffff700 1 -- 172.21.15.97:0/2671318303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 msgr2=0x7fa4200b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:31.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41ffff700 1 --2- 172.21.15.97:0/2671318303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 0x7fa4200b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41ffff700 1 -- 172.21.15.97:0/2671318303 >> v1:172.21.15.183:6789/0 conn(0x7fa42015e160 legacy=0x7fa420073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41ffff700 1 -- 172.21.15.97:0/2671318303 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4200b0140 con 0x7fa4200afc90 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41effd700 1 --2- 172.21.15.97:0/2671318303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 0x7fa4200b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41ffff700 1 --2- 172.21.15.97:0/2671318303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa42006d930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4100098f0 tx=0x7fa410009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefbf4c4c318b31a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41e7fc700 1 -- 172.21.15.97:0/2671318303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41000b8a0 con 0x7fa4200afc90 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41e7fc700 1 -- 172.21.15.97:0/2671318303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41000ba00 con 0x7fa4200afc90 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa41e7fc700 1 -- 172.21.15.97:0/2671318303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41000bd90 con 0x7fa4200afc90 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa425c9c700 1 -- 172.21.15.97:0/2671318303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 msgr2=0x7fa42006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.113+0000 7fa425c9c700 1 --2- 172.21.15.97:0/2671318303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa42006d930 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4100098f0 tx=0x7fa410009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.114+0000 7fa425c9c700 1 -- 172.21.15.97:0/2671318303 shutdown_connections 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.114+0000 7fa425c9c700 1 --2- 172.21.15.97:0/2671318303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa42006d930 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.114+0000 7fa425c9c700 1 --2- 172.21.15.97:0/2671318303 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 0x7fa4200b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.114+0000 7fa425c9c700 1 -- 172.21.15.97:0/2671318303 >> 172.21.15.97:0/2671318303 conn(0x7fa4200d0bf0 msgr2=0x7fa4200d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:31.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.114+0000 7fa425c9c700 1 -- 172.21.15.97:0/2671318303 shutdown_connections 2023-11-24T06:09:31.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.114+0000 7fa425c9c700 1 -- 172.21.15.97:0/2671318303 wait complete. 2023-11-24T06:09:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.115+0000 7fa425c9c700 1 Processor -- start 2023-11-24T06:09:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.115+0000 7fa425c9c700 1 -- start start 2023-11-24T06:09:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa425c9c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 0x7fa4200d0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa425c9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa4201f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa425c9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa42015e160 0x7fa42015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa425c9c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200c5d70 con 0x7fa4200afc90 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa425c9c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200c6050 con 0x7fa42015e160 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa425c9c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200c6330 con 0x7fa4200746a0 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa4201f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa4201f7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46122/0 (socket says 172.21.15.97:46122) 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 -- 172.21.15.97:0/4165199768 learned_addr learned my addr 172.21.15.97:0/4165199768 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41ffff700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa42015e160 0x7fa42015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 -- 172.21.15.97:0/4165199768 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 msgr2=0x7fa4200d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 0x7fa4200d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 -- 172.21.15.97:0/4165199768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa42015e160 msgr2=0x7fa42015bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa42015e160 0x7fa42015bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 -- 172.21.15.97:0/4165199768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa410009550 con 0x7fa4200afc90 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41ffff700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa42015e160 0x7fa42015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa41effd700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa4201f7760 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa41800e810 tx=0x7fa41800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa417fff700 1 -- 172.21.15.97:0/4165199768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418013610 con 0x7fa4200afc90 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa417fff700 1 -- 172.21.15.97:0/4165199768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa418013770 con 0x7fa4200afc90 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4200c6610 con 0x7fa4200afc90 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa417fff700 1 -- 172.21.15.97:0/4165199768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418013b20 con 0x7fa4200afc90 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.116+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4200d0560 con 0x7fa4200afc90 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.117+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa420068fb0 con 0x7fa4200afc90 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.118+0000 7fa417fff700 1 -- 172.21.15.97:0/4165199768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa418013c80 con 0x7fa4200afc90 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.118+0000 7fa417fff700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa40006aed0 0x7fa40006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.118+0000 7fa417fff700 1 -- 172.21.15.97:0/4165199768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa418089220 con 0x7fa4200afc90 2023-11-24T06:09:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.118+0000 7fa41f7fe700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa40006aed0 0x7fa40006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.118+0000 7fa41f7fe700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa40006aed0 0x7fa40006d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa4080012e0 tx=0x7fa4080016e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.122+0000 7fa417fff700 1 -- 172.21.15.97:0/4165199768 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa418054360 con 0x7fa4200afc90 2023-11-24T06:09:31.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.311+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4200c6b30 con 0x7fa4200afc90 2023-11-24T06:09:31.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.311+0000 7fa417fff700 1 -- 172.21.15.97:0/4165199768 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa4200c6b30 con 0x7fa4200afc90 2023-11-24T06:09:31.315 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa40006aed0 msgr2=0x7fa40006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa40006aed0 0x7fa40006d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa4080012e0 tx=0x7fa4080016e0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 msgr2=0x7fa4201f7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa4201f7760 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa41800e810 tx=0x7fa41800ebd0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 shutdown_connections 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa42015e160 0x7fa42015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa40006aed0 0x7fa40006d390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa4200746a0 0x7fa4200d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 --2- 172.21.15.97:0/4165199768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4200afc90 0x7fa4201f7760 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 >> 172.21.15.97:0/4165199768 conn(0x7fa4200d0bf0 msgr2=0x7fa42006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:31.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 shutdown_connections 2023-11-24T06:09:31.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.314+0000 7fa425c9c700 1 -- 172.21.15.97:0/4165199768 wait complete. 2023-11-24T06:09:31.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:31.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.449+0000 7f8952685700 1 Processor -- start 2023-11-24T06:09:31.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.449+0000 7f8952685700 1 -- start start 2023-11-24T06:09:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.449+0000 7f8952685700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f894c15e160 0x7f894c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.449+0000 7f8952685700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894c0b3c60 con 0x7f894c0afc90 2023-11-24T06:09:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.449+0000 7f8952685700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894c0b58c0 con 0x7f894c0746a0 2023-11-24T06:09:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.449+0000 7f8952685700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894c0b4500 con 0x7f894c15e160 2023-11-24T06:09:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f894c15e160 0x7f894c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f894c0afc90 0x7f894c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38580/0 (socket says 172.21.15.97:38580) 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894b7fe700 1 -- 172.21.15.97:0/1977624318 learned_addr learned my addr 172.21.15.97:0/1977624318 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894affd700 1 -- 172.21.15.97:0/1977624318 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353180287 0 0) 0x7f894c0b3c60 con 0x7f894c0afc90 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894affd700 1 -- 172.21.15.97:0/1977624318 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89340035b0 con 0x7f894c0afc90 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894bfff700 1 -- 172.21.15.97:0/1977624318 >> v1:172.21.15.183:6789/0 conn(0x7f894c0746a0 legacy=0x7f894c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894bfff700 1 -- 172.21.15.97:0/1977624318 >> v1:172.21.15.97:6789/0 conn(0x7f894c0afc90 legacy=0x7f894c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894bfff700 1 -- 172.21.15.97:0/1977624318 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c0b0140 con 0x7f894c15e160 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894bfff700 1 --2- 172.21.15.97:0/1977624318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c15e160 0x7f894c0b5da0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f893c009840 tx=0x7f893c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d73390cf9820e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894affd700 1 -- 172.21.15.97:0/1977624318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c011830 con 0x7f894c15e160 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894affd700 1 -- 172.21.15.97:0/1977624318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893c011990 con 0x7f894c15e160 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.450+0000 7f894affd700 1 -- 172.21.15.97:0/1977624318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c011d20 con 0x7f894c15e160 2023-11-24T06:09:31.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.451+0000 7f8952685700 1 -- 172.21.15.97:0/1977624318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c15e160 msgr2=0x7f894c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.451+0000 7f8952685700 1 --2- 172.21.15.97:0/1977624318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c15e160 0x7f894c0b5da0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f893c009840 tx=0x7f893c009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.451+0000 7f8952685700 1 -- 172.21.15.97:0/1977624318 shutdown_connections 2023-11-24T06:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.451+0000 7f8952685700 1 --2- 172.21.15.97:0/1977624318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c15e160 0x7f894c0b5da0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.451+0000 7f8952685700 1 -- 172.21.15.97:0/1977624318 >> 172.21.15.97:0/1977624318 conn(0x7f894c0d0bf0 msgr2=0x7f894c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.451+0000 7f8952685700 1 -- 172.21.15.97:0/1977624318 shutdown_connections 2023-11-24T06:09:31.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.451+0000 7f8952685700 1 -- 172.21.15.97:0/1977624318 wait complete. 2023-11-24T06:09:31.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.452+0000 7f8952685700 1 Processor -- start 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.452+0000 7f8952685700 1 -- start start 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8952685700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c0746a0 0x7f894c0c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8952685700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c0afc90 0x7f894c06eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8952685700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15e160 0x7f894c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8952685700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0c65b0 con 0x7f894c0afc90 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8952685700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0c6890 con 0x7f894c15e160 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8952685700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c0c6b70 con 0x7f894c0746a0 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c0afc90 0x7f894c06eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c0afc90 0x7f894c06eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46134/0 (socket says 172.21.15.97:46134) 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 -- 172.21.15.97:0/3160907397 learned_addr learned my addr 172.21.15.97:0/3160907397 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:31.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8950c22700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15e160 0x7f894c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894bfff700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c0746a0 0x7f894c0c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 -- 172.21.15.97:0/3160907397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c0746a0 msgr2=0x7f894c0c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c0746a0 0x7f894c0c7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 -- 172.21.15.97:0/3160907397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15e160 msgr2=0x7f894c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15e160 0x7f894c0c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 -- 172.21.15.97:0/3160907397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c0094a0 con 0x7f894c0afc90 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f894b7fe700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c0afc90 0x7f894c06eb70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f894000c700 tx=0x7f894000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8948ff9700 1 -- 172.21.15.97:0/3160907397 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940005d70 con 0x7f894c0afc90 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8948ff9700 1 -- 172.21.15.97:0/3160907397 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89400070d0 con 0x7f894c0afc90 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.453+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c0c6e50 con 0x7f894c0afc90 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.454+0000 7f8948ff9700 1 -- 172.21.15.97:0/3160907397 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940007460 con 0x7f894c0afc90 2023-11-24T06:09:31.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.454+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c06f0b0 con 0x7f894c0afc90 2023-11-24T06:09:31.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.454+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894c068fb0 con 0x7f894c0afc90 2023-11-24T06:09:31.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.455+0000 7f8948ff9700 1 -- 172.21.15.97:0/3160907397 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89400075c0 con 0x7f894c0afc90 2023-11-24T06:09:31.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.455+0000 7f8948ff9700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f893806aed0 0x7f893806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.455+0000 7f8948ff9700 1 -- 172.21.15.97:0/3160907397 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8940088bb0 con 0x7f894c0afc90 2023-11-24T06:09:31.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.455+0000 7f894bfff700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f893806aed0 0x7f893806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.456+0000 7f894bfff700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f893806aed0 0x7f893806d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f893c0090b0 tx=0x7f893c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.460+0000 7f8948ff9700 1 -- 172.21.15.97:0/3160907397 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8940053c70 con 0x7f894c0afc90 2023-11-24T06:09:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.649+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f894c06c140 con 0x7f894c0afc90 2023-11-24T06:09:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.650+0000 7f8948ff9700 1 -- 172.21.15.97:0/3160907397 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8940057290 con 0x7f894c0afc90 2023-11-24T06:09:31.653 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f893806aed0 msgr2=0x7f893806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f893806aed0 0x7f893806d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f893c0090b0 tx=0x7f893c008030 comp rx=0 tx=0).stop 2023-11-24T06:09:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c0afc90 msgr2=0x7f894c06eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c0afc90 0x7f894c06eb70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f894000c700 tx=0x7f894000cac0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 shutdown_connections 2023-11-24T06:09:31.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f894c15e160 0x7f894c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f893806aed0 0x7f893806d390 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f894c0746a0 0x7f894c0c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 --2- 172.21.15.97:0/3160907397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f894c0afc90 0x7f894c06eb70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 >> 172.21.15.97:0/3160907397 conn(0x7f894c0d0bf0 msgr2=0x7f894c0ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:31.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 shutdown_connections 2023-11-24T06:09:31.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.652+0000 7f8952685700 1 -- 172.21.15.97:0/3160907397 wait complete. 2023-11-24T06:09:31.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T06:09:31.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.786+0000 7f5e86a3e700 1 Processor -- start 2023-11-24T06:09:31.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.786+0000 7f5e86a3e700 1 -- start start 2023-11-24T06:09:31.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e86a3e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e8015c140 0x7f5e800726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e86a3e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e800622a0 0x7f5e80072c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e86a3e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e800adfd0 con 0x7f5e800af070 2023-11-24T06:09:31.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e86a3e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e800afbf0 con 0x7f5e800622a0 2023-11-24T06:09:31.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e86a3e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e800ae2b0 con 0x7f5e8015c140 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e7f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e800622a0 0x7f5e80072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e84fdb700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5e800af070 0x7f5e800af4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38596/0 (socket says 172.21.15.97:38596) 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e84fdb700 1 -- 172.21.15.97:0/740824130 learned_addr learned my addr 172.21.15.97:0/740824130 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e7f7fe700 1 -- 172.21.15.97:0/740824130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e8015c140 msgr2=0x7f5e800726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e7f7fe700 1 --2- 172.21.15.97:0/740824130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e8015c140 0x7f5e800726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e7f7fe700 1 -- 172.21.15.97:0/740824130 >> v1:172.21.15.97:6789/0 conn(0x7f5e800af070 legacy=0x7f5e800af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e7f7fe700 1 -- 172.21.15.97:0/740824130 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e800ae590 con 0x7f5e800622a0 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.787+0000 7f5e7effd700 1 -- 172.21.15.97:0/740824130 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682344357 0 0) 0x7f5e800adfd0 con 0x7f5e800af070 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e7f7fe700 1 --2- 172.21.15.97:0/740824130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800622a0 0x7f5e80072c20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009840 tx=0x7f5e70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cf88eb9f101c63 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e7effd700 1 -- 172.21.15.97:0/740824130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70011830 con 0x7f5e800622a0 2023-11-24T06:09:31.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e7effd700 1 -- 172.21.15.97:0/740824130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e70011990 con 0x7f5e800622a0 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e7effd700 1 -- 172.21.15.97:0/740824130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70011d20 con 0x7f5e800622a0 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e86a3e700 1 -- 172.21.15.97:0/740824130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800622a0 msgr2=0x7f5e80072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/740824130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800622a0 0x7f5e80072c20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5e70009840 tx=0x7f5e70009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e86a3e700 1 -- 172.21.15.97:0/740824130 shutdown_connections 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/740824130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e8015c140 0x7f5e800726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/740824130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800622a0 0x7f5e80072c20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e86a3e700 1 -- 172.21.15.97:0/740824130 >> 172.21.15.97:0/740824130 conn(0x7f5e800b57b0 msgr2=0x7f5e800b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e86a3e700 1 -- 172.21.15.97:0/740824130 shutdown_connections 2023-11-24T06:09:31.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.788+0000 7f5e86a3e700 1 -- 172.21.15.97:0/740824130 wait complete. 2023-11-24T06:09:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.790+0000 7f5e86a3e700 1 Processor -- start 2023-11-24T06:09:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.790+0000 7f5e86a3e700 1 -- start start 2023-11-24T06:09:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e86a3e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e800622a0 0x7f5e801583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e86a3e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800af070 0x7f5e801f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e86a3e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e8015c140 0x7f5e80159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e86a3e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e801540d0 con 0x7f5e800af070 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e86a3e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e801543b0 con 0x7f5e8015c140 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e86a3e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80154690 con 0x7f5e800622a0 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800af070 0x7f5e801f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800af070 0x7f5e801f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46148/0 (socket says 172.21.15.97:46148) 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7f7fe700 1 -- 172.21.15.97:0/3469551877 learned_addr learned my addr 172.21.15.97:0/3469551877 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7ffff700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e800622a0 0x7f5e801583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e84fdb700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e8015c140 0x7f5e80159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7f7fe700 1 -- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e800622a0 msgr2=0x7f5e801583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7f7fe700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e800622a0 0x7f5e801583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7f7fe700 1 -- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e8015c140 msgr2=0x7f5e80159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7f7fe700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e8015c140 0x7f5e80159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7f7fe700 1 -- 172.21.15.97:0/3469551877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e700094a0 con 0x7f5e800af070 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e84fdb700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e8015c140 0x7f5e80159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.791+0000 7f5e7ffff700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e800622a0 0x7f5e801583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.792+0000 7f5e7f7fe700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800af070 0x7f5e801f48d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e700090b0 tx=0x7f5e70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.792+0000 7f5e7cff9700 1 -- 172.21.15.97:0/3469551877 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e700045f0 con 0x7f5e800af070 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.792+0000 7f5e7cff9700 1 -- 172.21.15.97:0/3469551877 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7001f070 con 0x7f5e800af070 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.792+0000 7f5e7cff9700 1 -- 172.21.15.97:0/3469551877 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70003b00 con 0x7f5e800af070 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.792+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e80154970 con 0x7f5e800af070 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.792+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8006d860 con 0x7f5e800af070 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.793+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8006d9f0 con 0x7f5e800af070 2023-11-24T06:09:31.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.793+0000 7f5e7cff9700 1 -- 172.21.15.97:0/3469551877 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e70003c60 con 0x7f5e800af070 2023-11-24T06:09:31.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.794+0000 7f5e7cff9700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e6c06aed0 0x7f5e6c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:31.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.794+0000 7f5e7ffff700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e6c06aed0 0x7f5e6c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:31.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.794+0000 7f5e7cff9700 1 -- 172.21.15.97:0/3469551877 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e70021070 con 0x7f5e800af070 2023-11-24T06:09:31.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.794+0000 7f5e7ffff700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e6c06aed0 0x7f5e6c06d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5e800b6250 tx=0x7f5e74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:31.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:31.799+0000 7f5e7cff9700 1 -- 172.21.15.97:0/3469551877 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e700561b0 con 0x7f5e800af070 2023-11-24T06:09:32.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.066+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f5e80067f60 con 0x7f5e800af070 2023-11-24T06:09:32.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.730+0000 7f5e7cff9700 1 -- 172.21.15.97:0/3469551877 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v53) v1 ==== 184+0+0 (secure 0 0 0) 0x7f5e70006cc0 con 0x7f5e800af070 2023-11-24T06:09:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e6c06aed0 msgr2=0x7f5e6c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e6c06aed0 0x7f5e6c06d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5e800b6250 tx=0x7f5e74008040 comp rx=0 tx=0).stop 2023-11-24T06:09:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800af070 msgr2=0x7f5e801f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800af070 0x7f5e801f48d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e700090b0 tx=0x7f5e70004750 comp rx=0 tx=0).stop 2023-11-24T06:09:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 shutdown_connections 2023-11-24T06:09:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e8015c140 0x7f5e80159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e6c06aed0 0x7f5e6c06d390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e800622a0 0x7f5e801583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 --2- 172.21.15.97:0/3469551877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e800af070 0x7f5e801f48d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 >> 172.21.15.97:0/3469551877 conn(0x7f5e800b57b0 msgr2=0x7f5e800726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.732+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 shutdown_connections 2023-11-24T06:09:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.733+0000 7f5e86a3e700 1 -- 172.21.15.97:0/3469551877 wait complete. 2023-11-24T06:09:32.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:09:32.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.875+0000 7f6f4a77b700 1 Processor -- start 2023-11-24T06:09:32.878 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.875+0000 7f6f4a77b700 1 -- start start 2023-11-24T06:09:32.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.875+0000 7f6f4a77b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6f4415e160 0x7f6f440b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:32.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.875+0000 7f6f4a77b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 0x7f6f4406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:32.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.875+0000 7f6f4a77b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f440b3c60 con 0x7f6f440afc90 2023-11-24T06:09:32.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.875+0000 7f6f4a77b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f440b4500 con 0x7f6f4415e160 2023-11-24T06:09:32.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.875+0000 7f6f4a77b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f440b58c0 con 0x7f6f440746a0 2023-11-24T06:09:32.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f43fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6f4415e160 0x7f6f440b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:32.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f437fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6f440afc90 0x7f6f44073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38608/0 (socket says 172.21.15.97:38608) 2023-11-24T06:09:32.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f437fe700 1 -- 172.21.15.97:0/954824868 learned_addr learned my addr 172.21.15.97:0/954824868 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:32.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f43fff700 1 -- 172.21.15.97:0/954824868 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 msgr2=0x7f6f4406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:32.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f43fff700 1 --2- 172.21.15.97:0/954824868 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 0x7f6f4406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f43fff700 1 -- 172.21.15.97:0/954824868 >> v1:172.21.15.97:6789/0 conn(0x7f6f440afc90 legacy=0x7f6f44073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:32.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f42ffd700 1 -- 172.21.15.97:0/954824868 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153298119 0 0) 0x7f6f440b3c60 con 0x7f6f440afc90 2023-11-24T06:09:32.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f43fff700 1 -- 172.21.15.97:0/954824868 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f440b0140 con 0x7f6f4415e160 2023-11-24T06:09:32.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f43fff700 1 --2- 172.21.15.97:0/954824868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415e160 0x7f6f440b5da0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009840 tx=0x7f6f34009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a12817e61b48a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:32.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f42ffd700 1 -- 172.21.15.97:0/954824868 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34011830 con 0x7f6f4415e160 2023-11-24T06:09:32.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f42ffd700 1 -- 172.21.15.97:0/954824868 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f34011990 con 0x7f6f4415e160 2023-11-24T06:09:32.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.876+0000 7f6f42ffd700 1 -- 172.21.15.97:0/954824868 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34011d20 con 0x7f6f4415e160 2023-11-24T06:09:32.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.877+0000 7f6f4a77b700 1 -- 172.21.15.97:0/954824868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415e160 msgr2=0x7f6f440b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:32.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.877+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/954824868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415e160 0x7f6f440b5da0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009840 tx=0x7f6f34009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:32.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.877+0000 7f6f4a77b700 1 -- 172.21.15.97:0/954824868 shutdown_connections 2023-11-24T06:09:32.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.877+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/954824868 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 0x7f6f4406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.877+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/954824868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f4415e160 0x7f6f440b5da0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.877+0000 7f6f4a77b700 1 -- 172.21.15.97:0/954824868 >> 172.21.15.97:0/954824868 conn(0x7f6f440d0bf0 msgr2=0x7f6f440d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:32.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.877+0000 7f6f4a77b700 1 -- 172.21.15.97:0/954824868 shutdown_connections 2023-11-24T06:09:32.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.877+0000 7f6f4a77b700 1 -- 172.21.15.97:0/954824868 wait complete. 2023-11-24T06:09:32.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.878+0000 7f6f4a77b700 1 Processor -- start 2023-11-24T06:09:32.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.878+0000 7f6f4a77b700 1 -- start start 2023-11-24T06:09:32.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f4a77b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 0x7f6f44161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:32.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f4a77b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f440afc90 0x7f6f44155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:32.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f4a77b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f4415e160 0x7f6f4415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:32.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f4a77b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f44160c20 con 0x7f6f440afc90 2023-11-24T06:09:32.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f4a77b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f44155fb0 con 0x7f6f4415e160 2023-11-24T06:09:32.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f4a77b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4415ffa0 con 0x7f6f440746a0 2023-11-24T06:09:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f440afc90 0x7f6f44155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f440afc90 0x7f6f44155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46162/0 (socket says 172.21.15.97:46162) 2023-11-24T06:09:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 -- 172.21.15.97:0/1531824851 learned_addr learned my addr 172.21.15.97:0/1531824851 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f43fff700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 0x7f6f44161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f48d18700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f4415e160 0x7f6f4415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:32.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 -- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 msgr2=0x7f6f44161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 0x7f6f44161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 -- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f4415e160 msgr2=0x7f6f4415f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f4415e160 0x7f6f4415f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 -- 172.21.15.97:0/1531824851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f340094a0 con 0x7f6f440afc90 2023-11-24T06:09:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f48d18700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f4415e160 0x7f6f4415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:32.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f43fff700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 0x7f6f44161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f437fe700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f440afc90 0x7f6f44155a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6f380011e0 tx=0x7f6f3800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.879+0000 7f6f40ff9700 1 -- 172.21.15.97:0/1531824851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38004c80 con 0x7f6f440afc90 2023-11-24T06:09:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.880+0000 7f6f40ff9700 1 -- 172.21.15.97:0/1531824851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38004e00 con 0x7f6f440afc90 2023-11-24T06:09:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.880+0000 7f6f40ff9700 1 -- 172.21.15.97:0/1531824851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38007310 con 0x7f6f440afc90 2023-11-24T06:09:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.880+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f44160280 con 0x7f6f440afc90 2023-11-24T06:09:32.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.880+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f44160840 con 0x7f6f440afc90 2023-11-24T06:09:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.880+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f44160d60 con 0x7f6f440afc90 2023-11-24T06:09:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.881+0000 7f6f40ff9700 1 -- 172.21.15.97:0/1531824851 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f38007470 con 0x7f6f440afc90 2023-11-24T06:09:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.881+0000 7f6f40ff9700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6f3006aed0 0x7f6f3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.881+0000 7f6f40ff9700 1 -- 172.21.15.97:0/1531824851 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f38089410 con 0x7f6f440afc90 2023-11-24T06:09:32.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.881+0000 7f6f43fff700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6f3006aed0 0x7f6f3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:32.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.882+0000 7f6f43fff700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6f3006aed0 0x7f6f3006d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f340090b0 tx=0x7f6f3401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:32.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:32.886+0000 7f6f40ff9700 1 -- 172.21.15.97:0/1531824851 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f44160d60 con 0x7f6f440afc90 2023-11-24T06:09:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.075+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6f44160d60 con 0x7f6f440afc90 2023-11-24T06:09:33.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.078+0000 7f6f40ff9700 1 -- 172.21.15.97:0/1531824851 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f44160d60 con 0x7f6f440afc90 2023-11-24T06:09:33.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6f3006aed0 msgr2=0x7f6f3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6f3006aed0 0x7f6f3006d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f340090b0 tx=0x7f6f3401c040 comp rx=0 tx=0).stop 2023-11-24T06:09:33.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f440afc90 msgr2=0x7f6f44155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f440afc90 0x7f6f44155a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6f380011e0 tx=0x7f6f3800cae0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 shutdown_connections 2023-11-24T06:09:33.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f4415e160 0x7f6f4415f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6f3006aed0 0x7f6f3006d390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6f440746a0 0x7f6f44161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 --2- 172.21.15.97:0/1531824851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f440afc90 0x7f6f44155a70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.080+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 >> 172.21.15.97:0/1531824851 conn(0x7f6f440d0bf0 msgr2=0x7f6f4415c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:33.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.081+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 shutdown_connections 2023-11-24T06:09:33.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.081+0000 7f6f4a77b700 1 -- 172.21.15.97:0/1531824851 wait complete. 2023-11-24T06:09:33.096 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:09:33.096 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:09:33.141 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:09:33.192 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:09:33.192 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:09:33.192 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:33.209 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:09:33.209 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:33.267 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:09:33.268 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:33.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.482+0000 7f551513f700 1 Processor -- start 2023-11-24T06:09:33.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.482+0000 7f551513f700 1 -- start start 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f551513f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f551015bdd0 0x7f55101604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f551513f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f55100b3370 0x7f55100b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f551513f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55100b2030 con 0x7f55100d1c70 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f551513f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55100b2f50 con 0x7f55100b3370 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f551513f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100b22d0 con 0x7f551015bdd0 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550f59e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f55100d1c70 0x7f55100ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38624/0 (socket says 172.21.15.97:38624) 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550f59e700 1 -- 172.21.15.97:0/3675078839 learned_addr learned my addr 172.21.15.97:0/3675078839 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550ed9d700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f551015bdd0 0x7f55101604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550e59c700 1 --2- 172.21.15.97:0/3675078839 >> v2:172.21.15.183:3300/0 conn(0x7f55100b3370 0x7f55100b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889591721 0 0) 0x7f55100b2030 con 0x7f55100d1c70 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54f80035b0 con 0x7f55100d1c70 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2442063741 0 0) 0x7f54f80035b0 con 0x7f55100d1c70 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f551015bdd0 msgr2=0x7f55101604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f551015bdd0 0x7f55101604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 >> v2:172.21.15.183:3300/0 conn(0x7f55100b3370 msgr2=0x7f55100b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 --2- 172.21.15.97:0/3675078839 >> v2:172.21.15.183:3300/0 conn(0x7f55100b3370 0x7f55100b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100b25b0 con 0x7f55100d1c70 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f55080019f0 con 0x7f55100d1c70 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 >> v1:172.21.15.97:6789/0 conn(0x7f55100d1c70 legacy=0x7f55100ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54f8003a00 0x7f54f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8004850 0x7f54f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f551015bdd0 0x7f54f8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100b2f50 con 0x7f54f8003a00 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100b22d0 con 0x7f54f8004850 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.483+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f800ba10 con 0x7f551015bdd0 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550ed9d700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54f8003a00 0x7f54f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550e59c700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8004850 0x7f54f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550f59e700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f551015bdd0 0x7f54f8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550ed9d700 1 -- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f551015bdd0 msgr2=0x7f54f8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550ed9d700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f551015bdd0 0x7f54f8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550ed9d700 1 -- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8004850 msgr2=0x7f54f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:33.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550ed9d700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8004850 0x7f54f8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550ed9d700 1 -- 172.21.15.97:0/3675078839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80035b0 con 0x7f54f8003a00 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550ed9d700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54f8003a00 0x7f54f8003e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5504009b60 tx=0x7f54f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7181046fd492550c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400e6c0 con 0x7f54f8003a00 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400e820 con 0x7f54f8003a00 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.484+0000 7f550dd9b700 1 -- 172.21.15.97:0/3675078839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400eb90 con 0x7f54f8003a00 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.485+0000 7f551513f700 1 -- 172.21.15.97:0/3675078839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54f8003a00 msgr2=0x7f54f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.485+0000 7f551513f700 1 --2- 172.21.15.97:0/3675078839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54f8003a00 0x7f54f8003e20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5504009b60 tx=0x7f54f8005880 comp rx=0 tx=0).stop 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.485+0000 7f550f59e700 1 -- 172.21.15.97:0/3675078839 reap_dead start 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.485+0000 7f551513f700 1 -- 172.21.15.97:0/3675078839 shutdown_connections 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.485+0000 7f551513f700 1 -- 172.21.15.97:0/3675078839 >> 172.21.15.97:0/3675078839 conn(0x7f55100731a0 msgr2=0x7f551009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.485+0000 7f551513f700 1 -- 172.21.15.97:0/3675078839 shutdown_connections 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.485+0000 7f551513f700 1 -- 172.21.15.97:0/3675078839 wait complete. 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.486+0000 7f551513f700 1 Processor -- start 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.486+0000 7f551513f700 1 -- start start 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f551513f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8003a00 0x7f551015e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f551513f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55100c4960 0x7f55100c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f551513f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55100ca4a0 0x7f55101e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f551513f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100cb570 con 0x7f55100c4960 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f551513f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100ca960 con 0x7f54f8003a00 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f551513f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100cac40 con 0x7f55100ca4a0 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55100c4960 0x7f55100c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55100c4960 0x7f55100c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46180/0 (socket says 172.21.15.97:46180) 2023-11-24T06:09:33.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550e59c700 1 -- 172.21.15.97:0/137040220 learned_addr learned my addr 172.21.15.97:0/137040220 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550ed9d700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8003a00 0x7f551015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550f59e700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55100ca4a0 0x7f55101e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550e59c700 1 -- 172.21.15.97:0/137040220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55100ca4a0 msgr2=0x7f55101e7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550e59c700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55100ca4a0 0x7f55101e7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550e59c700 1 -- 172.21.15.97:0/137040220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8003a00 msgr2=0x7f551015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550e59c700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8003a00 0x7f551015e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.487+0000 7f550e59c700 1 -- 172.21.15.97:0/137040220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100caf20 con 0x7f55100c4960 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.488+0000 7f550e59c700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55100c4960 0x7f55100c9f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5500011aa0 tx=0x7f5500011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.488+0000 7f54f77fe700 1 -- 172.21.15.97:0/137040220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550000f940 con 0x7f55100c4960 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.488+0000 7f54f77fe700 1 -- 172.21.15.97:0/137040220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550000faa0 con 0x7f55100c4960 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.488+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55100cb200 con 0x7f55100c4960 2023-11-24T06:09:33.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.488+0000 7f54f77fe700 1 -- 172.21.15.97:0/137040220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550000fe50 con 0x7f55100c4960 2023-11-24T06:09:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.488+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551015ed60 con 0x7f55100c4960 2023-11-24T06:09:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.489+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5510067f60 con 0x7f55100c4960 2023-11-24T06:09:33.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.489+0000 7f54f77fe700 1 -- 172.21.15.97:0/137040220 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5500014440 con 0x7f55100c4960 2023-11-24T06:09:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.489+0000 7f54f77fe700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f54fc06aed0 0x7f54fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.490+0000 7f54f77fe700 1 -- 172.21.15.97:0/137040220 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5500089ae0 con 0x7f55100c4960 2023-11-24T06:09:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.490+0000 7f550ed9d700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f54fc06aed0 0x7f54fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.490+0000 7f550ed9d700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f54fc06aed0 0x7f54fc06d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55040090b0 tx=0x7f550401f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:33.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.494+0000 7f54f77fe700 1 -- 172.21.15.97:0/137040220 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5500054b90 con 0x7f55100c4960 2023-11-24T06:09:33.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.685+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f551015ed60 con 0x7f55100c4960 2023-11-24T06:09:33.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.686+0000 7f54f77fe700 1 -- 172.21.15.97:0/137040220 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f55000581b0 con 0x7f55100c4960 2023-11-24T06:09:33.689 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:33.690 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f54fc06aed0 msgr2=0x7f54fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f54fc06aed0 0x7f54fc06d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f55040090b0 tx=0x7f550401f040 comp rx=0 tx=0).stop 2023-11-24T06:09:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55100c4960 msgr2=0x7f55100c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55100c4960 0x7f55100c9f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5500011aa0 tx=0x7f5500011db0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 shutdown_connections 2023-11-24T06:09:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f54fc06aed0 0x7f54fc06d390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55100ca4a0 0x7f55101e7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54f8003a00 0x7f551015e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 --2- 172.21.15.97:0/137040220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f55100c4960 0x7f55100c9f60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 >> 172.21.15.97:0/137040220 conn(0x7f55100731a0 msgr2=0x7f55100aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:33.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 shutdown_connections 2023-11-24T06:09:33.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.689+0000 7f551513f700 1 -- 172.21.15.97:0/137040220 wait complete. 2023-11-24T06:09:33.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:33.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:33.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.825+0000 7f5eacb77700 1 Processor -- start 2023-11-24T06:09:33.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.826+0000 7f5eacb77700 1 -- start start 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.826+0000 7f5eacb77700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ea80c6c30 0x7f5ea80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.826+0000 7f5eacb77700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5ea80cef40 0x7f5ea80cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5eacb77700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea80d0140 con 0x7f5ea815bf50 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5eacb77700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea815c330 con 0x7f5ea80c6c30 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5eacb77700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea80cf3d0 con 0x7f5ea80cef40 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea5d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ea80c6c30 0x7f5ea80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea5d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5ea80c6c30 0x7f5ea80cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46188/0 (socket says 172.21.15.97:46188) 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea5d9b700 1 -- 172.21.15.97:0/197523192 learned_addr learned my addr 172.21.15.97:0/197523192 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea6d9d700 1 --2- 172.21.15.97:0/197523192 >> v2:172.21.15.183:3300/0 conn(0x7f5ea80cef40 0x7f5ea80cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea5d9b700 1 -- 172.21.15.97:0/197523192 >> v2:172.21.15.183:3300/0 conn(0x7f5ea80cef40 msgr2=0x7f5ea80cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea5d9b700 1 --2- 172.21.15.97:0/197523192 >> v2:172.21.15.183:3300/0 conn(0x7f5ea80cef40 0x7f5ea80cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea5d9b700 1 -- 172.21.15.97:0/197523192 >> v1:172.21.15.183:6789/0 conn(0x7f5ea815bf50 legacy=0x7f5ea80ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea5d9b700 1 -- 172.21.15.97:0/197523192 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea80d03e0 con 0x7f5ea80c6c30 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea5d9b700 1 --2- 172.21.15.97:0/197523192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80c6c30 0x7f5ea80cea00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80c8c00 tx=0x7f5e98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7efdfd3eae3ffbd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea559a700 1 -- 172.21.15.97:0/197523192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800b870 con 0x7f5ea80c6c30 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea559a700 1 -- 172.21.15.97:0/197523192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9800b9d0 con 0x7f5ea80c6c30 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.827+0000 7f5ea559a700 1 -- 172.21.15.97:0/197523192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800bd60 con 0x7f5ea80c6c30 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.828+0000 7f5eacb77700 1 -- 172.21.15.97:0/197523192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80c6c30 msgr2=0x7f5ea80cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.828+0000 7f5eacb77700 1 --2- 172.21.15.97:0/197523192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80c6c30 0x7f5ea80cea00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80c8c00 tx=0x7f5e98009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.828+0000 7f5eacb77700 1 -- 172.21.15.97:0/197523192 shutdown_connections 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.828+0000 7f5eacb77700 1 --2- 172.21.15.97:0/197523192 >> v2:172.21.15.183:3300/0 conn(0x7f5ea80cef40 0x7f5ea80cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.828+0000 7f5eacb77700 1 --2- 172.21.15.97:0/197523192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80c6c30 0x7f5ea80cea00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.828+0000 7f5eacb77700 1 -- 172.21.15.97:0/197523192 >> 172.21.15.97:0/197523192 conn(0x7f5ea80b3780 msgr2=0x7f5ea80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:33.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.828+0000 7f5eacb77700 1 -- 172.21.15.97:0/197523192 shutdown_connections 2023-11-24T06:09:33.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.828+0000 7f5eacb77700 1 -- 172.21.15.97:0/197523192 wait complete. 2023-11-24T06:09:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.829+0000 7f5eacb77700 1 Processor -- start 2023-11-24T06:09:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.829+0000 7f5eacb77700 1 -- start start 2023-11-24T06:09:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5eacb77700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ea80c6c30 0x7f5ea8155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5eacb77700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80cef40 0x7f5ea80cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5eacb77700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ea815bf50 0x7f5ea8154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5eacb77700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea80cbcb0 con 0x7f5ea80cef40 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5eacb77700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8155170 con 0x7f5ea80c6c30 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5eacb77700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea8156150 con 0x7f5ea815bf50 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80cef40 0x7f5ea80cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80cef40 0x7f5ea80cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46196/0 (socket says 172.21.15.97:46196) 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea5d9b700 1 -- 172.21.15.97:0/2956110219 learned_addr learned my addr 172.21.15.97:0/2956110219 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea659c700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ea80c6c30 0x7f5ea8155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea6d9d700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ea815bf50 0x7f5ea8154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea5d9b700 1 -- 172.21.15.97:0/2956110219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ea815bf50 msgr2=0x7f5ea8154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea5d9b700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ea815bf50 0x7f5ea8154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea5d9b700 1 -- 172.21.15.97:0/2956110219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ea80c6c30 msgr2=0x7f5ea8155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:33.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea5d9b700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ea80c6c30 0x7f5ea8155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.830+0000 7f5ea5d9b700 1 -- 172.21.15.97:0/2956110219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009550 con 0x7f5ea80cef40 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.831+0000 7f5ea5d9b700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80cef40 0x7f5ea80cb770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e98000c00 tx=0x7f5e98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.831+0000 7f5e96ffd700 1 -- 172.21.15.97:0/2956110219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98004540 con 0x7f5ea80cef40 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.831+0000 7f5e96ffd700 1 -- 172.21.15.97:0/2956110219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e980036a0 con 0x7f5ea80cef40 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.831+0000 7f5e96ffd700 1 -- 172.21.15.97:0/2956110219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98003a50 con 0x7f5ea80cef40 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.831+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea8156430 con 0x7f5ea80cef40 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.831+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea81569f0 con 0x7f5ea80cef40 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.832+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea8068fb0 con 0x7f5ea80cef40 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.832+0000 7f5e96ffd700 1 -- 172.21.15.97:0/2956110219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e98021070 con 0x7f5ea80cef40 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.832+0000 7f5e96ffd700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e8806aed0 0x7f5e8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:33.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.833+0000 7f5e96ffd700 1 -- 172.21.15.97:0/2956110219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e9808abd0 con 0x7f5ea80cef40 2023-11-24T06:09:33.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.833+0000 7f5ea659c700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e8806aed0 0x7f5e8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:33.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.833+0000 7f5ea659c700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e8806aed0 0x7f5e8806d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c007fd0 tx=0x7f5e9c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:33.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:33.837+0000 7f5e96ffd700 1 -- 172.21.15.97:0/2956110219 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e98055dc0 con 0x7f5ea80cef40 2023-11-24T06:09:34.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.027+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ea80c59e0 con 0x7f5ea80cef40 2023-11-24T06:09:34.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.027+0000 7f5e96ffd700 1 -- 172.21.15.97:0/2956110219 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e980290a0 con 0x7f5ea80cef40 2023-11-24T06:09:34.030 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e8806aed0 msgr2=0x7f5e8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e8806aed0 0x7f5e8806d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c007fd0 tx=0x7f5e9c009040 comp rx=0 tx=0).stop 2023-11-24T06:09:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80cef40 msgr2=0x7f5ea80cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80cef40 0x7f5ea80cb770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e98000c00 tx=0x7f5e98004750 comp rx=0 tx=0).stop 2023-11-24T06:09:34.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 shutdown_connections 2023-11-24T06:09:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e8806aed0 0x7f5e8806d390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5ea815bf50 0x7f5ea8154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ea80c6c30 0x7f5ea8155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 --2- 172.21.15.97:0/2956110219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ea80cef40 0x7f5ea80cb770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 >> 172.21.15.97:0/2956110219 conn(0x7f5ea80b3780 msgr2=0x7f5ea80c7980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 shutdown_connections 2023-11-24T06:09:34.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.029+0000 7f5eacb77700 1 -- 172.21.15.97:0/2956110219 wait complete. 2023-11-24T06:09:34.044 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:34.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.164+0000 7f1c82fe6700 1 Processor -- start 2023-11-24T06:09:34.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.164+0000 7f1c82fe6700 1 -- start start 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.165+0000 7f1c82fe6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1c7c0746a0 0x7f1c7c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.165+0000 7f1c82fe6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0afc90 0x7f1c7c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.165+0000 7f1c82fe6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7c0b3c60 con 0x7f1c7c15e160 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.165+0000 7f1c82fe6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b4500 con 0x7f1c7c0746a0 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.165+0000 7f1c82fe6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b58c0 con 0x7f1c7c0afc90 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.165+0000 7f1c7bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1c7c0746a0 0x7f1c7c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.165+0000 7f1c80d82700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1c7c15e160 0x7f1c7c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38652/0 (socket says 172.21.15.97:38652) 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.165+0000 7f1c80d82700 1 -- 172.21.15.97:0/971401119 learned_addr learned my addr 172.21.15.97:0/971401119 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c81583700 1 --2- 172.21.15.97:0/971401119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0afc90 0x7f1c7c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7bfff700 1 -- 172.21.15.97:0/971401119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0afc90 msgr2=0x7f1c7c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:34.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7bfff700 1 --2- 172.21.15.97:0/971401119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0afc90 0x7f1c7c06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7bfff700 1 -- 172.21.15.97:0/971401119 >> v1:172.21.15.97:6789/0 conn(0x7f1c7c15e160 legacy=0x7f1c7c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7bfff700 1 -- 172.21.15.97:0/971401119 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0b0120 con 0x7f1c7c0746a0 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7b7fe700 1 -- 172.21.15.97:0/971401119 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590846575 0 0) 0x7f1c7c0b3c60 con 0x7f1c7c15e160 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7bfff700 1 --2- 172.21.15.97:0/971401119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c0746a0 0x7f1c7c0b5da0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009840 tx=0x7f1c68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac301986632e00e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7b7fe700 1 -- 172.21.15.97:0/971401119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011830 con 0x7f1c7c0746a0 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7b7fe700 1 -- 172.21.15.97:0/971401119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c68011990 con 0x7f1c7c0746a0 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c7b7fe700 1 -- 172.21.15.97:0/971401119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68011d20 con 0x7f1c7c0746a0 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c82fe6700 1 -- 172.21.15.97:0/971401119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c0746a0 msgr2=0x7f1c7c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.166+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/971401119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c0746a0 0x7f1c7c0b5da0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009840 tx=0x7f1c68009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.167+0000 7f1c82fe6700 1 -- 172.21.15.97:0/971401119 shutdown_connections 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.167+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/971401119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0afc90 0x7f1c7c06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.167+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/971401119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c0746a0 0x7f1c7c0b5da0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.167+0000 7f1c82fe6700 1 -- 172.21.15.97:0/971401119 >> 172.21.15.97:0/971401119 conn(0x7f1c7c0d0bf0 msgr2=0x7f1c7c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:34.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.167+0000 7f1c82fe6700 1 -- 172.21.15.97:0/971401119 shutdown_connections 2023-11-24T06:09:34.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.167+0000 7f1c82fe6700 1 -- 172.21.15.97:0/971401119 wait complete. 2023-11-24T06:09:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.168+0000 7f1c82fe6700 1 Processor -- start 2023-11-24T06:09:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.168+0000 7f1c82fe6700 1 -- start start 2023-11-24T06:09:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c82fe6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0746a0 0x7f1c7c154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c82fe6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c82fe6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c15e160 0x7f1c7c141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c82fe6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c155750 con 0x7f1c7c15e160 2023-11-24T06:09:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c82fe6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c154b40 con 0x7f1c7c0afc90 2023-11-24T06:09:34.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c82fe6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c154e20 con 0x7f1c7c0746a0 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c15e160 0x7f1c7c141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c15e160 0x7f1c7c141710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46208/0 (socket says 172.21.15.97:46208) 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 -- 172.21.15.97:0/2032716963 learned_addr learned my addr 172.21.15.97:0/2032716963 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 -- 172.21.15.97:0/2032716963 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0746a0 msgr2=0x7f1c7c154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c80d82700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0746a0 0x7f1c7c154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c7bfff700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0746a0 0x7f1c7c154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 -- 172.21.15.97:0/2032716963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7c0afc90 msgr2=0x7f1c7c153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 -- 172.21.15.97:0/2032716963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c680094a0 con 0x7f1c7c15e160 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c81583700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c15e160 0x7f1c7c141710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c74007e40 tx=0x7f1c7400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c797fa700 1 -- 172.21.15.97:0/2032716963 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400eea0 con 0x7f1c7c15e160 2023-11-24T06:09:34.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c797fa700 1 -- 172.21.15.97:0/2032716963 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c74014420 con 0x7f1c7c15e160 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c155100 con 0x7f1c7c15e160 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c797fa700 1 -- 172.21.15.97:0/2032716963 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740147b0 con 0x7f1c7c15e160 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.169+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c153e70 con 0x7f1c7c15e160 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.170+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c1558c0 con 0x7f1c7c15e160 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.171+0000 7f1c797fa700 1 -- 172.21.15.97:0/2032716963 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c74014910 con 0x7f1c7c15e160 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.171+0000 7f1c797fa700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1c6006aed0 0x7f1c6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.171+0000 7f1c80d82700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1c6006aed0 0x7f1c6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.171+0000 7f1c797fa700 1 -- 172.21.15.97:0/2032716963 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c74089e20 con 0x7f1c7c15e160 2023-11-24T06:09:34.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.172+0000 7f1c80d82700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1c6006aed0 0x7f1c6006d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000f60 tx=0x7f1c70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.176+0000 7f1c797fa700 1 -- 172.21.15.97:0/2032716963 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c74054ed0 con 0x7f1c7c15e160 2023-11-24T06:09:34.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.368+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c7c068fb0 con 0x7f1c7c15e160 2023-11-24T06:09:34.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.368+0000 7f1c797fa700 1 -- 172.21.15.97:0/2032716963 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c740584f0 con 0x7f1c7c15e160 2023-11-24T06:09:34.371 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.370+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1c6006aed0 msgr2=0x7f1c6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.370+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1c6006aed0 0x7f1c6006d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000f60 tx=0x7f1c70009040 comp rx=0 tx=0).stop 2023-11-24T06:09:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.370+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c15e160 msgr2=0x7f1c7c141710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.370+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c15e160 0x7f1c7c141710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c74007e40 tx=0x7f1c7400e3c0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.371+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 shutdown_connections 2023-11-24T06:09:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.371+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c7c15e160 0x7f1c7c141710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.371+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1c6006aed0 0x7f1c6006d390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.371+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1c7c0746a0 0x7f1c7c154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.371+0000 7f1c82fe6700 1 --2- 172.21.15.97:0/2032716963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7c0afc90 0x7f1c7c153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.371+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 >> 172.21.15.97:0/2032716963 conn(0x7f1c7c0d0bf0 msgr2=0x7f1c7c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.371+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 shutdown_connections 2023-11-24T06:09:34.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.371+0000 7f1c82fe6700 1 -- 172.21.15.97:0/2032716963 wait complete. 2023-11-24T06:09:34.386 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.506+0000 7f77fd4b4700 1 Processor -- start 2023-11-24T06:09:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.506+0000 7f77fd4b4700 1 -- start start 2023-11-24T06:09:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77fd4b4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77f815e150 0x7f77f806f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77fd4b4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8073f90 0x7f77f8074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77fd4b4700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f80b0590 con 0x7f77f80b3bf0 2023-11-24T06:09:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77fd4b4700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f80b4360 con 0x7f77f815e150 2023-11-24T06:09:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77fd4b4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8074960 con 0x7f77f8073f90 2023-11-24T06:09:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77f815e150 0x7f77f806f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f67fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f77f80b3bf0 0x7f77f806e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38664/0 (socket says 172.21.15.97:38664) 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f67fc700 1 -- 172.21.15.97:0/3151576075 learned_addr learned my addr 172.21.15.97:0/3151576075 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f6ffd700 1 -- 172.21.15.97:0/3151576075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8073f90 msgr2=0x7f77f8074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f6ffd700 1 --2- 172.21.15.97:0/3151576075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8073f90 0x7f77f8074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f6ffd700 1 -- 172.21.15.97:0/3151576075 >> v1:172.21.15.97:6789/0 conn(0x7f77f80b3bf0 legacy=0x7f77f806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f6ffd700 1 -- 172.21.15.97:0/3151576075 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f806ecd0 con 0x7f77f815e150 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f5ffb700 1 -- 172.21.15.97:0/3151576075 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176284560 0 0) 0x7f77f80b0590 con 0x7f77f80b3bf0 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f6ffd700 1 --2- 172.21.15.97:0/3151576075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f806f480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009840 tx=0x7f77ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a47f6e18c57c0b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f5ffb700 1 -- 172.21.15.97:0/3151576075 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011830 con 0x7f77f815e150 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.507+0000 7f77f5ffb700 1 -- 172.21.15.97:0/3151576075 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec011990 con 0x7f77f815e150 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77f5ffb700 1 -- 172.21.15.97:0/3151576075 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011d20 con 0x7f77f815e150 2023-11-24T06:09:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77fd4b4700 1 -- 172.21.15.97:0/3151576075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 msgr2=0x7f77f806f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/3151576075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f806f480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009840 tx=0x7f77ec009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77fd4b4700 1 -- 172.21.15.97:0/3151576075 shutdown_connections 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/3151576075 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8073f90 0x7f77f8074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/3151576075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f806f480 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77fd4b4700 1 -- 172.21.15.97:0/3151576075 >> 172.21.15.97:0/3151576075 conn(0x7f77f80d0be0 msgr2=0x7f77f80d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77fd4b4700 1 -- 172.21.15.97:0/3151576075 shutdown_connections 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.508+0000 7f77fd4b4700 1 -- 172.21.15.97:0/3151576075 wait complete. 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.509+0000 7f77fd4b4700 1 Processor -- start 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.509+0000 7f77fd4b4700 1 -- start start 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77fd4b4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f8073f90 0x7f77f8153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77fd4b4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f81547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77fd4b4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8148700 0x7f77f8155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77fd4b4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8148bb0 con 0x7f77f815e150 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77fd4b4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8155f60 con 0x7f77f8073f90 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77fd4b4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f8153730 con 0x7f77f8148700 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77f67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f81547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77f67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f81547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46220/0 (socket says 172.21.15.97:46220) 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77f67fc700 1 -- 172.21.15.97:0/83539846 learned_addr learned my addr 172.21.15.97:0/83539846 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77f6ffd700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f8073f90 0x7f77f8153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77f77fe700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8148700 0x7f77f8155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.510+0000 7f77f67fc700 1 -- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8148700 msgr2=0x7f77f8155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77f67fc700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8148700 0x7f77f8155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77f67fc700 1 -- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f8073f90 msgr2=0x7f77f8153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77f67fc700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f8073f90 0x7f77f8153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77f67fc700 1 -- 172.21.15.97:0/83539846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec0094a0 con 0x7f77f815e150 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77f6ffd700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f8073f90 0x7f77f8153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77f67fc700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f81547d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f77e80011e0 tx=0x7f77e800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77df7fe700 1 -- 172.21.15.97:0/83539846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8004c80 con 0x7f77f815e150 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77df7fe700 1 -- 172.21.15.97:0/83539846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8004e00 con 0x7f77f815e150 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f8153a10 con 0x7f77f815e150 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77df7fe700 1 -- 172.21.15.97:0/83539846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8007310 con 0x7f77f815e150 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.511+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f8153fd0 con 0x7f77f815e150 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.512+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f8068fb0 con 0x7f77f815e150 2023-11-24T06:09:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.512+0000 7f77df7fe700 1 -- 172.21.15.97:0/83539846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77e8007470 con 0x7f77f815e150 2023-11-24T06:09:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.513+0000 7f77df7fe700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77e406aed0 0x7f77e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.513+0000 7f77df7fe700 1 -- 172.21.15.97:0/83539846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77e8089450 con 0x7f77f815e150 2023-11-24T06:09:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.513+0000 7f77f6ffd700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77e406aed0 0x7f77e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.513+0000 7f77f6ffd700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77e406aed0 0x7f77e406d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0090b0 tx=0x7f77ec01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.517+0000 7f77df7fe700 1 -- 172.21.15.97:0/83539846 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77e8054460 con 0x7f77f815e150 2023-11-24T06:09:34.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.709+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77f806c850 con 0x7f77f815e150 2023-11-24T06:09:34.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.710+0000 7f77df7fe700 1 -- 172.21.15.97:0/83539846 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77e8057a80 con 0x7f77f815e150 2023-11-24T06:09:34.714 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:34.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77e406aed0 msgr2=0x7f77e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77e406aed0 0x7f77e406d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0090b0 tx=0x7f77ec01c040 comp rx=0 tx=0).stop 2023-11-24T06:09:34.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 msgr2=0x7f77f81547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f81547d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f77e80011e0 tx=0x7f77e800cae0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 shutdown_connections 2023-11-24T06:09:34.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77e406aed0 0x7f77e406d390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77f8148700 0x7f77f8155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77f8073f90 0x7f77f8153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 --2- 172.21.15.97:0/83539846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f815e150 0x7f77f81547d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 >> 172.21.15.97:0/83539846 conn(0x7f77f80d0be0 msgr2=0x7f77f806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:34.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 shutdown_connections 2023-11-24T06:09:34.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.712+0000 7f77fd4b4700 1 -- 172.21.15.97:0/83539846 wait complete. 2023-11-24T06:09:34.727 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:09:34.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.845+0000 7f64d8a0d700 1 Processor -- start 2023-11-24T06:09:34.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.845+0000 7f64d8a0d700 1 -- start start 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.846+0000 7f64d8a0d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.846+0000 7f64d8a0d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f64d406c710 0x7f64d40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.846+0000 7f64d8a0d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d40ae430 con 0x7f64d40af4d0 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.846+0000 7f64d8a0d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d40af0b0 con 0x7f64d406c710 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.846+0000 7f64d8a0d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40b05a0 con 0x7f64d415e080 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d2d9d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f64d406c710 0x7f64d40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d1d9b700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f64d40af4d0 0x7f64d406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:48000/0 (socket says 172.21.15.97:48000) 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d259c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d1d9b700 1 -- 172.21.15.97:0/1592304947 learned_addr learned my addr 172.21.15.97:0/1592304947 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d159a700 1 -- 172.21.15.97:0/1592304947 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166094570 0 0) 0x7f64d40ae430 con 0x7f64d40af4d0 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d159a700 1 -- 172.21.15.97:0/1592304947 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64bc0035b0 con 0x7f64d40af4d0 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d259c700 1 -- 172.21.15.97:0/1592304947 >> v2:172.21.15.183:3300/0 conn(0x7f64d406c710 msgr2=0x7f64d40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d259c700 1 --2- 172.21.15.97:0/1592304947 >> v2:172.21.15.183:3300/0 conn(0x7f64d406c710 0x7f64d40afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d259c700 1 -- 172.21.15.97:0/1592304947 >> v1:172.21.15.183:6789/0 conn(0x7f64d40af4d0 legacy=0x7f64d406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d259c700 1 -- 172.21.15.97:0/1592304947 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40ae710 con 0x7f64d415e080 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d2d9d700 1 --2- 172.21.15.97:0/1592304947 >> v2:172.21.15.183:3300/0 conn(0x7f64d406c710 0x7f64d40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d259c700 1 --2- 172.21.15.97:0/1592304947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d40b5050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009840 tx=0x7f64c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c147a3b18b0d28 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d159a700 1 -- 172.21.15.97:0/1592304947 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8011830 con 0x7f64d415e080 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d159a700 1 -- 172.21.15.97:0/1592304947 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8011990 con 0x7f64d415e080 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.847+0000 7f64d159a700 1 -- 172.21.15.97:0/1592304947 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8011d00 con 0x7f64d415e080 2023-11-24T06:09:34.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.848+0000 7f64d8a0d700 1 -- 172.21.15.97:0/1592304947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 msgr2=0x7f64d40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.848+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/1592304947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d40b5050 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009840 tx=0x7f64c8009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.848+0000 7f64d8a0d700 1 -- 172.21.15.97:0/1592304947 shutdown_connections 2023-11-24T06:09:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.848+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/1592304947 >> v2:172.21.15.183:3300/0 conn(0x7f64d406c710 0x7f64d40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.848+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/1592304947 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d40b5050 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.848+0000 7f64d8a0d700 1 -- 172.21.15.97:0/1592304947 >> 172.21.15.97:0/1592304947 conn(0x7f64d4071b90 msgr2=0x7f64d4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.848+0000 7f64d8a0d700 1 -- 172.21.15.97:0/1592304947 shutdown_connections 2023-11-24T06:09:34.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.848+0000 7f64d8a0d700 1 -- 172.21.15.97:0/1592304947 wait complete. 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.849+0000 7f64d8a0d700 1 Processor -- start 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.849+0000 7f64d8a0d700 1 -- start start 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d8a0d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d406c710 0x7f64d41f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d8a0d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64d40af4d0 0x7f64d4153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d8a0d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d41565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d8a0d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4153b40 con 0x7f64d406c710 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d8a0d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4155550 con 0x7f64d40af4d0 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d8a0d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4155830 con 0x7f64d415e080 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d406c710 0x7f64d41f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d406c710 0x7f64d41f3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46230/0 (socket says 172.21.15.97:46230) 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d259c700 1 -- 172.21.15.97:0/2526730283 learned_addr learned my addr 172.21.15.97:0/2526730283 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d1d9b700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64d40af4d0 0x7f64d4153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.850+0000 7f64d2d9d700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d41565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64d259c700 1 -- 172.21.15.97:0/2526730283 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 msgr2=0x7f64d41565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64d259c700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d41565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64d259c700 1 -- 172.21.15.97:0/2526730283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64d40af4d0 msgr2=0x7f64d4153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64d259c700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64d40af4d0 0x7f64d4153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64d259c700 1 -- 172.21.15.97:0/2526730283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc009a70 con 0x7f64d406c710 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64d259c700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d406c710 0x7f64d41f3660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64c80090b0 tx=0x7f64c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64c2ffd700 1 -- 172.21.15.97:0/2526730283 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80045d0 con 0x7f64d406c710 2023-11-24T06:09:34.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64c2ffd700 1 -- 172.21.15.97:0/2526730283 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c801e070 con 0x7f64d406c710 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c80094a0 con 0x7f64d406c710 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64c2ffd700 1 -- 172.21.15.97:0/2526730283 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80038c0 con 0x7f64d406c710 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.851+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d4155df0 con 0x7f64d406c710 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.852+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d40753f0 con 0x7f64d406c710 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.852+0000 7f64c2ffd700 1 -- 172.21.15.97:0/2526730283 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64c8003a20 con 0x7f64d406c710 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.853+0000 7f64c2ffd700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64b406aed0 0x7f64b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.853+0000 7f64d1d9b700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64b406aed0 0x7f64b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.853+0000 7f64c2ffd700 1 -- 172.21.15.97:0/2526730283 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64c808a360 con 0x7f64d406c710 2023-11-24T06:09:34.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.853+0000 7f64d1d9b700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64b406aed0 0x7f64b406d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64c4007fd0 tx=0x7f64c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:34.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:34.857+0000 7f64c2ffd700 1 -- 172.21.15.97:0/2526730283 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64c80554a0 con 0x7f64d406c710 2023-11-24T06:09:35.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.064+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f64d4067f60 con 0x7f64d406c710 2023-11-24T06:09:35.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.064+0000 7f64c2ffd700 1 -- 172.21.15.97:0/2526730283 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f64c8058ac0 con 0x7f64d406c710 2023-11-24T06:09:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64b406aed0 msgr2=0x7f64b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64b406aed0 0x7f64b406d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64c4007fd0 tx=0x7f64c4009040 comp rx=0 tx=0).stop 2023-11-24T06:09:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d406c710 msgr2=0x7f64d41f3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d406c710 0x7f64d41f3660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64c80090b0 tx=0x7f64c8004750 comp rx=0 tx=0).stop 2023-11-24T06:09:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 shutdown_connections 2023-11-24T06:09:35.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64b406aed0 0x7f64b406d390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f64d415e080 0x7f64d41565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64d406c710 0x7f64d41f3660 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 --2- 172.21.15.97:0/2526730283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64d40af4d0 0x7f64d4153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.066+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 >> 172.21.15.97:0/2526730283 conn(0x7f64d4071b90 msgr2=0x7f64d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.067+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 shutdown_connections 2023-11-24T06:09:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.067+0000 7f64d8a0d700 1 -- 172.21.15.97:0/2526730283 wait complete. 2023-11-24T06:09:35.071 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:09:35.081 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:09:35.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:35.082 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-24T06:09:35.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:09:35.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:35.133 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:09:35.154 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:09:35.154 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:35.175 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:09:35.175 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:35.236 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:09:35.236 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:35.295 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.413+0000 7fb05863b700 1 Processor -- start 2023-11-24T06:09:35.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.413+0000 7fb05863b700 1 -- start start 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.414+0000 7fb05863b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb050152cc0 0x7fb05006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.414+0000 7fb05863b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb0500d0630 0x7fb05006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.414+0000 7fb05863b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb05015c4b0 con 0x7fb0500b49a0 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.414+0000 7fb05863b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05015c790 con 0x7fb050152cc0 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.414+0000 7fb05863b700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05015ca70 con 0x7fb0500d0630 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.414+0000 7fb0563d7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb050152cc0 0x7fb05006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0563d7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb050152cc0 0x7fb05006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46240/0 (socket says 172.21.15.97:46240) 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0563d7700 1 -- 172.21.15.97:0/4197555822 learned_addr learned my addr 172.21.15.97:0/4197555822 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0563d7700 1 -- 172.21.15.97:0/4197555822 >> v2:172.21.15.183:3300/0 conn(0x7fb0500d0630 msgr2=0x7fb05006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:35.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0563d7700 1 --2- 172.21.15.97:0/4197555822 >> v2:172.21.15.183:3300/0 conn(0x7fb0500d0630 0x7fb05006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0563d7700 1 -- 172.21.15.97:0/4197555822 >> v1:172.21.15.183:6789/0 conn(0x7fb0500b49a0 legacy=0x7fb0500cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0563d7700 1 -- 172.21.15.97:0/4197555822 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05015cd50 con 0x7fb050152cc0 2023-11-24T06:09:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0563d7700 1 --2- 172.21.15.97:0/4197555822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05006c000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0500b4e80 tx=0x7fb04c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa24cad602866b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0553d5700 1 -- 172.21.15.97:0/4197555822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c004230 con 0x7fb050152cc0 2023-11-24T06:09:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0553d5700 1 -- 172.21.15.97:0/4197555822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c004390 con 0x7fb050152cc0 2023-11-24T06:09:35.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.415+0000 7fb0553d5700 1 -- 172.21.15.97:0/4197555822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c0036a0 con 0x7fb050152cc0 2023-11-24T06:09:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.416+0000 7fb05863b700 1 -- 172.21.15.97:0/4197555822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 msgr2=0x7fb05006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.416+0000 7fb05863b700 1 --2- 172.21.15.97:0/4197555822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05006c000 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0500b4e80 tx=0x7fb04c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.416+0000 7fb05863b700 1 -- 172.21.15.97:0/4197555822 shutdown_connections 2023-11-24T06:09:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.416+0000 7fb05863b700 1 --2- 172.21.15.97:0/4197555822 >> v2:172.21.15.183:3300/0 conn(0x7fb0500d0630 0x7fb05006c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.416+0000 7fb05863b700 1 --2- 172.21.15.97:0/4197555822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05006c000 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.416+0000 7fb05863b700 1 -- 172.21.15.97:0/4197555822 >> 172.21.15.97:0/4197555822 conn(0x7fb050071b90 msgr2=0x7fb050071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:35.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.416+0000 7fb05863b700 1 -- 172.21.15.97:0/4197555822 shutdown_connections 2023-11-24T06:09:35.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.416+0000 7fb05863b700 1 -- 172.21.15.97:0/4197555822 wait complete. 2023-11-24T06:09:35.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.417+0000 7fb05863b700 1 Processor -- start 2023-11-24T06:09:35.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.417+0000 7fb05863b700 1 -- start start 2023-11-24T06:09:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb05863b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0500b49a0 0x7fb0501472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb05863b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0500d0630 0x7fb05015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb05863b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb05863b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501532b0 con 0x7fb050152cc0 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb05863b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501575b0 con 0x7fb0500b49a0 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb05863b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501569e0 con 0x7fb0500d0630 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05015f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46246/0 (socket says 172.21.15.97:46246) 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 -- 172.21.15.97:0/2725382161 learned_addr learned my addr 172.21.15.97:0/2725382161 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb0563d7700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0500b49a0 0x7fb0501472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb055bd6700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0500d0630 0x7fb05015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 -- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0500d0630 msgr2=0x7fb05015eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0500d0630 0x7fb05015eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 -- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0500b49a0 msgr2=0x7fb0501472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0500b49a0 0x7fb0501472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 -- 172.21.15.97:0/2725382161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c009550 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb0563d7700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0500b49a0 0x7fb0501472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb056bd8700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05015f060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb04800b9a0 tx=0x7fb04800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb046ffd700 1 -- 172.21.15.97:0/2725382161 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04800c9b0 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb046ffd700 1 -- 172.21.15.97:0/2725382161 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04800cb30 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb050156c80 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb046ffd700 1 -- 172.21.15.97:0/2725382161 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048014440 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.418+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb050157240 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.420+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0500753f0 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.420+0000 7fb046ffd700 1 -- 172.21.15.97:0/2725382161 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb04801d3b0 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.420+0000 7fb046ffd700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb03406aed0 0x7fb03406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.420+0000 7fb046ffd700 1 -- 172.21.15.97:0/2725382161 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb048006e40 con 0x7fb050152cc0 2023-11-24T06:09:35.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.420+0000 7fb0563d7700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb03406aed0 0x7fb03406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.421+0000 7fb0563d7700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb03406aed0 0x7fb03406d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009130 tx=0x7fb04c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:35.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.425+0000 7fb046ffd700 1 -- 172.21.15.97:0/2725382161 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb048054430 con 0x7fb050152cc0 2023-11-24T06:09:35.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.617+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0501576f0 con 0x7fb050152cc0 2023-11-24T06:09:35.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.618+0000 7fb046ffd700 1 -- 172.21.15.97:0/2725382161 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7fb048057a50 con 0x7fb050152cc0 2023-11-24T06:09:35.621 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:35.621 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:35.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb03406aed0 msgr2=0x7fb03406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb03406aed0 0x7fb03406d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009130 tx=0x7fb04c008030 comp rx=0 tx=0).stop 2023-11-24T06:09:35.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 msgr2=0x7fb05015f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05015f060 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb04800b9a0 tx=0x7fb04800bd60 comp rx=0 tx=0).stop 2023-11-24T06:09:35.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 shutdown_connections 2023-11-24T06:09:35.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050152cc0 0x7fb05015f060 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0500b49a0 0x7fb0501472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb03406aed0 0x7fb03406d390 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 --2- 172.21.15.97:0/2725382161 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb0500d0630 0x7fb05015eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 >> 172.21.15.97:0/2725382161 conn(0x7fb050071b90 msgr2=0x7fb05015ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:35.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 shutdown_connections 2023-11-24T06:09:35.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.620+0000 7fb05863b700 1 -- 172.21.15.97:0/2725382161 wait complete. 2023-11-24T06:09:35.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:35.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:35.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.756+0000 7f31af8ea700 1 Processor -- start 2023-11-24T06:09:35.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.757+0000 7f31af8ea700 1 -- start start 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.757+0000 7f31af8ea700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f31a8151c30 0x7f31a80ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.757+0000 7f31af8ea700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f31a80cf0b0 0x7f31a815e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31af8ea700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a80b55a0 con 0x7f31a806fab0 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31af8ea700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a8062900 con 0x7f31a8151c30 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31af8ea700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80b52c0 con 0x7f31a80cf0b0 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ad686700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f31a8151c30 0x7f31a80ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ace85700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f31a806fab0 0x7f31a80ce9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38702/0 (socket says 172.21.15.97:38702) 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ad686700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f31a8151c30 0x7f31a80ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46250/0 (socket says 172.21.15.97:46250) 2023-11-24T06:09:35.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ace85700 1 -- 172.21.15.97:0/3963806010 learned_addr learned my addr 172.21.15.97:0/3963806010 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ad686700 1 -- 172.21.15.97:0/3963806010 >> v2:172.21.15.183:3300/0 conn(0x7f31a80cf0b0 msgr2=0x7f31a815e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ad686700 1 --2- 172.21.15.97:0/3963806010 >> v2:172.21.15.183:3300/0 conn(0x7f31a80cf0b0 0x7f31a815e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ad686700 1 -- 172.21.15.97:0/3963806010 >> v1:172.21.15.97:6789/0 conn(0x7f31a806fab0 legacy=0x7f31a80ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f319ffff700 1 -- 172.21.15.97:0/3963806010 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96589501 0 0) 0x7f31a80b55a0 con 0x7f31a806fab0 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ad686700 1 -- 172.21.15.97:0/3963806010 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a8062340 con 0x7f31a8151c30 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f31ad686700 1 --2- 172.21.15.97:0/3963806010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a8151c30 0x7f31a80ccfb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31a80ce410 tx=0x7f3198009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910daecb6e6909a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f319ffff700 1 -- 172.21.15.97:0/3963806010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198011830 con 0x7f31a8151c30 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f319ffff700 1 -- 172.21.15.97:0/3963806010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3198011990 con 0x7f31a8151c30 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.758+0000 7f319ffff700 1 -- 172.21.15.97:0/3963806010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198011d20 con 0x7f31a8151c30 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.759+0000 7f31af8ea700 1 -- 172.21.15.97:0/3963806010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a8151c30 msgr2=0x7f31a80ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.759+0000 7f31af8ea700 1 --2- 172.21.15.97:0/3963806010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a8151c30 0x7f31a80ccfb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f31a80ce410 tx=0x7f3198009b20 comp rx=0 tx=0).stop 2023-11-24T06:09:35.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.759+0000 7f31af8ea700 1 -- 172.21.15.97:0/3963806010 shutdown_connections 2023-11-24T06:09:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.759+0000 7f31af8ea700 1 --2- 172.21.15.97:0/3963806010 >> v2:172.21.15.183:3300/0 conn(0x7f31a80cf0b0 0x7f31a815e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.759+0000 7f31af8ea700 1 --2- 172.21.15.97:0/3963806010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a8151c30 0x7f31a80ccfb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.759+0000 7f31af8ea700 1 -- 172.21.15.97:0/3963806010 >> 172.21.15.97:0/3963806010 conn(0x7f31a80b2050 msgr2=0x7f31a80b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.759+0000 7f31af8ea700 1 -- 172.21.15.97:0/3963806010 shutdown_connections 2023-11-24T06:09:35.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.759+0000 7f31af8ea700 1 -- 172.21.15.97:0/3963806010 wait complete. 2023-11-24T06:09:35.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.760+0000 7f31af8ea700 1 Processor -- start 2023-11-24T06:09:35.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.760+0000 7f31af8ea700 1 -- start start 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31af8ea700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806fab0 0x7f31a8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31af8ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a80cf0b0 0x7f31a8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31af8ea700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f31a8151c30 0x7f31a8157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31af8ea700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8154ac0 con 0x7f31a80cf0b0 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31af8ea700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8153e40 con 0x7f31a806fab0 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31af8ea700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a8154120 con 0x7f31a8151c30 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a80cf0b0 0x7f31a8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a80cf0b0 0x7f31a8159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46258/0 (socket says 172.21.15.97:46258) 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 -- 172.21.15.97:0/2419595344 learned_addr learned my addr 172.21.15.97:0/2419595344 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ad686700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806fab0 0x7f31a8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 -- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f31a8151c30 msgr2=0x7f31a8157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ade87700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f31a8151c30 0x7f31a8157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f31a8151c30 0x7f31a8157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 -- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806fab0 msgr2=0x7f31a8153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806fab0 0x7f31a8153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 -- 172.21.15.97:0/2419595344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31980094a0 con 0x7f31a80cf0b0 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ad686700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806fab0 0x7f31a8153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31ace85700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a80cf0b0 0x7f31a8159fd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31a400c520 tx=0x7f31a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f319dffb700 1 -- 172.21.15.97:0/2419595344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4004c80 con 0x7f31a80cf0b0 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f319dffb700 1 -- 172.21.15.97:0/2419595344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a4004de0 con 0x7f31a80cf0b0 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a8154400 con 0x7f31a80cf0b0 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.761+0000 7f319dffb700 1 -- 172.21.15.97:0/2419595344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40072d0 con 0x7f31a80cf0b0 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.762+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a814d060 con 0x7f31a80cf0b0 2023-11-24T06:09:35.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.763+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a8068d90 con 0x7f31a80cf0b0 2023-11-24T06:09:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.763+0000 7f319dffb700 1 -- 172.21.15.97:0/2419595344 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f31a4007430 con 0x7f31a80cf0b0 2023-11-24T06:09:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.763+0000 7f319dffb700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f318c06aed0 0x7f318c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.763+0000 7f319dffb700 1 -- 172.21.15.97:0/2419595344 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31a4089330 con 0x7f31a80cf0b0 2023-11-24T06:09:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.763+0000 7f31ad686700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f318c06aed0 0x7f318c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:35.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.764+0000 7f31ad686700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f318c06aed0 0x7f318c06d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31a80b4b80 tx=0x7f3198008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:35.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.768+0000 7f319dffb700 1 -- 172.21.15.97:0/2419595344 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f31a40543e0 con 0x7f31a80cf0b0 2023-11-24T06:09:35.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.956+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31a8068fb0 con 0x7f31a80cf0b0 2023-11-24T06:09:35.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.957+0000 7f319dffb700 1 -- 172.21.15.97:0/2419595344 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31a4057a00 con 0x7f31a80cf0b0 2023-11-24T06:09:35.960 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f318c06aed0 msgr2=0x7f318c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f318c06aed0 0x7f318c06d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31a80b4b80 tx=0x7f3198008030 comp rx=0 tx=0).stop 2023-11-24T06:09:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a80cf0b0 msgr2=0x7f31a8159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a80cf0b0 0x7f31a8159fd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f31a400c520 tx=0x7f31a400c8e0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 shutdown_connections 2023-11-24T06:09:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f318c06aed0 0x7f318c06d390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f31a8151c30 0x7f31a8157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806fab0 0x7f31a8153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 --2- 172.21.15.97:0/2419595344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a80cf0b0 0x7f31a8159fd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 >> 172.21.15.97:0/2419595344 conn(0x7f31a80b2050 msgr2=0x7f31a815b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 shutdown_connections 2023-11-24T06:09:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:35.959+0000 7f31af8ea700 1 -- 172.21.15.97:0/2419595344 wait complete. 2023-11-24T06:09:35.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:36.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.094+0000 7f79c16a5700 1 Processor -- start 2023-11-24T06:09:36.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.094+0000 7f79c16a5700 1 -- start start 2023-11-24T06:09:36.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79c16a5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f79bc15e160 0x7f79bc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79c16a5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79bc0afc90 0x7f79bc073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79c16a5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc0b3c60 con 0x7f79bc0744d0 2023-11-24T06:09:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79c16a5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc0b4500 con 0x7f79bc0afc90 2023-11-24T06:09:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79c16a5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc074940 con 0x7f79bc15e160 2023-11-24T06:09:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79ba7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79bc0afc90 0x7f79bc073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79bb7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f79bc0744d0 0x7f79bc06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38720/0 (socket says 172.21.15.97:38720) 2023-11-24T06:09:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79bb7fe700 1 -- 172.21.15.97:0/1079309998 learned_addr learned my addr 172.21.15.97:0/1079309998 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:36.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.095+0000 7f79baffd700 1 --2- 172.21.15.97:0/1079309998 >> v2:172.21.15.183:3300/0 conn(0x7f79bc15e160 0x7f79bc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79ba7fc700 1 -- 172.21.15.97:0/1079309998 >> v2:172.21.15.183:3300/0 conn(0x7f79bc15e160 msgr2=0x7f79bc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79ba7fc700 1 --2- 172.21.15.97:0/1079309998 >> v2:172.21.15.183:3300/0 conn(0x7f79bc15e160 0x7f79bc0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79ba7fc700 1 -- 172.21.15.97:0/1079309998 >> v1:172.21.15.97:6789/0 conn(0x7f79bc0744d0 legacy=0x7f79bc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79ba7fc700 1 -- 172.21.15.97:0/1079309998 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc0b58c0 con 0x7f79bc0afc90 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79b9ffb700 1 -- 172.21.15.97:0/1079309998 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196175756 0 0) 0x7f79bc0b3c60 con 0x7f79bc0744d0 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79ba7fc700 1 --2- 172.21.15.97:0/1079309998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc073f90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79bc09efa0 tx=0x7f79ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a493b40327e95c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79b9ffb700 1 -- 172.21.15.97:0/1079309998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac011830 con 0x7f79bc0afc90 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79b9ffb700 1 -- 172.21.15.97:0/1079309998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac011990 con 0x7f79bc0afc90 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79b9ffb700 1 -- 172.21.15.97:0/1079309998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac011d20 con 0x7f79bc0afc90 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79c16a5700 1 -- 172.21.15.97:0/1079309998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 msgr2=0x7f79bc073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79c16a5700 1 --2- 172.21.15.97:0/1079309998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc073f90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f79bc09efa0 tx=0x7f79ac009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:36.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79c16a5700 1 -- 172.21.15.97:0/1079309998 shutdown_connections 2023-11-24T06:09:36.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79c16a5700 1 --2- 172.21.15.97:0/1079309998 >> v2:172.21.15.183:3300/0 conn(0x7f79bc15e160 0x7f79bc0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79c16a5700 1 --2- 172.21.15.97:0/1079309998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc073f90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.096+0000 7f79c16a5700 1 -- 172.21.15.97:0/1079309998 >> 172.21.15.97:0/1079309998 conn(0x7f79bc0d0bf0 msgr2=0x7f79bc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:36.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.097+0000 7f79c16a5700 1 -- 172.21.15.97:0/1079309998 shutdown_connections 2023-11-24T06:09:36.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.097+0000 7f79c16a5700 1 -- 172.21.15.97:0/1079309998 wait complete. 2023-11-24T06:09:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.098+0000 7f79c16a5700 1 Processor -- start 2023-11-24T06:09:36.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.098+0000 7f79c16a5700 1 -- start start 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79c16a5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79bc0744d0 0x7f79bc159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79c16a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79c16a5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc15e160 0x7f79bc06e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79c16a5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1606d0 con 0x7f79bc0afc90 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79c16a5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc15fb50 con 0x7f79bc15e160 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79c16a5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc15fe30 con 0x7f79bc0744d0 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46272/0 (socket says 172.21.15.97:46272) 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 -- 172.21.15.97:0/448042746 learned_addr learned my addr 172.21.15.97:0/448042746 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 -- 172.21.15.97:0/448042746 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79bc0744d0 msgr2=0x7f79bc159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79bb7fe700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc15e160 0x7f79bc06e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79bc0744d0 0x7f79bc159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 -- 172.21.15.97:0/448042746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc15e160 msgr2=0x7f79bc06e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc15e160 0x7f79bc06e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 -- 172.21.15.97:0/448042746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0094a0 con 0x7f79bc0afc90 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79ba7fc700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc161020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f79ac000c00 tx=0x7f79ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79a37fe700 1 -- 172.21.15.97:0/448042746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac01aa90 con 0x7f79bc0afc90 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79a37fe700 1 -- 172.21.15.97:0/448042746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ac01abf0 con 0x7f79bc0afc90 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc160110 con 0x7f79bc0afc90 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79a37fe700 1 -- 172.21.15.97:0/448042746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0243f0 con 0x7f79bc0afc90 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.099+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc159e40 con 0x7f79bc0afc90 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.100+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79bc068fb0 con 0x7f79bc0afc90 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.101+0000 7f79a37fe700 1 -- 172.21.15.97:0/448042746 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79ac02d450 con 0x7f79bc0afc90 2023-11-24T06:09:36.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.101+0000 7f79a37fe700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79a806aed0 0x7f79a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.101+0000 7f79a37fe700 1 -- 172.21.15.97:0/448042746 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79ac08ab40 con 0x7f79bc0afc90 2023-11-24T06:09:36.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.101+0000 7f79baffd700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79a806aed0 0x7f79a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.101+0000 7f79baffd700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79a806aed0 0x7f79a806d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f79bc15a160 tx=0x7f79b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.106+0000 7f79a37fe700 1 -- 172.21.15.97:0/448042746 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79ac055c80 con 0x7f79bc0afc90 2023-11-24T06:09:36.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.295+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79bc06c140 con 0x7f79bc0afc90 2023-11-24T06:09:36.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.296+0000 7f79a37fe700 1 -- 172.21.15.97:0/448042746 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79ac006cc0 con 0x7f79bc0afc90 2023-11-24T06:09:36.299 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79a806aed0 msgr2=0x7f79a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79a806aed0 0x7f79a806d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f79bc15a160 tx=0x7f79b0008040 comp rx=0 tx=0).stop 2023-11-24T06:09:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 msgr2=0x7f79bc161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc161020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f79ac000c00 tx=0x7f79ac004750 comp rx=0 tx=0).stop 2023-11-24T06:09:36.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 shutdown_connections 2023-11-24T06:09:36.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79bc15e160 0x7f79bc06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79a806aed0 0x7f79a806d390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79bc0744d0 0x7f79bc159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 --2- 172.21.15.97:0/448042746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79bc0afc90 0x7f79bc161020 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 >> 172.21.15.97:0/448042746 conn(0x7f79bc0d0bf0 msgr2=0x7f79bc0b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:36.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 shutdown_connections 2023-11-24T06:09:36.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.298+0000 7f79c16a5700 1 -- 172.21.15.97:0/448042746 wait complete. 2023-11-24T06:09:36.313 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:36.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.433+0000 7f0c4bfff700 1 Processor -- start 2023-11-24T06:09:36.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.434+0000 7f0c4bfff700 1 -- start start 2023-11-24T06:09:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.434+0000 7f0c4bfff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44070b40 0x7f0c4415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.434+0000 7f0c4bfff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c440b2b00 con 0x7f0c44151660 2023-11-24T06:09:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.434+0000 7f0c4bfff700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4406e3a0 con 0x7f0c4406dec0 2023-11-24T06:09:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.434+0000 7f0c4bfff700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4406ee40 con 0x7f0c44070b40 2023-11-24T06:09:36.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.434+0000 7f0c4959a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f0c44151660 0x7f0c440cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38730/0 (socket says 172.21.15.97:38730) 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.434+0000 7f0c4959a700 1 -- 172.21.15.97:0/3663954521 learned_addr learned my addr 172.21.15.97:0/3663954521 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787917262 0 0) 0x7f0c440b2b00 con 0x7f0c44151660 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c300035b0 con 0x7f0c44151660 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c4a59c700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44070b40 0x7f0c4415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 345343879 0 0) 0x7f0c300035b0 con 0x7f0c44151660 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44070b40 msgr2=0x7f0c4415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44070b40 0x7f0c4415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 >> v1:172.21.15.183:6789/0 conn(0x7f0c4406dec0 legacy=0x7f0c440cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c440b2da0 con 0x7f0c44151660 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f0c40001670 con 0x7f0c44151660 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 >> v1:172.21.15.97:6789/0 conn(0x7f0c44151660 legacy=0x7f0c440cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30003a30 0x7f0c30003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30004850 0x7f0c30004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44070b40 0x7f0c30005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4406ee40 con 0x7f0c30003a30 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3000baf0 con 0x7f0c30004850 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3000dec0 con 0x7f0c44070b40 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c49d9b700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30003a30 0x7f0c30003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c4a59c700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44070b40 0x7f0c30005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c4959a700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30004850 0x7f0c30004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c49d9b700 1 -- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44070b40 msgr2=0x7f0c30005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c49d9b700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44070b40 0x7f0c30005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c49d9b700 1 -- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30004850 msgr2=0x7f0c30004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c49d9b700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30004850 0x7f0c30004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c49d9b700 1 -- 172.21.15.97:0/3663954521 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c300035b0 con 0x7f0c30003a30 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.435+0000 7f0c4959a700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30004850 0x7f0c30004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c49d9b700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30003a30 0x7f0c30003e50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c34007fd0 tx=0x7f0c30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcf1c85b9794e90 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c340012f0 con 0x7f0c30003a30 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c340052f0 con 0x7f0c30003a30 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c48d99700 1 -- 172.21.15.97:0/3663954521 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34006860 con 0x7f0c30003a30 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c4bfff700 1 -- 172.21.15.97:0/3663954521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30003a30 msgr2=0x7f0c30003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c4bfff700 1 --2- 172.21.15.97:0/3663954521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c30003a30 0x7f0c30003e50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c34007fd0 tx=0x7f0c30005880 comp rx=0 tx=0).stop 2023-11-24T06:09:36.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c4a59c700 1 -- 172.21.15.97:0/3663954521 reap_dead start 2023-11-24T06:09:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c4bfff700 1 -- 172.21.15.97:0/3663954521 shutdown_connections 2023-11-24T06:09:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c4bfff700 1 -- 172.21.15.97:0/3663954521 >> 172.21.15.97:0/3663954521 conn(0x7f0c4406b410 msgr2=0x7f0c4406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c4bfff700 1 -- 172.21.15.97:0/3663954521 shutdown_connections 2023-11-24T06:09:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.436+0000 7f0c4bfff700 1 -- 172.21.15.97:0/3663954521 wait complete. 2023-11-24T06:09:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4bfff700 1 Processor -- start 2023-11-24T06:09:36.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4bfff700 1 -- start start 2023-11-24T06:09:36.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4bfff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30003a30 0x7f0c4414d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4bfff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44157b90 0x7f0c4414b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c4415bbe0 0x7f0c4415c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4bfff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4415c5a0 con 0x7f0c4415bbe0 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4bfff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4414ce70 con 0x7f0c30003a30 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4bfff700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4414c260 con 0x7f0c44157b90 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c4415bbe0 0x7f0c4415c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c4415bbe0 0x7f0c4415c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46292/0 (socket says 172.21.15.97:46292) 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.438+0000 7f0c4a59c700 1 -- 172.21.15.97:0/126521970 learned_addr learned my addr 172.21.15.97:0/126521970 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4959a700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44157b90 0x7f0c4414b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c49d9b700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30003a30 0x7f0c4414d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4a59c700 1 -- 172.21.15.97:0/126521970 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44157b90 msgr2=0x7f0c4414b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4a59c700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44157b90 0x7f0c4414b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4a59c700 1 -- 172.21.15.97:0/126521970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30003a30 msgr2=0x7f0c4414d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4a59c700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30003a30 0x7f0c4414d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4a59c700 1 -- 172.21.15.97:0/126521970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4414c540 con 0x7f0c4415bbe0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4a59c700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c4415bbe0 0x7f0c4415c060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c440b5110 tx=0x7f0c3c00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c3a7fc700 1 -- 172.21.15.97:0/126521970 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c00e040 con 0x7f0c4415bbe0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c3a7fc700 1 -- 172.21.15.97:0/126521970 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c00f040 con 0x7f0c4415bbe0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4414c820 con 0x7f0c4415bbe0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c3a7fc700 1 -- 172.21.15.97:0/126521970 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c015430 con 0x7f0c4415bbe0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.439+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4414d840 con 0x7f0c4415bbe0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.440+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c44068480 con 0x7f0c4415bbe0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.440+0000 7f0c3a7fc700 1 -- 172.21.15.97:0/126521970 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c3c015590 con 0x7f0c4415bbe0 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.441+0000 7f0c3a7fc700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c2806aed0 0x7f0c2806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.441+0000 7f0c3a7fc700 1 -- 172.21.15.97:0/126521970 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c3c089b50 con 0x7f0c4415bbe0 2023-11-24T06:09:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.441+0000 7f0c49d9b700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c2806aed0 0x7f0c2806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.441+0000 7f0c49d9b700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c2806aed0 0x7f0c2806d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c34014040 tx=0x7f0c34011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.445+0000 7f0c3a7fc700 1 -- 172.21.15.97:0/126521970 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c3c054c00 con 0x7f0c4415bbe0 2023-11-24T06:09:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.634+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c44068fb0 con 0x7f0c4415bbe0 2023-11-24T06:09:36.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.634+0000 7f0c3a7fc700 1 -- 172.21.15.97:0/126521970 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0c3c058220 con 0x7f0c4415bbe0 2023-11-24T06:09:36.638 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:36.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c2806aed0 msgr2=0x7f0c2806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c2806aed0 0x7f0c2806d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c34014040 tx=0x7f0c34011000 comp rx=0 tx=0).stop 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c4415bbe0 msgr2=0x7f0c4415c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c4415bbe0 0x7f0c4415c060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0c440b5110 tx=0x7f0c3c00f490 comp rx=0 tx=0).stop 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 shutdown_connections 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c4415bbe0 0x7f0c4415c060 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c30003a30 0x7f0c4414d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c2806aed0 0x7f0c2806d390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 --2- 172.21.15.97:0/126521970 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c44157b90 0x7f0c4414b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 >> 172.21.15.97:0/126521970 conn(0x7f0c4406b410 msgr2=0x7f0c4414e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:36.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 shutdown_connections 2023-11-24T06:09:36.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.636+0000 7f0c4bfff700 1 -- 172.21.15.97:0/126521970 wait complete. 2023-11-24T06:09:36.651 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:09:36.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.772+0000 7f1a72163700 1 Processor -- start 2023-11-24T06:09:36.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.772+0000 7f1a72163700 1 -- start start 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.772+0000 7f1a72163700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c15bdd0 0x7f1a6c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.772+0000 7f1a72163700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a6c0d1c70 0x7f1a6c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.772+0000 7f1a72163700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0b22d0 con 0x7f1a6c0b3370 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.772+0000 7f1a72163700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b2f50 con 0x7f1a6c0d1c70 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.772+0000 7f1a72163700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0d2640 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.772+0000 7f1a6bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a6c0d1c70 0x7f1a6c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a6c0d1c70 0x7f1a6c0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46300/0 (socket says 172.21.15.97:46300) 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6bfff700 1 -- 172.21.15.97:0/1537527188 learned_addr learned my addr 172.21.15.97:0/1537527188 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:36.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6b7fe700 1 --2- 172.21.15.97:0/1537527188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c15bdd0 0x7f1a6c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6bfff700 1 -- 172.21.15.97:0/1537527188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c15bdd0 msgr2=0x7f1a6c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6bfff700 1 --2- 172.21.15.97:0/1537527188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c15bdd0 0x7f1a6c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6a7fc700 1 -- 172.21.15.97:0/1537527188 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243573096 0 0) 0x7f1a6c0b22d0 con 0x7f1a6c0b3370 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6bfff700 1 -- 172.21.15.97:0/1537527188 >> v1:172.21.15.97:6789/0 conn(0x7f1a6c0b3370 legacy=0x7f1a6c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6bfff700 1 -- 172.21.15.97:0/1537527188 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0b25b0 con 0x7f1a6c0d1c70 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6b7fe700 1 --2- 172.21.15.97:0/1537527188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c15bdd0 0x7f1a6c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6bfff700 1 --2- 172.21.15.97:0/1537527188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d1c70 0x7f1a6c0d2100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009e50 tx=0x7f1a6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57e3f6ed0f78e91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6a7fc700 1 -- 172.21.15.97:0/1537527188 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400e070 con 0x7f1a6c0d1c70 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6a7fc700 1 -- 172.21.15.97:0/1537527188 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6400b9c0 con 0x7f1a6c0d1c70 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.773+0000 7f1a6a7fc700 1 -- 172.21.15.97:0/1537527188 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400bd70 con 0x7f1a6c0d1c70 2023-11-24T06:09:36.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.774+0000 7f1a72163700 1 -- 172.21.15.97:0/1537527188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d1c70 msgr2=0x7f1a6c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.774+0000 7f1a72163700 1 --2- 172.21.15.97:0/1537527188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d1c70 0x7f1a6c0d2100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009e50 tx=0x7f1a6400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:36.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.774+0000 7f1a72163700 1 -- 172.21.15.97:0/1537527188 shutdown_connections 2023-11-24T06:09:36.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.774+0000 7f1a72163700 1 --2- 172.21.15.97:0/1537527188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c0d1c70 0x7f1a6c0d2100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.774+0000 7f1a72163700 1 --2- 172.21.15.97:0/1537527188 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c15bdd0 0x7f1a6c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.774+0000 7f1a72163700 1 -- 172.21.15.97:0/1537527188 >> 172.21.15.97:0/1537527188 conn(0x7f1a6c0731a0 msgr2=0x7f1a6c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:36.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.774+0000 7f1a72163700 1 -- 172.21.15.97:0/1537527188 shutdown_connections 2023-11-24T06:09:36.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.774+0000 7f1a72163700 1 -- 172.21.15.97:0/1537527188 wait complete. 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.775+0000 7f1a72163700 1 Processor -- start 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.775+0000 7f1a72163700 1 -- start start 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a72163700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c0b3370 0x7f1a6c1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a72163700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c0d1c70 0x7f1a6c14e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a72163700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c15bdd0 0x7f1a6c0c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a72163700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c06be80 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a72163700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c06aff0 con 0x7f1a6c0b3370 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a72163700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c06b200 con 0x7f1a6c0d1c70 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c15bdd0 0x7f1a6c0c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c15bdd0 0x7f1a6c0c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46306/0 (socket says 172.21.15.97:46306) 2023-11-24T06:09:36.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 -- 172.21.15.97:0/11913086 learned_addr learned my addr 172.21.15.97:0/11913086 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:36.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6affd700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c0d1c70 0x7f1a6c14e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6b7fe700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c0b3370 0x7f1a6c1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 -- 172.21.15.97:0/11913086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c0d1c70 msgr2=0x7f1a6c14e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c0d1c70 0x7f1a6c14e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 -- 172.21.15.97:0/11913086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c0b3370 msgr2=0x7f1a6c1584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c0b3370 0x7f1a6c1584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 -- 172.21.15.97:0/11913086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64009b30 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a6bfff700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c15bdd0 0x7f1a6c0c81b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0716d0 tx=0x7f1a64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.776+0000 7f1a63fff700 1 -- 172.21.15.97:0/11913086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64004880 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.777+0000 7f1a63fff700 1 -- 172.21.15.97:0/11913086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a640092e0 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.777+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c06b4e0 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.777+0000 7f1a63fff700 1 -- 172.21.15.97:0/11913086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6401c680 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.777+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c06baa0 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.777+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c067f60 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.778+0000 7f1a63fff700 1 -- 172.21.15.97:0/11913086 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a6401c7e0 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.778+0000 7f1a63fff700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1a4c06aed0 0x7f1a4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.778+0000 7f1a6b7fe700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1a4c06aed0 0x7f1a4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:36.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.778+0000 7f1a63fff700 1 -- 172.21.15.97:0/11913086 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a6408b260 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.779+0000 7f1a6b7fe700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1a4c06aed0 0x7f1a4c06d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c006fd0 tx=0x7f1a5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:36.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.783+0000 7f1a63fff700 1 -- 172.21.15.97:0/11913086 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a640563a0 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.991+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1a6c0b7c60 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.994 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.991+0000 7f1a63fff700 1 -- 172.21.15.97:0/11913086 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1a640599c0 con 0x7f1a6c15bdd0 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1a4c06aed0 msgr2=0x7f1a4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1a4c06aed0 0x7f1a4c06d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c006fd0 tx=0x7f1a5c008040 comp rx=0 tx=0).stop 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c15bdd0 msgr2=0x7f1a6c0c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c15bdd0 0x7f1a6c0c81b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0716d0 tx=0x7f1a64004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 shutdown_connections 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a6c15bdd0 0x7f1a6c0c81b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a6c0b3370 0x7f1a6c1584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1a4c06aed0 0x7f1a4c06d390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 --2- 172.21.15.97:0/11913086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a6c0d1c70 0x7f1a6c14e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:36.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 >> 172.21.15.97:0/11913086 conn(0x7f1a6c0731a0 msgr2=0x7f1a6c0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:36.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 shutdown_connections 2023-11-24T06:09:36.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:36.993+0000 7f1a72163700 1 -- 172.21.15.97:0/11913086 wait complete. 2023-11-24T06:09:36.998 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:09:37.008 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:09:37.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:37.009 DEBUG:teuthology.orchestra.run.smithi183:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-24T06:09:37.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:09:37.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:37.063 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:09:37.093 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:09:37.093 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:37.111 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:09:37.111 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:37.168 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:09:37.168 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:37.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.383+0000 7f1ccfd49700 1 Processor -- start 2023-11-24T06:09:37.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.383+0000 7f1ccfd49700 1 -- start start 2023-11-24T06:09:37.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1ccfd49700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 0x7f1cc80d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1ccfd49700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80d1210 con 0x7f1cc80b6ef0 2023-11-24T06:09:37.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1ccfd49700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80d20b0 con 0x7f1cc815e1e0 2023-11-24T06:09:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1ccfd49700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80d3150 con 0x7f1cc80d0870 2023-11-24T06:09:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cce2e6700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1cc80b6ef0 0x7f1cc80b7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38766/0 (socket says 172.21.15.97:38766) 2023-11-24T06:09:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cce2e6700 1 -- 172.21.15.97:0/2936668056 learned_addr learned my addr 172.21.15.97:0/2936668056 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1ccd2e4700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 0x7f1cc80d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367122237 0 0) 0x7f1cc80d1210 con 0x7f1cc80b6ef0 2023-11-24T06:09:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb00035b0 con 0x7f1cc80b6ef0 2023-11-24T06:09:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 493875331 0 0) 0x7f1cb00035b0 con 0x7f1cc80b6ef0 2023-11-24T06:09:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 msgr2=0x7f1cc80d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cccae3700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 0x7f1cc80d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 >> v1:172.21.15.183:6789/0 conn(0x7f1cc815e1e0 legacy=0x7f1cc80d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80d2390 con 0x7f1cc80b6ef0 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.384+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f1cb8001670 con 0x7f1cc80b6ef0 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 >> v1:172.21.15.97:6789/0 conn(0x7f1cc80b6ef0 legacy=0x7f1cc80b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0003a30 0x7f1cb0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 0x7f1cb0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0004d90 0x7f1cb0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80d3150 con 0x7f1cb0003a30 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb000ba10 con 0x7f1cb0004d90 2023-11-24T06:09:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb000bbd0 con 0x7f1cc80d0870 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccdae5700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0003a30 0x7f1cb0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cce2e6700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0004d90 0x7f1cb0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccd2e4700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 0x7f1cb0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccdae5700 1 -- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 msgr2=0x7f1cb0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccdae5700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 0x7f1cb0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccdae5700 1 -- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0004d90 msgr2=0x7f1cb0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccdae5700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0004d90 0x7f1cb0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccdae5700 1 -- 172.21.15.97:0/2936668056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00035b0 con 0x7f1cb0003a30 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccd2e4700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc80d0870 0x7f1cb0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1ccdae5700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0003a30 0x7f1cb0003e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c290 tx=0x7f1cb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd198a883b48b41e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0001e70 con 0x7f1cb0003a30 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc000cd70 con 0x7f1cb0003a30 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.385+0000 7f1cccae3700 1 -- 172.21.15.97:0/2936668056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0012660 con 0x7f1cb0003a30 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.386+0000 7f1ccfd49700 1 -- 172.21.15.97:0/2936668056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0003a30 msgr2=0x7f1cb0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.386+0000 7f1ccfd49700 1 --2- 172.21.15.97:0/2936668056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb0003a30 0x7f1cb0003e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000c290 tx=0x7f1cb0005880 comp rx=0 tx=0).stop 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.386+0000 7f1cce2e6700 1 -- 172.21.15.97:0/2936668056 reap_dead start 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.386+0000 7f1ccfd49700 1 -- 172.21.15.97:0/2936668056 shutdown_connections 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.386+0000 7f1ccfd49700 1 -- 172.21.15.97:0/2936668056 >> 172.21.15.97:0/2936668056 conn(0x7f1cc80b5b50 msgr2=0x7f1cc80b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.386+0000 7f1ccfd49700 1 -- 172.21.15.97:0/2936668056 shutdown_connections 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.386+0000 7f1ccfd49700 1 -- 172.21.15.97:0/2936668056 wait complete. 2023-11-24T06:09:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.387+0000 7f1ccfd49700 1 Processor -- start 2023-11-24T06:09:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.388+0000 7f1ccfd49700 1 -- start start 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.388+0000 7f1ccfd49700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0003a30 0x7f1cc8159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.388+0000 7f1ccfd49700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc815f790 0x7f1cc8157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.388+0000 7f1ccfd49700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc81f2900 0x7f1cc81f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.388+0000 7f1ccfd49700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8160aa0 con 0x7f1cc81f2900 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.388+0000 7f1ccfd49700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc815fc10 con 0x7f1cb0003a30 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.388+0000 7f1ccfd49700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc815fe90 con 0x7f1cc815f790 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc81f2900 0x7f1cc81f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc81f2900 0x7f1cc81f2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46326/0 (socket says 172.21.15.97:46326) 2023-11-24T06:09:37.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 -- 172.21.15.97:0/3288325421 learned_addr learned my addr 172.21.15.97:0/3288325421 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1ccdae5700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0003a30 0x7f1cc8159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1ccd2e4700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc815f790 0x7f1cc8157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 -- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc815f790 msgr2=0x7f1cc8157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc815f790 0x7f1cc8157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 -- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0003a30 msgr2=0x7f1cc8159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0003a30 0x7f1cc8159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 -- 172.21.15.97:0/3288325421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8160170 con 0x7f1cc81f2900 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1ccdae5700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0003a30 0x7f1cc8159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cce2e6700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc81f2900 0x7f1cc81f2d80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80b6d60 tx=0x7f1cb80073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cc67fc700 1 -- 172.21.15.97:0/3288325421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8017070 con 0x7f1cc81f2900 2023-11-24T06:09:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cc67fc700 1 -- 172.21.15.97:0/3288325421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8011580 con 0x7f1cc81f2900 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc8160450 con 0x7f1cc81f2900 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1cc67fc700 1 -- 172.21.15.97:0/3288325421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8011770 con 0x7f1cc81f2900 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.389+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc8160f30 con 0x7f1cc81f2900 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.390+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc8160d80 con 0x7f1cc81f2900 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.391+0000 7f1cc67fc700 1 -- 172.21.15.97:0/3288325421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cb80118d0 con 0x7f1cc81f2900 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.391+0000 7f1cc67fc700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cb406aed0 0x7f1cb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.391+0000 7f1cc67fc700 1 -- 172.21.15.97:0/3288325421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cb8088de0 con 0x7f1cc81f2900 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.391+0000 7f1ccdae5700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cb406aed0 0x7f1cb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.391+0000 7f1ccdae5700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cb406aed0 0x7f1cb406d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0014040 tx=0x7f1cc0005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:37.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.395+0000 7f1cc67fc700 1 -- 172.21.15.97:0/3288325421 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cb8053f20 con 0x7f1cc81f2900 2023-11-24T06:09:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.590+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cc8160be0 con 0x7f1cc81f2900 2023-11-24T06:09:37.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.592+0000 7f1cc67fc700 1 -- 172.21.15.97:0/3288325421 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16517 (secure 0 0 0) 0x7f1cc8160be0 con 0x7f1cc81f2900 2023-11-24T06:09:37.595 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:37.595 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:08:59.513299+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:37.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cb406aed0 msgr2=0x7f1cb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cb406aed0 0x7f1cb406d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0014040 tx=0x7f1cc0005040 comp rx=0 tx=0).stop 2023-11-24T06:09:37.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc81f2900 msgr2=0x7f1cc81f2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc81f2900 0x7f1cc81f2d80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80b6d60 tx=0x7f1cb80073f0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 shutdown_connections 2023-11-24T06:09:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cc81f2900 0x7f1cc81f2d80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cb0003a30 0x7f1cc8159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cb406aed0 0x7f1cb406d390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 --2- 172.21.15.97:0/3288325421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1cc815f790 0x7f1cc8157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 >> 172.21.15.97:0/3288325421 conn(0x7f1cc80b5b50 msgr2=0x7f1cc80b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 shutdown_connections 2023-11-24T06:09:37.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.594+0000 7f1ccfd49700 1 -- 172.21.15.97:0/3288325421 wait complete. 2023-11-24T06:09:37.609 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:37.610 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:37.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.730+0000 7f7e4a8de700 1 Processor -- start 2023-11-24T06:09:37.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.730+0000 7f7e4a8de700 1 -- start start 2023-11-24T06:09:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.730+0000 7f7e4a8de700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 0x7f7e4406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.730+0000 7f7e4a8de700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e440b5730 0x7f7e440b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e4a8de700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7e4415f8f0 0x7f7e4415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e4a8de700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e440b60d0 con 0x7f7e440b5730 2023-11-24T06:09:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e4a8de700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4406b750 con 0x7f7e4415f8f0 2023-11-24T06:09:37.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e4a8de700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44160870 con 0x7f7e44145720 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e440b5730 0x7f7e440b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7e440b5730 0x7f7e440b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46330/0 (socket says 172.21.15.97:46330) 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 -- 172.21.15.97:0/3300087523 learned_addr learned my addr 172.21.15.97:0/3300087523 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e43fff700 1 --2- 172.21.15.97:0/3300087523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 0x7f7e4406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e48e7b700 1 --2- 172.21.15.97:0/3300087523 >> v2:172.21.15.183:3300/0 conn(0x7f7e4415f8f0 0x7f7e4415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 -- 172.21.15.97:0/3300087523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 msgr2=0x7f7e4406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 --2- 172.21.15.97:0/3300087523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 0x7f7e4406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 -- 172.21.15.97:0/3300087523 >> v2:172.21.15.183:3300/0 conn(0x7f7e4415f8f0 msgr2=0x7f7e4415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:37.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 --2- 172.21.15.97:0/3300087523 >> v2:172.21.15.183:3300/0 conn(0x7f7e4415f8f0 0x7f7e4415fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 -- 172.21.15.97:0/3300087523 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e441602c0 con 0x7f7e440b5730 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e48e7b700 1 --2- 172.21.15.97:0/3300087523 >> v2:172.21.15.183:3300/0 conn(0x7f7e4415f8f0 0x7f7e4415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e437fe700 1 --2- 172.21.15.97:0/3300087523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e440b5730 0x7f7e440b5b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e44072d00 tx=0x7f7e38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb0eb606798d9b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e42ffd700 1 -- 172.21.15.97:0/3300087523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800b870 con 0x7f7e440b5730 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e42ffd700 1 -- 172.21.15.97:0/3300087523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3800b9d0 con 0x7f7e440b5730 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.731+0000 7f7e42ffd700 1 -- 172.21.15.97:0/3300087523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800bd60 con 0x7f7e440b5730 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3300087523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e440b5730 msgr2=0x7f7e440b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3300087523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e440b5730 0x7f7e440b5b90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e44072d00 tx=0x7f7e38009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3300087523 shutdown_connections 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3300087523 >> v2:172.21.15.183:3300/0 conn(0x7f7e4415f8f0 0x7f7e4415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3300087523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 0x7f7e4406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3300087523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e440b5730 0x7f7e440b5b90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3300087523 >> 172.21.15.97:0/3300087523 conn(0x7f7e440d0be0 msgr2=0x7f7e440d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3300087523 shutdown_connections 2023-11-24T06:09:37.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.732+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3300087523 wait complete. 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.733+0000 7f7e4a8de700 1 Processor -- start 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.733+0000 7f7e4a8de700 1 -- start start 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e4a8de700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e440b5730 0x7f7e4415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e4a8de700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 0x7f7e4406eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e4a8de700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4415f8f0 0x7f7e4415d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e4a8de700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e441f3a00 con 0x7f7e4415f8f0 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e4a8de700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e441f3ce0 con 0x7f7e440b5730 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e4a8de700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e441f3fc0 con 0x7f7e44145720 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4415f8f0 0x7f7e4415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4415f8f0 0x7f7e4415d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46340/0 (socket says 172.21.15.97:46340) 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 -- 172.21.15.97:0/3386610673 learned_addr learned my addr 172.21.15.97:0/3386610673 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e437fe700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 0x7f7e4406eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e43fff700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e440b5730 0x7f7e4415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 -- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 msgr2=0x7f7e4406eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 0x7f7e4406eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 -- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e440b5730 msgr2=0x7f7e4415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e440b5730 0x7f7e4415bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 -- 172.21.15.97:0/3386610673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e38009550 con 0x7f7e4415f8f0 2023-11-24T06:09:37.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e43fff700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e440b5730 0x7f7e4415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.734+0000 7f7e48e7b700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4415f8f0 0x7f7e4415d890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c007df0 tx=0x7f7e3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.735+0000 7f7e40ff9700 1 -- 172.21.15.97:0/3386610673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c00ddd0 con 0x7f7e4415f8f0 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.735+0000 7f7e40ff9700 1 -- 172.21.15.97:0/3386610673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c00f040 con 0x7f7e4415f8f0 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.735+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e441f42a0 con 0x7f7e4415f8f0 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.735+0000 7f7e40ff9700 1 -- 172.21.15.97:0/3386610673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c003c60 con 0x7f7e4415f8f0 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.735+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e441f3660 con 0x7f7e4415f8f0 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.736+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e44068fb0 con 0x7f7e4415f8f0 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.736+0000 7f7e40ff9700 1 -- 172.21.15.97:0/3386610673 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e3c004030 con 0x7f7e4415f8f0 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.736+0000 7f7e40ff9700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7e3006aed0 0x7f7e3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:37.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.736+0000 7f7e40ff9700 1 -- 172.21.15.97:0/3386610673 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e3c0896a0 con 0x7f7e4415f8f0 2023-11-24T06:09:37.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.736+0000 7f7e43fff700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7e3006aed0 0x7f7e3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:37.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.737+0000 7f7e43fff700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7e3006aed0 0x7f7e3006d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009970 tx=0x7f7e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:37.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.741+0000 7f7e40ff9700 1 -- 172.21.15.97:0/3386610673 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e3c054790 con 0x7f7e4415f8f0 2023-11-24T06:09:37.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.932+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e441f3b40 con 0x7f7e4415f8f0 2023-11-24T06:09:37.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.932+0000 7f7e40ff9700 1 -- 172.21.15.97:0/3386610673 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e441f3b40 con 0x7f7e4415f8f0 2023-11-24T06:09:37.935 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:37.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7e3006aed0 msgr2=0x7f7e3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7e3006aed0 0x7f7e3006d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009970 tx=0x7f7e34008040 comp rx=0 tx=0).stop 2023-11-24T06:09:37.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4415f8f0 msgr2=0x7f7e4415d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4415f8f0 0x7f7e4415d890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c007df0 tx=0x7f7e3c00d3b0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 shutdown_connections 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e4415f8f0 0x7f7e4415d890 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e440b5730 0x7f7e4415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7e3006aed0 0x7f7e3006d390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 --2- 172.21.15.97:0/3386610673 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7e44145720 0x7f7e4406eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 >> 172.21.15.97:0/3386610673 conn(0x7f7e440d0be0 msgr2=0x7f7e440b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 shutdown_connections 2023-11-24T06:09:37.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:37.934+0000 7f7e4a8de700 1 -- 172.21.15.97:0/3386610673 wait complete. 2023-11-24T06:09:37.949 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:38.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.069+0000 7feed6f5e700 1 Processor -- start 2023-11-24T06:09:38.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.070+0000 7feed6f5e700 1 -- start start 2023-11-24T06:09:38.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.070+0000 7feed6f5e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7feed015be20 0x7feed00af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.070+0000 7feed6f5e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed00b4970 con 0x7feed00cdf50 2023-11-24T06:09:38.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.070+0000 7feed6f5e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed00cfed0 con 0x7feed00b4e00 2023-11-24T06:09:38.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.070+0000 7feed6f5e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed00b4360 con 0x7feed015be20 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feed54fb700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7feed00cdf50 0x7feed015c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38800/0 (socket says 172.21.15.97:38800) 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feed54fb700 1 -- 172.21.15.97:0/2543204820 learned_addr learned my addr 172.21.15.97:0/2543204820 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feed4cfa700 1 --2- 172.21.15.97:0/2543204820 >> v2:172.21.15.183:3300/0 conn(0x7feed015be20 0x7feed00af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feecf7fe700 1 -- 172.21.15.97:0/2543204820 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649378141 0 0) 0x7feed00b4970 con 0x7feed00cdf50 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feecf7fe700 1 -- 172.21.15.97:0/2543204820 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec00035b0 con 0x7feed00cdf50 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feed4cfa700 1 -- 172.21.15.97:0/2543204820 >> v1:172.21.15.183:6789/0 conn(0x7feed00b4e00 legacy=0x7feed00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feed4cfa700 1 -- 172.21.15.97:0/2543204820 >> v1:172.21.15.97:6789/0 conn(0x7feed00cdf50 legacy=0x7feed015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feecf7fe700 1 -- 172.21.15.97:0/2543204820 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1210149121 0 0) 0x7feec00035b0 con 0x7feed00cdf50 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feed4cfa700 1 -- 172.21.15.97:0/2543204820 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed006f940 con 0x7feed015be20 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feed4cfa700 1 --2- 172.21.15.97:0/2543204820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 0x7feed00af790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feec4009840 tx=0x7feec4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a51c25632163638 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feecf7fe700 1 -- 172.21.15.97:0/2543204820 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80019f0 con 0x7feed015be20 2023-11-24T06:09:38.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feecf7fe700 1 -- 172.21.15.97:0/2543204820 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4004030 con 0x7feed015be20 2023-11-24T06:09:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.071+0000 7feecf7fe700 1 -- 172.21.15.97:0/2543204820 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40043c0 con 0x7feed015be20 2023-11-24T06:09:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.072+0000 7feed6f5e700 1 -- 172.21.15.97:0/2543204820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 msgr2=0x7feed00af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.072+0000 7feed6f5e700 1 --2- 172.21.15.97:0/2543204820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 0x7feed00af790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feec4009840 tx=0x7feec4009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.072+0000 7feed6f5e700 1 -- 172.21.15.97:0/2543204820 shutdown_connections 2023-11-24T06:09:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.072+0000 7feed6f5e700 1 --2- 172.21.15.97:0/2543204820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 0x7feed00af790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.072+0000 7feed6f5e700 1 -- 172.21.15.97:0/2543204820 >> 172.21.15.97:0/2543204820 conn(0x7feed00b2050 msgr2=0x7feed00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.072+0000 7feed6f5e700 1 -- 172.21.15.97:0/2543204820 shutdown_connections 2023-11-24T06:09:38.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.072+0000 7feed6f5e700 1 -- 172.21.15.97:0/2543204820 wait complete. 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.073+0000 7feed6f5e700 1 Processor -- start 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.073+0000 7feed6f5e700 1 -- start start 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed6f5e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed00b4e00 0x7feed01522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed6f5e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feed00cdf50 0x7feed014dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed6f5e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 0x7feed015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed6f5e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed014e940 con 0x7feed00b4e00 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed6f5e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00ad930 con 0x7feed015be20 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed6f5e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed0151270 con 0x7feed00cdf50 2023-11-24T06:09:38.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed4cfa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed00b4e00 0x7feed01522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed4cfa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed00b4e00 0x7feed01522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46350/0 (socket says 172.21.15.97:46350) 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed4cfa700 1 -- 172.21.15.97:0/241383480 learned_addr learned my addr 172.21.15.97:0/241383480 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed54fb700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 0x7feed015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feecffff700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feed00cdf50 0x7feed014dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed4cfa700 1 -- 172.21.15.97:0/241383480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feed00cdf50 msgr2=0x7feed014dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed4cfa700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feed00cdf50 0x7feed014dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed4cfa700 1 -- 172.21.15.97:0/241383480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 msgr2=0x7feed015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed4cfa700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 0x7feed015dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.074+0000 7feed4cfa700 1 -- 172.21.15.97:0/241383480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40094a0 con 0x7feed00b4e00 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.075+0000 7feed4cfa700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed00b4e00 0x7feed01522b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feec400a010 tx=0x7feec4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.075+0000 7feecd7fa700 1 -- 172.21.15.97:0/241383480 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4011630 con 0x7feed00b4e00 2023-11-24T06:09:38.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.075+0000 7feecd7fa700 1 -- 172.21.15.97:0/241383480 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec401f070 con 0x7feed00b4e00 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.075+0000 7feecd7fa700 1 -- 172.21.15.97:0/241383480 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40119d0 con 0x7feed00b4e00 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.075+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed0151510 con 0x7feed00b4e00 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.075+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed0151ad0 con 0x7feed00b4e00 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.076+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed0068d90 con 0x7feed00b4e00 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.076+0000 7feecd7fa700 1 -- 172.21.15.97:0/241383480 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feec4011b30 con 0x7feed00b4e00 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.076+0000 7feecd7fa700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feeb406aed0 0x7feeb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.076+0000 7feecffff700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feeb406aed0 0x7feeb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.076+0000 7feecd7fa700 1 -- 172.21.15.97:0/241383480 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feec4021070 con 0x7feed00b4e00 2023-11-24T06:09:38.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.077+0000 7feecffff700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feeb406aed0 0x7feeb406d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feed015a400 tx=0x7feebc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.081+0000 7feecd7fa700 1 -- 172.21.15.97:0/241383480 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feec4056000 con 0x7feed00b4e00 2023-11-24T06:09:38.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.270+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feed0068fb0 con 0x7feed00b4e00 2023-11-24T06:09:38.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.270+0000 7feecd7fa700 1 -- 172.21.15.97:0/241383480 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7feec402b020 con 0x7feed00b4e00 2023-11-24T06:09:38.273 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:38.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feeb406aed0 msgr2=0x7feeb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feeb406aed0 0x7feeb406d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feed015a400 tx=0x7feebc009040 comp rx=0 tx=0).stop 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed00b4e00 msgr2=0x7feed01522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed00b4e00 0x7feed01522b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feec400a010 tx=0x7feec4004750 comp rx=0 tx=0).stop 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 shutdown_connections 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feed015be20 0x7feed015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feed00b4e00 0x7feed01522b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feeb406aed0 0x7feeb406d390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 --2- 172.21.15.97:0/241383480 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feed00cdf50 0x7feed014dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 >> 172.21.15.97:0/241383480 conn(0x7feed00b2050 msgr2=0x7feed0071340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 shutdown_connections 2023-11-24T06:09:38.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.272+0000 7feed6f5e700 1 -- 172.21.15.97:0/241383480 wait complete. 2023-11-24T06:09:38.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.409+0000 7f0802175700 1 Processor -- start 2023-11-24T06:09:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.409+0000 7f0802175700 1 -- start start 2023-11-24T06:09:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f0802175700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc0c6c30 0x7f07fc0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f0802175700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07fc0cef40 0x7f07fc0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f0802175700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07fc0d0140 con 0x7f07fc15bf50 2023-11-24T06:09:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f0802175700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07fc15c330 con 0x7f07fc0cef40 2023-11-24T06:09:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f0802175700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc0cf3d0 con 0x7f07fc0c6c30 2023-11-24T06:09:38.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fbfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07fc0cef40 0x7f07fc0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fb7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f07fc15bf50 0x7f07fc0ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38810/0 (socket says 172.21.15.97:38810) 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fb7fe700 1 -- 172.21.15.97:0/3336228624 learned_addr learned my addr 172.21.15.97:0/3336228624 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07faffd700 1 --2- 172.21.15.97:0/3336228624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc0c6c30 0x7f07fc0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fbfff700 1 -- 172.21.15.97:0/3336228624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc0c6c30 msgr2=0x7f07fc0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fbfff700 1 --2- 172.21.15.97:0/3336228624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc0c6c30 0x7f07fc0cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fbfff700 1 -- 172.21.15.97:0/3336228624 >> v1:172.21.15.97:6789/0 conn(0x7f07fc15bf50 legacy=0x7f07fc0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fbfff700 1 -- 172.21.15.97:0/3336228624 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc0d03e0 con 0x7f07fc0cef40 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fa7fc700 1 -- 172.21.15.97:0/3336228624 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86428407 0 0) 0x7f07fc0d0140 con 0x7f07fc15bf50 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.410+0000 7f07fbfff700 1 --2- 172.21.15.97:0/3336228624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc0cc6c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009d90 tx=0x7f07f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6b009e4e41c4b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f07fa7fc700 1 -- 172.21.15.97:0/3336228624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f400e070 con 0x7f07fc0cef40 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f07fa7fc700 1 -- 172.21.15.97:0/3336228624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f400bb20 con 0x7f07fc0cef40 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f07fa7fc700 1 -- 172.21.15.97:0/3336228624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4012430 con 0x7f07fc0cef40 2023-11-24T06:09:38.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f0802175700 1 -- 172.21.15.97:0/3336228624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 msgr2=0x7f07fc0cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f0802175700 1 --2- 172.21.15.97:0/3336228624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc0cc6c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009d90 tx=0x7f07f400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f0802175700 1 -- 172.21.15.97:0/3336228624 shutdown_connections 2023-11-24T06:09:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f0802175700 1 --2- 172.21.15.97:0/3336228624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc0cc6c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f0802175700 1 --2- 172.21.15.97:0/3336228624 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc0c6c30 0x7f07fc0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f0802175700 1 -- 172.21.15.97:0/3336228624 >> 172.21.15.97:0/3336228624 conn(0x7f07fc0b3780 msgr2=0x7f07fc0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f0802175700 1 -- 172.21.15.97:0/3336228624 shutdown_connections 2023-11-24T06:09:38.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.411+0000 7f0802175700 1 -- 172.21.15.97:0/3336228624 wait complete. 2023-11-24T06:09:38.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.412+0000 7f0802175700 1 Processor -- start 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.412+0000 7f0802175700 1 -- start start 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f0802175700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc0c6c30 0x7f07fc1f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f0802175700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc1f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f0802175700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15bf50 0x7f07fc0c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f0802175700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc0ce770 con 0x7f07fc0cef40 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f0802175700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc1f3060 con 0x7f07fc0c6c30 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f0802175700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc1f3270 con 0x7f07fc15bf50 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f07faffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc1f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f07faffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc1f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46366/0 (socket says 172.21.15.97:46366) 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f07faffd700 1 -- 172.21.15.97:0/2238008294 learned_addr learned my addr 172.21.15.97:0/2238008294 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:38.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f07fb7fe700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc0c6c30 0x7f07fc1f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.413+0000 7f07fbfff700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15bf50 0x7f07fc0c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07faffd700 1 -- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15bf50 msgr2=0x7f07fc0c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07faffd700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15bf50 0x7f07fc0c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07faffd700 1 -- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc0c6c30 msgr2=0x7f07fc1f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07faffd700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc0c6c30 0x7f07fc1f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07faffd700 1 -- 172.21.15.97:0/2238008294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f4009a70 con 0x7f07fc0cef40 2023-11-24T06:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07fb7fe700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc0c6c30 0x7f07fc1f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:38.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07fbfff700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15bf50 0x7f07fc0c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07faffd700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc1f4310 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f07e400e9b0 tx=0x7f07e400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07f3fff700 1 -- 172.21.15.97:0/2238008294 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4009710 con 0x7f07fc0cef40 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07f3fff700 1 -- 172.21.15.97:0/2238008294 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e400c850 con 0x7f07fc0cef40 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc1f3550 con 0x7f07fc0cef40 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f07f3fff700 1 -- 172.21.15.97:0/2238008294 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e400cbc0 con 0x7f07fc0cef40 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.414+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc1f3b10 con 0x7f07fc0cef40 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.415+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc068fb0 con 0x7f07fc0cef40 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.415+0000 7f07f3fff700 1 -- 172.21.15.97:0/2238008294 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07e400cd20 con 0x7f07fc0cef40 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.416+0000 7f07f3fff700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06aed0 0x7f07dc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.416+0000 7f07f3fff700 1 -- 172.21.15.97:0/2238008294 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07e40896d0 con 0x7f07fc0cef40 2023-11-24T06:09:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.416+0000 7f07fb7fe700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06aed0 0x7f07dc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.416+0000 7f07fb7fe700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06aed0 0x7f07dc06d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f07ec000f60 tx=0x7f07ec007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.420+0000 7f07f3fff700 1 -- 172.21.15.97:0/2238008294 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07e4054780 con 0x7f07fc0cef40 2023-11-24T06:09:38.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.614+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f07fc158a50 con 0x7f07fc0cef40 2023-11-24T06:09:38.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.614+0000 7f07f3fff700 1 -- 172.21.15.97:0/2238008294 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f07e4057da0 con 0x7f07fc0cef40 2023-11-24T06:09:38.617 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:38.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06aed0 msgr2=0x7f07dc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06aed0 0x7f07dc06d390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f07ec000f60 tx=0x7f07ec007d30 comp rx=0 tx=0).stop 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 msgr2=0x7f07fc1f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc1f4310 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f07e400e9b0 tx=0x7f07e400ed70 comp rx=0 tx=0).stop 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 shutdown_connections 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06aed0 0x7f07dc06d390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15bf50 0x7f07fc0c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc0c6c30 0x7f07fc1f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 --2- 172.21.15.97:0/2238008294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc0cef40 0x7f07fc1f4310 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 >> 172.21.15.97:0/2238008294 conn(0x7f07fc0b3780 msgr2=0x7f07fc15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:38.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 shutdown_connections 2023-11-24T06:09:38.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.616+0000 7f0802175700 1 -- 172.21.15.97:0/2238008294 wait complete. 2023-11-24T06:09:38.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:09:38.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.749+0000 7efeec866700 1 Processor -- start 2023-11-24T06:09:38.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.749+0000 7efeec866700 1 -- start start 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.750+0000 7efeec866700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee806c710 0x7efee80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.750+0000 7efeec866700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efee80af4d0 0x7efee80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.750+0000 7efeec866700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee80ae430 con 0x7efee815e080 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.750+0000 7efeec866700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee80af0b0 con 0x7efee80af4d0 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.750+0000 7efeec866700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee80b05a0 con 0x7efee806c710 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.750+0000 7efee659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efee815e080 0x7efee806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38824/0 (socket says 172.21.15.97:38824) 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.750+0000 7efee659c700 1 -- 172.21.15.97:0/3683605222 learned_addr learned my addr 172.21.15.97:0/3683605222 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019088407 0 0) 0x7efee80ae430 con 0x7efee815e080 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee5d9b700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee806c710 0x7efee80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed00035b0 con 0x7efee815e080 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee6d9d700 1 --2- 172.21.15.97:0/3683605222 >> v2:172.21.15.183:3300/0 conn(0x7efee80af4d0 0x7efee80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370077046 0 0) 0x7efed00035b0 con 0x7efee815e080 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee806c710 msgr2=0x7efee80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee806c710 0x7efee80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 >> v2:172.21.15.183:3300/0 conn(0x7efee80af4d0 msgr2=0x7efee80afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 --2- 172.21.15.97:0/3683605222 >> v2:172.21.15.183:3300/0 conn(0x7efee80af4d0 0x7efee80afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80ae710 con 0x7efee815e080 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7efedc001670 con 0x7efee815e080 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 >> v1:172.21.15.97:6789/0 conn(0x7efee815e080 legacy=0x7efee806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0003a30 0x7efed0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee806c710 0x7efed0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed0004d90 0x7efed0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee80af0b0 con 0x7efed0004d90 2023-11-24T06:09:38.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee80b05a0 con 0x7efed0003a30 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed000ba10 con 0x7efee806c710 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee6d9d700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed0004d90 0x7efed0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee659c700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0003a30 0x7efed0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee5d9b700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee806c710 0x7efed0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee6d9d700 1 -- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee806c710 msgr2=0x7efed0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.751+0000 7efee6d9d700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee806c710 0x7efed0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.752+0000 7efee6d9d700 1 -- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0003a30 msgr2=0x7efed0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.752+0000 7efee6d9d700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0003a30 0x7efed0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.752+0000 7efee6d9d700 1 -- 172.21.15.97:0/3683605222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed00035b0 con 0x7efed0004d90 2023-11-24T06:09:38.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.752+0000 7efee6d9d700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed0004d90 0x7efed0005210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efee000e170 tx=0x7efed0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427275a14748ae4b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.756+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000ec90 con 0x7efed0004d90 2023-11-24T06:09:38.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.756+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee000f040 con 0x7efed0004d90 2023-11-24T06:09:38.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.756+0000 7efee559a700 1 -- 172.21.15.97:0/3683605222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0014480 con 0x7efed0004d90 2023-11-24T06:09:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.757+0000 7efeec866700 1 -- 172.21.15.97:0/3683605222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed0004d90 msgr2=0x7efed0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.757+0000 7efeec866700 1 --2- 172.21.15.97:0/3683605222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efed0004d90 0x7efed0005210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efee000e170 tx=0x7efed0005880 comp rx=0 tx=0).stop 2023-11-24T06:09:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.757+0000 7efee6d9d700 1 -- 172.21.15.97:0/3683605222 reap_dead start 2023-11-24T06:09:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.757+0000 7efeec866700 1 -- 172.21.15.97:0/3683605222 shutdown_connections 2023-11-24T06:09:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.757+0000 7efeec866700 1 -- 172.21.15.97:0/3683605222 >> 172.21.15.97:0/3683605222 conn(0x7efee8071b90 msgr2=0x7efee8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.757+0000 7efeec866700 1 -- 172.21.15.97:0/3683605222 shutdown_connections 2023-11-24T06:09:38.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.757+0000 7efeec866700 1 -- 172.21.15.97:0/3683605222 wait complete. 2023-11-24T06:09:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.758+0000 7efeec866700 1 Processor -- start 2023-11-24T06:09:38.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.759+0000 7efeec866700 1 -- start start 2023-11-24T06:09:38.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.759+0000 7efeec866700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0004d90 0x7efee81465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.759+0000 7efeec866700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee8160630 0x7efee8160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.759+0000 7efeec866700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee8158320 0x7efee81587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.759+0000 7efeec866700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee8160ff0 con 0x7efee8158320 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.759+0000 7efeec866700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee81f6d60 con 0x7efed0004d90 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.759+0000 7efeec866700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee81f7040 con 0x7efee8160630 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee8158320 0x7efee81587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee8158320 0x7efee81587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46388/0 (socket says 172.21.15.97:46388) 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 -- 172.21.15.97:0/2308512648 learned_addr learned my addr 172.21.15.97:0/2308512648 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee5d9b700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee8160630 0x7efee8160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee659c700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0004d90 0x7efee81465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 -- 172.21.15.97:0/2308512648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee8160630 msgr2=0x7efee8160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee8160630 0x7efee8160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 -- 172.21.15.97:0/2308512648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0004d90 msgr2=0x7efee81465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0004d90 0x7efee81465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 -- 172.21.15.97:0/2308512648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee81f7320 con 0x7efee8158320 2023-11-24T06:09:38.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.760+0000 7efee6d9d700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee8158320 0x7efee81587a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efee0016000 tx=0x7efee0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.761+0000 7efed6ffd700 1 -- 172.21.15.97:0/2308512648 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0007000 con 0x7efee8158320 2023-11-24T06:09:38.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.761+0000 7efed6ffd700 1 -- 172.21.15.97:0/2308512648 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee000ee80 con 0x7efee8158320 2023-11-24T06:09:38.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.761+0000 7efed6ffd700 1 -- 172.21.15.97:0/2308512648 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0003c60 con 0x7efee8158320 2023-11-24T06:09:38.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.761+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee81f7600 con 0x7efee8158320 2023-11-24T06:09:38.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.761+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee8158ce0 con 0x7efee8158320 2023-11-24T06:09:38.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.762+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee80753f0 con 0x7efee8158320 2023-11-24T06:09:38.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.763+0000 7efed6ffd700 1 -- 172.21.15.97:0/2308512648 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efee0024030 con 0x7efee8158320 2023-11-24T06:09:38.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.764+0000 7efed6ffd700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efec806af20 0x7efec806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:38.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.764+0000 7efed6ffd700 1 -- 172.21.15.97:0/2308512648 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7efee008acf0 con 0x7efee8158320 2023-11-24T06:09:38.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.764+0000 7efee659c700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efec806af20 0x7efec806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:38.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.764+0000 7efee659c700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efec806af20 0x7efec806d3e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efedc0017a0 tx=0x7efedc0015d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:38.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.769+0000 7efed6ffd700 1 -- 172.21.15.97:0/2308512648 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efee0055e90 con 0x7efee8158320 2023-11-24T06:09:38.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.976+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efee81f7790 con 0x7efee8158320 2023-11-24T06:09:38.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.977+0000 7efed6ffd700 1 -- 172.21.15.97:0/2308512648 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7efee00594b0 con 0x7efee8158320 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efec806af20 msgr2=0x7efec806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efec806af20 0x7efec806d3e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efedc0017a0 tx=0x7efedc0015d0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee8158320 msgr2=0x7efee81587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee8158320 0x7efee81587a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efee0016000 tx=0x7efee0004b40 comp rx=0 tx=0).stop 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 shutdown_connections 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efee8158320 0x7efee81587a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efed0004d90 0x7efee81465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efec806af20 0x7efec806d3e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 --2- 172.21.15.97:0/2308512648 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efee8160630 0x7efee8160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 >> 172.21.15.97:0/2308512648 conn(0x7efee8071b90 msgr2=0x7efee806f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 shutdown_connections 2023-11-24T06:09:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:38.979+0000 7efeec866700 1 -- 172.21.15.97:0/2308512648 wait complete. 2023-11-24T06:09:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:09:38.993 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-24T06:09:38.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:38.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-24T06:09:39.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:09:39.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:39.038 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:09:39.094 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:09:39.094 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:39.116 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:09:39.116 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:39.175 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:09:39.175 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:09:39.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:39.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.352+0000 7f8efe4a6700 1 Processor -- start 2023-11-24T06:09:39.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.352+0000 7f8efe4a6700 1 -- start start 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.353+0000 7f8efe4a6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ef80d22f0 0x7f8ef80d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.353+0000 7f8efe4a6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef80d0150 con 0x7f8ef815e080 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.353+0000 7f8efe4a6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef80cee50 con 0x7f8ef80ce4d0 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.353+0000 7f8efe4a6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef815e460 con 0x7f8ef80d22f0 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.353+0000 7f8efca43700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ef80d22f0 0x7f8ef80d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.353+0000 7f8ef7fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8ef815e080 0x7f8ef80cc2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38844/0 (socket says 172.21.15.97:38844) 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.353+0000 7f8ef7fff700 1 -- 172.21.15.97:0/4144622547 learned_addr learned my addr 172.21.15.97:0/4144622547 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8efca43700 1 -- 172.21.15.97:0/4144622547 >> v1:172.21.15.183:6789/0 conn(0x7f8ef80ce4d0 legacy=0x7f8ef80d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8efca43700 1 -- 172.21.15.97:0/4144622547 >> v1:172.21.15.97:6789/0 conn(0x7f8ef815e080 legacy=0x7f8ef80cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8efca43700 1 -- 172.21.15.97:0/4144622547 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80cf130 con 0x7f8ef80d22f0 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8ef6ffd700 1 -- 172.21.15.97:0/4144622547 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578834395 0 0) 0x7f8ef80d0150 con 0x7f8ef815e080 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8efca43700 1 --2- 172.21.15.97:0/4144622547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef80d22f0 0x7f8ef80d05e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009d90 tx=0x7f8ef000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c712c151245bb11a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8ef6ffd700 1 -- 172.21.15.97:0/4144622547 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000e070 con 0x7f8ef80d22f0 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8ef6ffd700 1 -- 172.21.15.97:0/4144622547 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef000bb90 con 0x7f8ef80d22f0 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8ef6ffd700 1 -- 172.21.15.97:0/4144622547 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0003a20 con 0x7f8ef80d22f0 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8efe4a6700 1 -- 172.21.15.97:0/4144622547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef80d22f0 msgr2=0x7f8ef80d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.354+0000 7f8efe4a6700 1 --2- 172.21.15.97:0/4144622547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef80d22f0 0x7f8ef80d05e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0009d90 tx=0x7f8ef000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.355+0000 7f8efe4a6700 1 -- 172.21.15.97:0/4144622547 shutdown_connections 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.355+0000 7f8efe4a6700 1 --2- 172.21.15.97:0/4144622547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef80d22f0 0x7f8ef80d05e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.355+0000 7f8efe4a6700 1 -- 172.21.15.97:0/4144622547 >> 172.21.15.97:0/4144622547 conn(0x7f8ef8073130 msgr2=0x7f8ef809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.355+0000 7f8efe4a6700 1 -- 172.21.15.97:0/4144622547 shutdown_connections 2023-11-24T06:09:39.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.355+0000 7f8efe4a6700 1 -- 172.21.15.97:0/4144622547 wait complete. 2023-11-24T06:09:39.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.356+0000 7f8efe4a6700 1 Processor -- start 2023-11-24T06:09:39.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.356+0000 7f8efe4a6700 1 -- start start 2023-11-24T06:09:39.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efe4a6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ef80ce4d0 0x7f8ef8154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efe4a6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef80d22f0 0x7f8ef815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efe4a6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef815e080 0x7f8ef815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efe4a6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80b7a50 con 0x7f8ef815e080 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efe4a6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8158900 con 0x7f8ef80d22f0 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efe4a6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80b6c90 con 0x7f8ef80ce4d0 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef815e080 0x7f8ef815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef815e080 0x7f8ef815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46400/0 (socket says 172.21.15.97:46400) 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 -- 172.21.15.97:0/1435924430 learned_addr learned my addr 172.21.15.97:0/1435924430 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8ef7fff700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ef80ce4d0 0x7f8ef8154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8ef77fe700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef80d22f0 0x7f8ef815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 -- 172.21.15.97:0/1435924430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ef80ce4d0 msgr2=0x7f8ef8154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ef80ce4d0 0x7f8ef8154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 -- 172.21.15.97:0/1435924430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef80d22f0 msgr2=0x7f8ef815a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef80d22f0 0x7f8ef815a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 -- 172.21.15.97:0/1435924430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef0009a70 con 0x7f8ef815e080 2023-11-24T06:09:39.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efca43700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef815e080 0x7f8ef815aa50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b730 tx=0x7f8ef0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8ef4ff9700 1 -- 172.21.15.97:0/1435924430 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0012630 con 0x7f8ef815e080 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8ef4ff9700 1 -- 172.21.15.97:0/1435924430 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef0012790 con 0x7f8ef815e080 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef80b6f70 con 0x7f8ef815e080 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8ef4ff9700 1 -- 172.21.15.97:0/1435924430 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0012b40 con 0x7f8ef815e080 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.357+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef80b7530 con 0x7f8ef815e080 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.358+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef80b76c0 con 0x7f8ef815e080 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.359+0000 7f8ef4ff9700 1 -- 172.21.15.97:0/1435924430 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ef0012ca0 con 0x7f8ef815e080 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.359+0000 7f8ef4ff9700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ee406aed0 0x7f8ee406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.359+0000 7f8ef4ff9700 1 -- 172.21.15.97:0/1435924430 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8ef008c100 con 0x7f8ef815e080 2023-11-24T06:09:39.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.359+0000 7f8ef7fff700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ee406aed0 0x7f8ee406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.360+0000 7f8ef7fff700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ee406aed0 0x7f8ee406d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8eec000f60 tx=0x7f8eec007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.364+0000 7f8ef4ff9700 1 -- 172.21.15.97:0/1435924430 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ef00571c0 con 0x7f8ef815e080 2023-11-24T06:09:39.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.555+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ef806b9d0 con 0x7f8ef815e080 2023-11-24T06:09:39.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.556+0000 7f8ef4ff9700 1 -- 172.21.15.97:0/1435924430 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16766 (secure 0 0 0) 0x7f8ef005a7e0 con 0x7f8ef815e080 2023-11-24T06:09:39.559 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:39.559 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":21,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:38.745125+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:39.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ee406aed0 msgr2=0x7f8ee406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ee406aed0 0x7f8ee406d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8eec000f60 tx=0x7f8eec007c90 comp rx=0 tx=0).stop 2023-11-24T06:09:39.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef815e080 msgr2=0x7f8ef815aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef815e080 0x7f8ef815aa50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b730 tx=0x7f8ef0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 shutdown_connections 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ef815e080 0x7f8ef815aa50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ee406aed0 0x7f8ee406d390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ef80ce4d0 0x7f8ef8154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 --2- 172.21.15.97:0/1435924430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ef80d22f0 0x7f8ef815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 >> 172.21.15.97:0/1435924430 conn(0x7f8ef8073130 msgr2=0x7f8ef806fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 shutdown_connections 2023-11-24T06:09:39.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.558+0000 7f8efe4a6700 1 -- 172.21.15.97:0/1435924430 wait complete. 2023-11-24T06:09:39.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:39.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.695+0000 7f0292a02700 1 Processor -- start 2023-11-24T06:09:39.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.695+0000 7f0292a02700 1 -- start start 2023-11-24T06:09:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0292a02700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c1587f0 0x7f028c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0292a02700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f028c15b370 0x7f028c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0292a02700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f028c0b3c60 0x7f028c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0292a02700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c0b4610 con 0x7f028c0b3c60 2023-11-24T06:09:39.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0292a02700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c06b530 con 0x7f028c15b370 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0292a02700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0ae5e0 con 0x7f028c1587f0 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f028c0b3c60 0x7f028c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f028c0b3c60 0x7f028c0b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46406/0 (socket says 172.21.15.97:46406) 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 -- 172.21.15.97:0/1448374365 learned_addr learned my addr 172.21.15.97:0/1448374365 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f028bfff700 1 --2- 172.21.15.97:0/1448374365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c1587f0 0x7f028c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f028b7fe700 1 --2- 172.21.15.97:0/1448374365 >> v2:172.21.15.183:3300/0 conn(0x7f028c15b370 0x7f028c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 -- 172.21.15.97:0/1448374365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c1587f0 msgr2=0x7f028c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 --2- 172.21.15.97:0/1448374365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c1587f0 0x7f028c06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 -- 172.21.15.97:0/1448374365 >> v2:172.21.15.183:3300/0 conn(0x7f028c15b370 msgr2=0x7f028c15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 --2- 172.21.15.97:0/1448374365 >> v2:172.21.15.183:3300/0 conn(0x7f028c15b370 0x7f028c15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 -- 172.21.15.97:0/1448374365 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c15be30 con 0x7f028c0b3c60 2023-11-24T06:09:39.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f0290f9f700 1 --2- 172.21.15.97:0/1448374365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3c60 0x7f028c0b40d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f028c072e10 tx=0x7f028400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735a7243a09f5094 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f028affd700 1 -- 172.21.15.97:0/1448374365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400e070 con 0x7f028c0b3c60 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.696+0000 7f028affd700 1 -- 172.21.15.97:0/1448374365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028400b9c0 con 0x7f028c0b3c60 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f028affd700 1 -- 172.21.15.97:0/1448374365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028400bd70 con 0x7f028c0b3c60 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 -- 172.21.15.97:0/1448374365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3c60 msgr2=0x7f028c0b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 --2- 172.21.15.97:0/1448374365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3c60 0x7f028c0b40d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f028c072e10 tx=0x7f028400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 -- 172.21.15.97:0/1448374365 shutdown_connections 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 --2- 172.21.15.97:0/1448374365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c0b3c60 0x7f028c0b40d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 --2- 172.21.15.97:0/1448374365 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c1587f0 0x7f028c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 --2- 172.21.15.97:0/1448374365 >> v2:172.21.15.183:3300/0 conn(0x7f028c15b370 0x7f028c15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 -- 172.21.15.97:0/1448374365 >> 172.21.15.97:0/1448374365 conn(0x7f028c0d0bf0 msgr2=0x7f028c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:39.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 -- 172.21.15.97:0/1448374365 shutdown_connections 2023-11-24T06:09:39.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.697+0000 7f0292a02700 1 -- 172.21.15.97:0/1448374365 wait complete. 2023-11-24T06:09:39.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.698+0000 7f0292a02700 1 Processor -- start 2023-11-24T06:09:39.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.698+0000 7f0292a02700 1 -- start start 2023-11-24T06:09:39.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0292a02700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0b3c60 0x7f028c146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0292a02700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c1587f0 0x7f028c09b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0292a02700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c15b370 0x7f028c149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0292a02700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c14c900 con 0x7f028c15b370 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0292a02700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c146750 con 0x7f028c1587f0 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0292a02700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c146f40 con 0x7f028c0b3c60 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0290f9f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c15b370 0x7f028c149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0290f9f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c15b370 0x7f028c149f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46412/0 (socket says 172.21.15.97:46412) 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f0290f9f700 1 -- 172.21.15.97:0/2339211926 learned_addr learned my addr 172.21.15.97:0/2339211926 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f028bfff700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0b3c60 0x7f028c146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.699+0000 7f028b7fe700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c1587f0 0x7f028c09b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0290f9f700 1 -- 172.21.15.97:0/2339211926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0b3c60 msgr2=0x7f028c146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0290f9f700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0b3c60 0x7f028c146210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0290f9f700 1 -- 172.21.15.97:0/2339211926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c1587f0 msgr2=0x7f028c09b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0290f9f700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c1587f0 0x7f028c09b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0290f9f700 1 -- 172.21.15.97:0/2339211926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284009b30 con 0x7f028c15b370 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0290f9f700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c15b370 0x7f028c149f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f028c0d1730 tx=0x7f0284004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0288ff9700 1 -- 172.21.15.97:0/2339211926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02840099b0 con 0x7f028c15b370 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0288ff9700 1 -- 172.21.15.97:0/2339211926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0284004250 con 0x7f028c15b370 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c147220 con 0x7f028c15b370 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0288ff9700 1 -- 172.21.15.97:0/2339211926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284004600 con 0x7f028c15b370 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.700+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c1477e0 con 0x7f028c15b370 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.701+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c068fb0 con 0x7f028c15b370 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.701+0000 7f0288ff9700 1 -- 172.21.15.97:0/2339211926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0284004760 con 0x7f028c15b370 2023-11-24T06:09:39.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.702+0000 7f0288ff9700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f027806aed0 0x7f027806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:39.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.702+0000 7f0288ff9700 1 -- 172.21.15.97:0/2339211926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f028408ad30 con 0x7f028c15b370 2023-11-24T06:09:39.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.702+0000 7f028bfff700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f027806aed0 0x7f027806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:39.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.702+0000 7f028bfff700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f027806aed0 0x7f027806d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0280006b20 tx=0x7f0280008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:39.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.706+0000 7f0288ff9700 1 -- 172.21.15.97:0/2339211926 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0284055df0 con 0x7f028c15b370 2023-11-24T06:09:39.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.901+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f028c09b950 con 0x7f028c15b370 2023-11-24T06:09:39.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.902+0000 7f0288ff9700 1 -- 172.21.15.97:0/2339211926 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0284020020 con 0x7f028c15b370 2023-11-24T06:09:39.905 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f027806aed0 msgr2=0x7f027806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f027806aed0 0x7f027806d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0280006b20 tx=0x7f0280008040 comp rx=0 tx=0).stop 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c15b370 msgr2=0x7f028c149f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c15b370 0x7f028c149f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f028c0d1730 tx=0x7f0284004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 shutdown_connections 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f028c15b370 0x7f028c149f70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f027806aed0 0x7f027806d390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f028c0b3c60 0x7f028c146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 --2- 172.21.15.97:0/2339211926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f028c1587f0 0x7f028c09b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:39.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 >> 172.21.15.97:0/2339211926 conn(0x7f028c0d0bf0 msgr2=0x7f028c156910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:39.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 shutdown_connections 2023-11-24T06:09:39.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:39.904+0000 7f0292a02700 1 -- 172.21.15.97:0/2339211926 wait complete. 2023-11-24T06:09:39.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:40.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.038+0000 7fd55ca86700 1 Processor -- start 2023-11-24T06:09:40.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.038+0000 7fd55ca86700 1 -- start start 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.039+0000 7fd55ca86700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd55815bf40 0x7fd5580af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.039+0000 7fd55ca86700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd55815ca30 0x7fd5580cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.039+0000 7fd55ca86700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5580cd670 con 0x7fd5580cccd0 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.039+0000 7fd55ca86700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55806fd30 con 0x7fd55815ca30 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.039+0000 7fd55ca86700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55806f770 con 0x7fd55815bf40 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd555d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd5580cccd0 0x7fd55815c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38868/0 (socket says 172.21.15.97:38868) 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd555d9b700 1 -- 172.21.15.97:0/3343741030 learned_addr learned my addr 172.21.15.97:0/3343741030 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004413861 0 0) 0x7fd5580cd670 con 0x7fd5580cccd0 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd556d9d700 1 --2- 172.21.15.97:0/3343741030 >> v2:172.21.15.183:3300/0 conn(0x7fd55815ca30 0x7fd5580cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55659c700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd55815bf40 0x7fd5580af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5400035b0 con 0x7fd5580cccd0 2023-11-24T06:09:40.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3787142895 0 0) 0x7fd5400035b0 con 0x7fd5580cccd0 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd55815bf40 msgr2=0x7fd5580af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd55815bf40 0x7fd5580af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 >> v2:172.21.15.183:3300/0 conn(0x7fd55815ca30 msgr2=0x7fd5580cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 --2- 172.21.15.97:0/3343741030 >> v2:172.21.15.183:3300/0 conn(0x7fd55815ca30 0x7fd5580cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55806fa50 con 0x7fd5580cccd0 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fd548001670 con 0x7fd5580cccd0 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 >> v1:172.21.15.97:6789/0 conn(0x7fd5580cccd0 legacy=0x7fd55815c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd55815bf40 0x7fd540003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd540003f80 0x7fd540004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd540004d90 0x7fd540005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55806fd30 con 0x7fd540004d90 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55806f770 con 0x7fd540003f80 2023-11-24T06:09:40.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54000ba10 con 0x7fd55815bf40 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd556d9d700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd540004d90 0x7fd540005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd55659c700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd55815bf40 0x7fd540003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.040+0000 7fd555d9b700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd540003f80 0x7fd540004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd556d9d700 1 -- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd55815bf40 msgr2=0x7fd540003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd556d9d700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd55815bf40 0x7fd540003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd556d9d700 1 -- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd540003f80 msgr2=0x7fd540004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd556d9d700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd540003f80 0x7fd540004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd556d9d700 1 -- 172.21.15.97:0/3343741030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400035b0 con 0x7fd540004d90 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd555d9b700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd540003f80 0x7fd540004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd556d9d700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd540004d90 0x7fd540005210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd558075520 tx=0x7fd540005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d8f88cefeac360 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000eb70 con 0x7fd540004d90 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55000f040 con 0x7fd540004d90 2023-11-24T06:09:40.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd55559a700 1 -- 172.21.15.97:0/3343741030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000eed0 con 0x7fd540004d90 2023-11-24T06:09:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd55ca86700 1 -- 172.21.15.97:0/3343741030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd540004d90 msgr2=0x7fd540005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd55ca86700 1 --2- 172.21.15.97:0/3343741030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd540004d90 0x7fd540005210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd558075520 tx=0x7fd540005880 comp rx=0 tx=0).stop 2023-11-24T06:09:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd556d9d700 1 -- 172.21.15.97:0/3343741030 reap_dead start 2023-11-24T06:09:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd55ca86700 1 -- 172.21.15.97:0/3343741030 shutdown_connections 2023-11-24T06:09:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd55ca86700 1 -- 172.21.15.97:0/3343741030 >> 172.21.15.97:0/3343741030 conn(0x7fd5580b2050 msgr2=0x7fd5580b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.041+0000 7fd55ca86700 1 -- 172.21.15.97:0/3343741030 shutdown_connections 2023-11-24T06:09:40.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.042+0000 7fd55ca86700 1 -- 172.21.15.97:0/3343741030 wait complete. 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.043+0000 7fd55ca86700 1 Processor -- start 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.043+0000 7fd55ca86700 1 -- start start 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55ca86700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd540004d90 0x7fd55814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55ca86700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd558151040 0x7fd5581e4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55ca86700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5581514c0 0x7fd5581e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55ca86700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd558152a30 con 0x7fd558151040 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55ca86700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd558151990 con 0x7fd5581514c0 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55ca86700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd558151c70 con 0x7fd540004d90 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd558151040 0x7fd5581e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd558151040 0x7fd5581e4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46430/0 (socket says 172.21.15.97:46430) 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 -- 172.21.15.97:0/2191861425 learned_addr learned my addr 172.21.15.97:0/2191861425 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:40.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55659c700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd540004d90 0x7fd55814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd556d9d700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5581514c0 0x7fd5581e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 -- 172.21.15.97:0/2191861425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd540004d90 msgr2=0x7fd55814ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd540004d90 0x7fd55814ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 -- 172.21.15.97:0/2191861425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5581514c0 msgr2=0x7fd5581e45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5581514c0 0x7fd5581e45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 -- 172.21.15.97:0/2191861425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558151f50 con 0x7fd558151040 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd555d9b700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd558151040 0x7fd5581e4080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd54800e8c0 tx=0x7fd54800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd546ffd700 1 -- 172.21.15.97:0/2191861425 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800c780 con 0x7fd558151040 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd546ffd700 1 -- 172.21.15.97:0/2191861425 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54800c8e0 con 0x7fd558151040 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd558152230 con 0x7fd558151040 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd546ffd700 1 -- 172.21.15.97:0/2191861425 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54800ccb0 con 0x7fd558151040 2023-11-24T06:09:40.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.044+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55814f200 con 0x7fd558151040 2023-11-24T06:09:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.045+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd558068d90 con 0x7fd558151040 2023-11-24T06:09:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.046+0000 7fd546ffd700 1 -- 172.21.15.97:0/2191861425 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd54801d460 con 0x7fd558151040 2023-11-24T06:09:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.046+0000 7fd546ffd700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd53806aed0 0x7fd53806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.046+0000 7fd546ffd700 1 -- 172.21.15.97:0/2191861425 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd548089540 con 0x7fd558151040 2023-11-24T06:09:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.046+0000 7fd55659c700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd53806aed0 0x7fd53806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.046+0000 7fd55659c700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd53806aed0 0x7fd53806d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009ba0 tx=0x7fd54c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.050+0000 7fd546ffd700 1 -- 172.21.15.97:0/2191861425 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd548054600 con 0x7fd558151040 2023-11-24T06:09:40.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.240+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd558068fb0 con 0x7fd558151040 2023-11-24T06:09:40.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.240+0000 7fd546ffd700 1 -- 172.21.15.97:0/2191861425 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5480015a0 con 0x7fd558151040 2023-11-24T06:09:40.243 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd53806aed0 msgr2=0x7fd53806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd53806aed0 0x7fd53806d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009ba0 tx=0x7fd54c008040 comp rx=0 tx=0).stop 2023-11-24T06:09:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd558151040 msgr2=0x7fd5581e4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd558151040 0x7fd5581e4080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd54800e8c0 tx=0x7fd54800ec80 comp rx=0 tx=0).stop 2023-11-24T06:09:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 shutdown_connections 2023-11-24T06:09:40.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5581514c0 0x7fd5581e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd53806aed0 0x7fd53806d390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd540004d90 0x7fd55814ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 --2- 172.21.15.97:0/2191861425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd558151040 0x7fd5581e4080 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 >> 172.21.15.97:0/2191861425 conn(0x7fd5580b2050 msgr2=0x7fd558158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 shutdown_connections 2023-11-24T06:09:40.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.243+0000 7fd55ca86700 1 -- 172.21.15.97:0/2191861425 wait complete. 2023-11-24T06:09:40.258 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.378+0000 7f164c47c700 1 Processor -- start 2023-11-24T06:09:40.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.378+0000 7f164c47c700 1 -- start start 2023-11-24T06:09:40.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164c47c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f16440ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164c47c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f16440ce830 0x7f16440cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164c47c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f16440cf1f0 0x7f16440c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164c47c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164415c330 con 0x7f16440ce830 2023-11-24T06:09:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164c47c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16440d0140 con 0x7f16440cf1f0 2023-11-24T06:09:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164c47c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16440d03e0 con 0x7f164415bf50 2023-11-24T06:09:40.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f16440ce830 0x7f16440cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f16440ce830 0x7f16440cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46436/0 (socket says 172.21.15.97:46436) 2023-11-24T06:09:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 -- 172.21.15.97:0/2843741230 learned_addr learned my addr 172.21.15.97:0/2843741230 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164a218700 1 --2- 172.21.15.97:0/2843741230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f16440ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164aa19700 1 --2- 172.21.15.97:0/2843741230 >> v2:172.21.15.183:3300/0 conn(0x7f16440cf1f0 0x7f16440c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 -- 172.21.15.97:0/2843741230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 msgr2=0x7f16440ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 --2- 172.21.15.97:0/2843741230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f16440ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 -- 172.21.15.97:0/2843741230 >> v2:172.21.15.183:3300/0 conn(0x7f16440cf1f0 msgr2=0x7f16440c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 --2- 172.21.15.97:0/2843741230 >> v2:172.21.15.183:3300/0 conn(0x7f16440cf1f0 0x7f16440c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 -- 172.21.15.97:0/2843741230 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440d06c0 con 0x7f16440ce830 2023-11-24T06:09:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f164aa19700 1 --2- 172.21.15.97:0/2843741230 >> v2:172.21.15.183:3300/0 conn(0x7f16440cf1f0 0x7f16440c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649a17700 1 --2- 172.21.15.97:0/2843741230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f16440cecb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16440c8c00 tx=0x7f1634009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4814ac776a3fa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.379+0000 7f1649216700 1 -- 172.21.15.97:0/2843741230 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163400b870 con 0x7f16440ce830 2023-11-24T06:09:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f1649216700 1 -- 172.21.15.97:0/2843741230 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163400b9d0 con 0x7f16440ce830 2023-11-24T06:09:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f1649216700 1 -- 172.21.15.97:0/2843741230 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163400bd60 con 0x7f16440ce830 2023-11-24T06:09:40.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 -- 172.21.15.97:0/2843741230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 msgr2=0x7f16440cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 --2- 172.21.15.97:0/2843741230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f16440cecb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16440c8c00 tx=0x7f1634009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 -- 172.21.15.97:0/2843741230 shutdown_connections 2023-11-24T06:09:40.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 --2- 172.21.15.97:0/2843741230 >> v2:172.21.15.183:3300/0 conn(0x7f16440cf1f0 0x7f16440c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 --2- 172.21.15.97:0/2843741230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f16440ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 --2- 172.21.15.97:0/2843741230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f16440cecb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 -- 172.21.15.97:0/2843741230 >> 172.21.15.97:0/2843741230 conn(0x7f16440b3780 msgr2=0x7f16440b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 -- 172.21.15.97:0/2843741230 shutdown_connections 2023-11-24T06:09:40.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.380+0000 7f164c47c700 1 -- 172.21.15.97:0/2843741230 wait complete. 2023-11-24T06:09:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.381+0000 7f164c47c700 1 Processor -- start 2023-11-24T06:09:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.381+0000 7f164c47c700 1 -- start start 2023-11-24T06:09:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164c47c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f164415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164c47c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16440cf1f0 0x7f164409e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164c47c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f164409ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164c47c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644147180 con 0x7f16440ce830 2023-11-24T06:09:40.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164c47c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440cae30 con 0x7f16440cf1f0 2023-11-24T06:09:40.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164c47c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1644146850 con 0x7f164415bf50 2023-11-24T06:09:40.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164a218700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f164415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164a218700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f164415a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46442/0 (socket says 172.21.15.97:46442) 2023-11-24T06:09:40.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164a218700 1 -- 172.21.15.97:0/2708341835 learned_addr learned my addr 172.21.15.97:0/2708341835 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:40.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f1649a17700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16440cf1f0 0x7f164409e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164aa19700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f164409ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164a218700 1 -- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 msgr2=0x7f164409ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164a218700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f164409ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164a218700 1 -- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16440cf1f0 msgr2=0x7f164409e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164a218700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16440cf1f0 0x7f164409e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164a218700 1 -- 172.21.15.97:0/2708341835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634009550 con 0x7f16440ce830 2023-11-24T06:09:40.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.382+0000 7f164aa19700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f164409ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:40.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.383+0000 7f164a218700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f164415a100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f164000d800 tx=0x7f164000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.383+0000 7f163affd700 1 -- 172.21.15.97:0/2708341835 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16400125f0 con 0x7f16440ce830 2023-11-24T06:09:40.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.383+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1644146b30 con 0x7f16440ce830 2023-11-24T06:09:40.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.383+0000 7f163affd700 1 -- 172.21.15.97:0/2708341835 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1640012750 con 0x7f16440ce830 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.383+0000 7f163affd700 1 -- 172.21.15.97:0/2708341835 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640012b00 con 0x7f16440ce830 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.383+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164415a640 con 0x7f16440ce830 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.384+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1644068fb0 con 0x7f16440ce830 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.384+0000 7f163affd700 1 -- 172.21.15.97:0/2708341835 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1640012c60 con 0x7f16440ce830 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.384+0000 7f163affd700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f162806aed0 0x7f162806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.385+0000 7f163affd700 1 -- 172.21.15.97:0/2708341835 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f16400899f0 con 0x7f16440ce830 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.385+0000 7f1649a17700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f162806aed0 0x7f162806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.385+0000 7f1649a17700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f162806aed0 0x7f162806d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1634000c00 tx=0x7f1634008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.389+0000 7f163affd700 1 -- 172.21.15.97:0/2708341835 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1640054910 con 0x7f16440ce830 2023-11-24T06:09:40.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.578+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16440cf890 con 0x7f16440ce830 2023-11-24T06:09:40.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.579+0000 7f163affd700 1 -- 172.21.15.97:0/2708341835 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1640057f30 con 0x7f16440ce830 2023-11-24T06:09:40.582 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f162806aed0 msgr2=0x7f162806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f162806aed0 0x7f162806d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1634000c00 tx=0x7f1634008030 comp rx=0 tx=0).stop 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 msgr2=0x7f164415a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f164415a100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f164000d800 tx=0x7f164000db10 comp rx=0 tx=0).stop 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 shutdown_connections 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f162806aed0 0x7f162806d390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f164415bf50 0x7f164409ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16440ce830 0x7f164415a100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 --2- 172.21.15.97:0/2708341835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16440cf1f0 0x7f164409e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 >> 172.21.15.97:0/2708341835 conn(0x7f16440b3780 msgr2=0x7f164406ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 shutdown_connections 2023-11-24T06:09:40.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.581+0000 7f164c47c700 1 -- 172.21.15.97:0/2708341835 wait complete. 2023-11-24T06:09:40.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:09:40.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.714+0000 7f2e879ee700 1 Processor -- start 2023-11-24T06:09:40.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.714+0000 7f2e879ee700 1 -- start start 2023-11-24T06:09:40.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e879ee700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2e8006c710 0x7f2e8015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e879ee700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8015f520 con 0x7f2e80144660 2023-11-24T06:09:40.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e879ee700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e8015f800 con 0x7f2e8015ea60 2023-11-24T06:09:40.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e879ee700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e800b4230 con 0x7f2e8006c710 2023-11-24T06:09:40.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e8578a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2e80144660 0x7f2e800b4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38902/0 (socket says 172.21.15.97:38902) 2023-11-24T06:09:40.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e8578a700 1 -- 172.21.15.97:0/1801802033 learned_addr learned my addr 172.21.15.97:0/1801802033 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149430806 0 0) 0x7f2e8015f520 con 0x7f2e80144660 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e85f8b700 1 --2- 172.21.15.97:0/1801802033 >> v2:172.21.15.183:3300/0 conn(0x7f2e8006c710 0x7f2e8015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e6c0035b0 con 0x7f2e80144660 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 378661885 0 0) 0x7f2e6c0035b0 con 0x7f2e80144660 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 >> v2:172.21.15.183:3300/0 conn(0x7f2e8006c710 msgr2=0x7f2e8015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 --2- 172.21.15.97:0/1801802033 >> v2:172.21.15.183:3300/0 conn(0x7f2e8006c710 0x7f2e8015eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 >> v1:172.21.15.183:6789/0 conn(0x7f2e8015ea60 legacy=0x7f2e8006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e800b3e50 con 0x7f2e80144660 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f2e7c001670 con 0x7f2e80144660 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 >> v1:172.21.15.97:6789/0 conn(0x7f2e80144660 legacy=0x7f2e800b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c003a00 0x7f2e6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e6c004850 0x7f2e6c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6c005210 0x7f2e6c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.715+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800b4230 con 0x7f2e6c005210 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c00c1b0 con 0x7f2e6c004850 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c00c370 con 0x7f2e6c003a00 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e85f8b700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6c005210 0x7f2e6c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e8578a700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c003a00 0x7f2e6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e84f89700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e6c004850 0x7f2e6c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e85f8b700 1 -- 172.21.15.97:0/1801802033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c003a00 msgr2=0x7f2e6c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e85f8b700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c003a00 0x7f2e6c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e85f8b700 1 -- 172.21.15.97:0/1801802033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e6c004850 msgr2=0x7f2e6c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e85f8b700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e6c004850 0x7f2e6c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e85f8b700 1 -- 172.21.15.97:0/1801802033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c0035b0 con 0x7f2e6c005210 2023-11-24T06:09:40.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e85f8b700 1 -- 172.21.15.97:0/1801802033 reap_dead start 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e85f8b700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6c005210 0x7f2e6c00ba10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009fc0 tx=0x7f2e6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6fa3adf9080b2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78003a20 con 0x7f2e6c005210 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7800f040 con 0x7f2e6c005210 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.716+0000 7f2e77fff700 1 -- 172.21.15.97:0/1801802033 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78003de0 con 0x7f2e6c005210 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.717+0000 7f2e879ee700 1 -- 172.21.15.97:0/1801802033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6c005210 msgr2=0x7f2e6c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.717+0000 7f2e879ee700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6c005210 0x7f2e6c00ba10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009fc0 tx=0x7f2e6c005880 comp rx=0 tx=0).stop 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.717+0000 7f2e879ee700 1 -- 172.21.15.97:0/1801802033 shutdown_connections 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.717+0000 7f2e879ee700 1 --2- 172.21.15.97:0/1801802033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e6c005210 0x7f2e6c00ba10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.717+0000 7f2e879ee700 1 -- 172.21.15.97:0/1801802033 >> 172.21.15.97:0/1801802033 conn(0x7f2e80071ac0 msgr2=0x7f2e80071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.717+0000 7f2e879ee700 1 -- 172.21.15.97:0/1801802033 shutdown_connections 2023-11-24T06:09:40.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.717+0000 7f2e879ee700 1 -- 172.21.15.97:0/1801802033 wait complete. 2023-11-24T06:09:40.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.718+0000 7f2e879ee700 1 Processor -- start 2023-11-24T06:09:40.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.718+0000 7f2e879ee700 1 -- start start 2023-11-24T06:09:40.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e879ee700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c005210 0x7f2e8013ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e879ee700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e801436a0 0x7f2e80153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e879ee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e801423c0 0x7f2e80151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e879ee700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80141c50 con 0x7f2e801423c0 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e879ee700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80141f30 con 0x7f2e801436a0 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e879ee700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80153ef0 con 0x7f2e6c005210 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e801423c0 0x7f2e80151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e801423c0 0x7f2e80151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46464/0 (socket says 172.21.15.97:46464) 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 -- 172.21.15.97:0/2502720936 learned_addr learned my addr 172.21.15.97:0/2502720936 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e84f89700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e801436a0 0x7f2e80153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e8578a700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c005210 0x7f2e8013ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 -- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c005210 msgr2=0x7f2e8013ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c005210 0x7f2e8013ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 -- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e801436a0 msgr2=0x7f2e80153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e801436a0 0x7f2e80153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 -- 172.21.15.97:0/2502720936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e801541d0 con 0x7f2e801423c0 2023-11-24T06:09:40.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e8578a700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c005210 0x7f2e8013ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e85f8b700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e801423c0 0x7f2e80151140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e700 tx=0x7f2e78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e75ffb700 1 -- 172.21.15.97:0/2502720936 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78003e10 con 0x7f2e801423c0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.719+0000 7f2e75ffb700 1 -- 172.21.15.97:0/2502720936 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78004230 con 0x7f2e801423c0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.720+0000 7f2e75ffb700 1 -- 172.21.15.97:0/2502720936 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780045e0 con 0x7f2e801423c0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.720+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e801544b0 con 0x7f2e801423c0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.720+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e80142840 con 0x7f2e801423c0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.721+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e80142b20 con 0x7f2e801423c0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.721+0000 7f2e75ffb700 1 -- 172.21.15.97:0/2502720936 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e78004740 con 0x7f2e801423c0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.721+0000 7f2e75ffb700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e6406aed0 0x7f2e6406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.722+0000 7f2e75ffb700 1 -- 172.21.15.97:0/2502720936 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2e7808a9e0 con 0x7f2e801423c0 2023-11-24T06:09:40.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.722+0000 7f2e8578a700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e6406aed0 0x7f2e6406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:40.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.722+0000 7f2e8578a700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e6406aed0 0x7f2e6406d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:40.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.726+0000 7f2e75ffb700 1 -- 172.21.15.97:0/2502720936 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e78055aa0 con 0x7f2e801423c0 2023-11-24T06:09:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.934+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e80154640 con 0x7f2e801423c0 2023-11-24T06:09:40.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.935+0000 7f2e75ffb700 1 -- 172.21.15.97:0/2502720936 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2e78018070 con 0x7f2e801423c0 2023-11-24T06:09:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e6406aed0 msgr2=0x7f2e6406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e6406aed0 0x7f2e6406d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c009040 comp rx=0 tx=0).stop 2023-11-24T06:09:40.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e801423c0 msgr2=0x7f2e80151140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e801423c0 0x7f2e80151140 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e700 tx=0x7f2e78009750 comp rx=0 tx=0).stop 2023-11-24T06:09:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 shutdown_connections 2023-11-24T06:09:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e801423c0 0x7f2e80151140 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e6406aed0 0x7f2e6406d390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e6c005210 0x7f2e8013ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 --2- 172.21.15.97:0/2502720936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e801436a0 0x7f2e80153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:40.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 >> 172.21.15.97:0/2502720936 conn(0x7f2e80071ac0 msgr2=0x7f2e800b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:40.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 shutdown_connections 2023-11-24T06:09:40.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:40.937+0000 7f2e879ee700 1 -- 172.21.15.97:0/2502720936 wait complete. 2023-11-24T06:09:40.942 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:09:40.952 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-24T06:09:40.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:40.952 DEBUG:teuthology.orchestra.run.smithi183:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-24T06:09:40.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:09:40.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:09:40.996 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:09:41.054 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:09:41.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:41.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.172+0000 7f442b65f700 1 Processor -- start 2023-11-24T06:09:41.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.173+0000 7f442b65f700 1 -- start start 2023-11-24T06:09:41.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.173+0000 7f442b65f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 0x7f44240d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f442b65f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4424162950 con 0x7f44240d44e0 2023-11-24T06:09:41.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f442b65f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44240d2d10 con 0x7f44240b2fc0 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f442b65f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44240cefd0 con 0x7f442415dfc0 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f4429bfc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f44240d44e0 0x7f4424162240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38924/0 (socket says 172.21.15.97:38924) 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f4429bfc700 1 -- 172.21.15.97:0/2404749825 learned_addr learned my addr 172.21.15.97:0/2404749825 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f44293fb700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 0x7f44240d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653662338 0 0) 0x7f4424162950 con 0x7f44240d44e0 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44100035b0 con 0x7f44240d44e0 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190384583 0 0) 0x7f44100035b0 con 0x7f44240d44e0 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 msgr2=0x7f44240d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 0x7f44240d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 >> v1:172.21.15.183:6789/0 conn(0x7f44240b2fc0 legacy=0x7f4424161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240cf2b0 con 0x7f44240d44e0 2023-11-24T06:09:41.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f441c0019f0 con 0x7f44240d44e0 2023-11-24T06:09:41.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 >> v1:172.21.15.97:6789/0 conn(0x7f44240d44e0 legacy=0x7f4424162240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410003a30 0x7f4410003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4410004850 0x7f4410004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 0x7f4410005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44240cefd0 con 0x7f4410003a30 2023-11-24T06:09:41.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441000bb40 con 0x7f4410004850 2023-11-24T06:09:41.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441000bd00 con 0x7f442415dfc0 2023-11-24T06:09:41.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.174+0000 7f44293fb700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410003a30 0x7f4410003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f4429bfc700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 0x7f4410005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f4428bfa700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4410004850 0x7f4410004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f44293fb700 1 -- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 msgr2=0x7f4410005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:41.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f44293fb700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 0x7f4410005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f44293fb700 1 -- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4410004850 msgr2=0x7f4410004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:41.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f44293fb700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4410004850 0x7f4410004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f44293fb700 1 -- 172.21.15.97:0/2404749825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44100035b0 con 0x7f4410003a30 2023-11-24T06:09:41.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f4429bfc700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442415dfc0 0x7f4410005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:41.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f44293fb700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410003a30 0x7f4410003e50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4414009b60 tx=0x7f4410005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e521e7f81584b2db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400e6c0 con 0x7f4410003a30 2023-11-24T06:09:41.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441400e820 con 0x7f4410003a30 2023-11-24T06:09:41.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f441bfff700 1 -- 172.21.15.97:0/2404749825 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400eb90 con 0x7f4410003a30 2023-11-24T06:09:41.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f442b65f700 1 -- 172.21.15.97:0/2404749825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410003a30 msgr2=0x7f4410003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f442b65f700 1 --2- 172.21.15.97:0/2404749825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4410003a30 0x7f4410003e50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4414009b60 tx=0x7f4410005880 comp rx=0 tx=0).stop 2023-11-24T06:09:41.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f4429bfc700 1 -- 172.21.15.97:0/2404749825 reap_dead start 2023-11-24T06:09:41.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f442b65f700 1 -- 172.21.15.97:0/2404749825 shutdown_connections 2023-11-24T06:09:41.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.175+0000 7f442b65f700 1 -- 172.21.15.97:0/2404749825 >> 172.21.15.97:0/2404749825 conn(0x7f442406b410 msgr2=0x7f442406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:41.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.176+0000 7f442b65f700 1 -- 172.21.15.97:0/2404749825 shutdown_connections 2023-11-24T06:09:41.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.176+0000 7f442b65f700 1 -- 172.21.15.97:0/2404749825 wait complete. 2023-11-24T06:09:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.177+0000 7f442b65f700 1 Processor -- start 2023-11-24T06:09:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.177+0000 7f442b65f700 1 -- start start 2023-11-24T06:09:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f442b65f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4410003a30 0x7f44240c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f442b65f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442415f630 0x7f4424160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f442b65f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4424158540 0x7f44240b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f442b65f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44241606c0 con 0x7f4424158540 2023-11-24T06:09:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f442b65f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44240c92c0 con 0x7f442415f630 2023-11-24T06:09:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f442b65f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44240c8410 con 0x7f4410003a30 2023-11-24T06:09:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4429bfc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4424158540 0x7f44240b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4429bfc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4424158540 0x7f44240b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46484/0 (socket says 172.21.15.97:46484) 2023-11-24T06:09:41.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4429bfc700 1 -- 172.21.15.97:0/1587205720 learned_addr learned my addr 172.21.15.97:0/1587205720 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:41.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4428bfa700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442415f630 0x7f4424160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f44293fb700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4410003a30 0x7f44240c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4429bfc700 1 -- 172.21.15.97:0/1587205720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4410003a30 msgr2=0x7f44240c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4429bfc700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4410003a30 0x7f44240c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4429bfc700 1 -- 172.21.15.97:0/1587205720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442415f630 msgr2=0x7f4424160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4429bfc700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442415f630 0x7f4424160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.178+0000 7f4429bfc700 1 -- 172.21.15.97:0/1587205720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240c86b0 con 0x7f4424158540 2023-11-24T06:09:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.179+0000 7f4429bfc700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4424158540 0x7f44240b8d30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f44240d2230 tx=0x7f441c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.179+0000 7f4419ffb700 1 -- 172.21.15.97:0/1587205720 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c00ebc0 con 0x7f4424158540 2023-11-24T06:09:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.179+0000 7f4419ffb700 1 -- 172.21.15.97:0/1587205720 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c00ed20 con 0x7f4424158540 2023-11-24T06:09:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.179+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44240c8990 con 0x7f4424158540 2023-11-24T06:09:41.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.179+0000 7f4419ffb700 1 -- 172.21.15.97:0/1587205720 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c013660 con 0x7f4424158540 2023-11-24T06:09:41.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.179+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44240c8f50 con 0x7f4424158540 2023-11-24T06:09:41.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.180+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4424067f60 con 0x7f4424158540 2023-11-24T06:09:41.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.180+0000 7f4419ffb700 1 -- 172.21.15.97:0/1587205720 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f441c0137c0 con 0x7f4424158540 2023-11-24T06:09:41.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.180+0000 7f4419ffb700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440806aed0 0x7f440806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.181+0000 7f4419ffb700 1 -- 172.21.15.97:0/1587205720 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f441c089b90 con 0x7f4424158540 2023-11-24T06:09:41.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.181+0000 7f44293fb700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440806aed0 0x7f440806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.181+0000 7f44293fb700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440806aed0 0x7f440806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44140090b0 tx=0x7f441401f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.185+0000 7f4419ffb700 1 -- 172.21.15.97:0/1587205720 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f441c054bb0 con 0x7f4424158540 2023-11-24T06:09:41.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.418+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44240c9400 con 0x7f4424158540 2023-11-24T06:09:41.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.418+0000 7f4419ffb700 1 -- 172.21.15.97:0/1587205720 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f441c0581d0 con 0x7f4424158540 2023-11-24T06:09:41.421 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:41.421 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:08:57.902510+0000","modified":"2023-11-24T06:09:00.921422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:09:41.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440806aed0 msgr2=0x7f440806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440806aed0 0x7f440806d390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f44140090b0 tx=0x7f441401f040 comp rx=0 tx=0).stop 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4424158540 msgr2=0x7f44240b8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4424158540 0x7f44240b8d30 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f44240d2230 tx=0x7f441c00e440 comp rx=0 tx=0).stop 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 shutdown_connections 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4424158540 0x7f44240b8d30 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440806aed0 0x7f440806d390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4410003a30 0x7f44240c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 --2- 172.21.15.97:0/1587205720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f442415f630 0x7f4424160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 >> 172.21.15.97:0/1587205720 conn(0x7f442406b410 msgr2=0x7f44240b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 shutdown_connections 2023-11-24T06:09:41.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.420+0000 7f442b65f700 1 -- 172.21.15.97:0/1587205720 wait complete. 2023-11-24T06:09:41.425 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-11-24T06:09:41.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:41.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.553+0000 7f022a7e5700 1 Processor -- start 2023-11-24T06:09:41.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.553+0000 7f022a7e5700 1 -- start start 2023-11-24T06:09:41.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f022a7e5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 0x7f02240b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f022a7e5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02240b2b60 0x7f02240ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f022a7e5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02240cf530 con 0x7f02240cf0d0 2023-11-24T06:09:41.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f022a7e5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022406bc00 con 0x7f02240b2b60 2023-11-24T06:09:41.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f022a7e5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022415e4d0 con 0x7f022415e0f0 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f02237fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02240b2b60 0x7f02240ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f02237fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f02240b2b60 0x7f02240ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46490/0 (socket says 172.21.15.97:46490) 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f02237fe700 1 -- 172.21.15.97:0/228280375 learned_addr learned my addr 172.21.15.97:0/228280375 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f0223fff700 1 --2- 172.21.15.97:0/228280375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 0x7f02240b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f02237fe700 1 -- 172.21.15.97:0/228280375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 msgr2=0x7f02240b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f02237fe700 1 --2- 172.21.15.97:0/228280375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 0x7f02240b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f02237fe700 1 -- 172.21.15.97:0/228280375 >> v1:172.21.15.183:6789/0 conn(0x7f02240cf0d0 legacy=0x7f0224161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.554+0000 7f02237fe700 1 -- 172.21.15.97:0/228280375 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022406aff0 con 0x7f02240b2b60 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f02237fe700 1 --2- 172.21.15.97:0/228280375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240b2b60 0x7f02240ceb90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f02140098f0 tx=0x7f0214009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52867c9eaca67c90 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f0222ffd700 1 -- 172.21.15.97:0/228280375 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214011830 con 0x7f02240b2b60 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f0222ffd700 1 -- 172.21.15.97:0/228280375 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214011990 con 0x7f02240b2b60 2023-11-24T06:09:41.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f0222ffd700 1 -- 172.21.15.97:0/228280375 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214011d20 con 0x7f02240b2b60 2023-11-24T06:09:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f022a7e5700 1 -- 172.21.15.97:0/228280375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240b2b60 msgr2=0x7f02240ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f022a7e5700 1 --2- 172.21.15.97:0/228280375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240b2b60 0x7f02240ceb90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f02140098f0 tx=0x7f0214009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f022a7e5700 1 -- 172.21.15.97:0/228280375 shutdown_connections 2023-11-24T06:09:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f022a7e5700 1 --2- 172.21.15.97:0/228280375 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 0x7f02240b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f022a7e5700 1 --2- 172.21.15.97:0/228280375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240b2b60 0x7f02240ceb90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f022a7e5700 1 -- 172.21.15.97:0/228280375 >> 172.21.15.97:0/228280375 conn(0x7f02240b5ce0 msgr2=0x7f02240b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f022a7e5700 1 -- 172.21.15.97:0/228280375 shutdown_connections 2023-11-24T06:09:41.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.555+0000 7f022a7e5700 1 -- 172.21.15.97:0/228280375 wait complete. 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.556+0000 7f022a7e5700 1 Processor -- start 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.557+0000 7f022a7e5700 1 -- start start 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.557+0000 7f022a7e5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02240b2b60 0x7f02240cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.557+0000 7f022a7e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240cf0d0 0x7f0224158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.557+0000 7f022a7e5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 0x7f02241587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.557+0000 7f022a7e5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02240cd550 con 0x7f02240cf0d0 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.557+0000 7f022a7e5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0224158d30 con 0x7f02240b2b60 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.557+0000 7f022a7e5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02240cc8d0 con 0x7f022415e0f0 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240cf0d0 0x7f0224158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240cf0d0 0x7f0224158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46496/0 (socket says 172.21.15.97:46496) 2023-11-24T06:09:41.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 -- 172.21.15.97:0/3274163957 learned_addr learned my addr 172.21.15.97:0/3274163957 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f0223fff700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02240b2b60 0x7f02240cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f0228d82700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 0x7f02241587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 -- 172.21.15.97:0/3274163957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 msgr2=0x7f02241587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 0x7f02241587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 -- 172.21.15.97:0/3274163957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02240b2b60 msgr2=0x7f02240cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02240b2b60 0x7f02240cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 -- 172.21.15.97:0/3274163957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0214009550 con 0x7f02240cf0d0 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f02237fe700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240cf0d0 0x7f0224158ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0214009c40 tx=0x7f0214004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f0220ff9700 1 -- 172.21.15.97:0/3274163957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02140036a0 con 0x7f02240cf0d0 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f0220ff9700 1 -- 172.21.15.97:0/3274163957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021401e070 con 0x7f02240cf0d0 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02240ccbb0 con 0x7f02240cf0d0 2023-11-24T06:09:41.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f0220ff9700 1 -- 172.21.15.97:0/3274163957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214003a60 con 0x7f02240cf0d0 2023-11-24T06:09:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.558+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02240cd170 con 0x7f02240cf0d0 2023-11-24T06:09:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.559+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02240751e0 con 0x7f02240cf0d0 2023-11-24T06:09:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.560+0000 7f0220ff9700 1 -- 172.21.15.97:0/3274163957 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0214003bc0 con 0x7f02240cf0d0 2023-11-24T06:09:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.560+0000 7f0220ff9700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f020406aed0 0x7f020406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.560+0000 7f0220ff9700 1 -- 172.21.15.97:0/3274163957 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f021408a5f0 con 0x7f02240cf0d0 2023-11-24T06:09:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.560+0000 7f0223fff700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f020406aed0 0x7f020406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.560+0000 7f0223fff700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f020406aed0 0x7f020406d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f022415e9b0 tx=0x7f020c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.564+0000 7f0220ff9700 1 -- 172.21.15.97:0/3274163957 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02140556f0 con 0x7f02240cf0d0 2023-11-24T06:09:41.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.755+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02240683f0 con 0x7f02240cf0d0 2023-11-24T06:09:41.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.756+0000 7f0220ff9700 1 -- 172.21.15.97:0/3274163957 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7f0214020090 con 0x7f02240cf0d0 2023-11-24T06:09:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:41.760 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":22,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:39.752606+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:41.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f020406aed0 msgr2=0x7f020406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f020406aed0 0x7f020406d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f022415e9b0 tx=0x7f020c008040 comp rx=0 tx=0).stop 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240cf0d0 msgr2=0x7f0224158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240cf0d0 0x7f0224158ef0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0214009c40 tx=0x7f0214004750 comp rx=0 tx=0).stop 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 shutdown_connections 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f020406aed0 0x7f020406d390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f022415e0f0 0x7f02241587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02240b2b60 0x7f02240cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 --2- 172.21.15.97:0/3274163957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02240cf0d0 0x7f0224158ef0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 >> 172.21.15.97:0/3274163957 conn(0x7f02240b5ce0 msgr2=0x7f02240b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 shutdown_connections 2023-11-24T06:09:41.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.758+0000 7f022a7e5700 1 -- 172.21.15.97:0/3274163957 wait complete. 2023-11-24T06:09:41.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:08:55.666878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:08:56.875018+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:41.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:41.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.894+0000 7f033604d700 1 Processor -- start 2023-11-24T06:09:41.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.894+0000 7f033604d700 1 -- start start 2023-11-24T06:09:41.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.895+0000 7f033604d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f033015e160 0x7f03300b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.895+0000 7f033604d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f0330073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.895+0000 7f033604d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03300b3c60 con 0x7f03300744d0 2023-11-24T06:09:41.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.895+0000 7f033604d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03300b4500 con 0x7f033015e160 2023-11-24T06:09:41.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.895+0000 7f033604d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0330074960 con 0x7f03300afc90 2023-11-24T06:09:41.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.895+0000 7f032effd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f0330073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.895+0000 7f032f7fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f033015e160 0x7f03300b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.895+0000 7f032effd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f0330073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:33538/0 (socket says 172.21.15.97:33538) 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032effd700 1 -- 172.21.15.97:0/337978742 learned_addr learned my addr 172.21.15.97:0/337978742 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032effd700 1 -- 172.21.15.97:0/337978742 >> v2:172.21.15.183:3300/0 conn(0x7f033015e160 msgr2=0x7f03300b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032effd700 1 --2- 172.21.15.97:0/337978742 >> v2:172.21.15.183:3300/0 conn(0x7f033015e160 0x7f03300b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032effd700 1 -- 172.21.15.97:0/337978742 >> v1:172.21.15.183:6789/0 conn(0x7f03300744d0 legacy=0x7f033006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032effd700 1 -- 172.21.15.97:0/337978742 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03300b58c0 con 0x7f03300afc90 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032f7fe700 1 --2- 172.21.15.97:0/337978742 >> v2:172.21.15.183:3300/0 conn(0x7f033015e160 0x7f03300b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032effd700 1 --2- 172.21.15.97:0/337978742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f0330073f90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03200098f0 tx=0x7f0320009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36b315d3f3f086b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032e7fc700 1 -- 172.21.15.97:0/337978742 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320011830 con 0x7f03300afc90 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032e7fc700 1 -- 172.21.15.97:0/337978742 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0320011990 con 0x7f03300afc90 2023-11-24T06:09:41.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.896+0000 7f032e7fc700 1 -- 172.21.15.97:0/337978742 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320011d20 con 0x7f03300afc90 2023-11-24T06:09:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.897+0000 7f033604d700 1 -- 172.21.15.97:0/337978742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 msgr2=0x7f0330073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.897+0000 7f033604d700 1 --2- 172.21.15.97:0/337978742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f0330073f90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03200098f0 tx=0x7f0320009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.897+0000 7f033604d700 1 -- 172.21.15.97:0/337978742 shutdown_connections 2023-11-24T06:09:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.897+0000 7f033604d700 1 --2- 172.21.15.97:0/337978742 >> v2:172.21.15.183:3300/0 conn(0x7f033015e160 0x7f03300b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.897+0000 7f033604d700 1 --2- 172.21.15.97:0/337978742 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f0330073f90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.897+0000 7f033604d700 1 -- 172.21.15.97:0/337978742 >> 172.21.15.97:0/337978742 conn(0x7f03300d0bf0 msgr2=0x7f03300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.897+0000 7f033604d700 1 -- 172.21.15.97:0/337978742 shutdown_connections 2023-11-24T06:09:41.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.897+0000 7f033604d700 1 -- 172.21.15.97:0/337978742 wait complete. 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.898+0000 7f033604d700 1 Processor -- start 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.898+0000 7f033604d700 1 -- start start 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f033604d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03300744d0 0x7f03300c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f033604d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f03300c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f033604d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033015e160 0x7f03300c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f033604d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03300c7660 con 0x7f033015e160 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f033604d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03300c8830 con 0x7f03300744d0 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f033604d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03300cc220 con 0x7f03300afc90 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033015e160 0x7f03300c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033015e160 0x7f03300c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46510/0 (socket says 172.21.15.97:46510) 2023-11-24T06:09:41.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032ffff700 1 -- 172.21.15.97:0/3576650127 learned_addr learned my addr 172.21.15.97:0/3576650127 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032f7fe700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03300744d0 0x7f03300c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032effd700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f03300c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032ffff700 1 -- 172.21.15.97:0/3576650127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 msgr2=0x7f03300c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032ffff700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f03300c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032ffff700 1 -- 172.21.15.97:0/3576650127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03300744d0 msgr2=0x7f03300c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032ffff700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03300744d0 0x7f03300c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.899+0000 7f032ffff700 1 -- 172.21.15.97:0/3576650127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180094a0 con 0x7f033015e160 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.900+0000 7f032ffff700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033015e160 0x7f03300c9b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f032800cc80 tx=0x7f032800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.900+0000 7f0327fff700 1 -- 172.21.15.97:0/3576650127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328003c00 con 0x7f033015e160 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.900+0000 7f0327fff700 1 -- 172.21.15.97:0/3576650127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0328003d60 con 0x7f033015e160 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.900+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0320009550 con 0x7f033015e160 2023-11-24T06:09:41.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.900+0000 7f0327fff700 1 -- 172.21.15.97:0/3576650127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032800ab50 con 0x7f033015e160 2023-11-24T06:09:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.900+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03300c9020 con 0x7f033015e160 2023-11-24T06:09:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.901+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033015d360 con 0x7f033015e160 2023-11-24T06:09:41.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.901+0000 7f0327fff700 1 -- 172.21.15.97:0/3576650127 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f032800acb0 con 0x7f033015e160 2023-11-24T06:09:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.901+0000 7f0327fff700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f031006aed0 0x7f031006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.902+0000 7f0327fff700 1 -- 172.21.15.97:0/3576650127 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0328089740 con 0x7f033015e160 2023-11-24T06:09:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.902+0000 7f032f7fe700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f031006aed0 0x7f031006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:41.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.902+0000 7f032f7fe700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f031006aed0 0x7f031006d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0318009cd0 tx=0x7f0318009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:41.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:41.906+0000 7f0327fff700 1 -- 172.21.15.97:0/3576650127 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03280546f0 con 0x7f033015e160 2023-11-24T06:09:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.094+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0330068fb0 con 0x7f033015e160 2023-11-24T06:09:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.094+0000 7f0327fff700 1 -- 172.21.15.97:0/3576650127 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0328057d10 con 0x7f033015e160 2023-11-24T06:09:42.097 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f031006aed0 msgr2=0x7f031006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f031006aed0 0x7f031006d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0318009cd0 tx=0x7f0318009210 comp rx=0 tx=0).stop 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033015e160 msgr2=0x7f03300c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033015e160 0x7f03300c9b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f032800cc80 tx=0x7f032800cf90 comp rx=0 tx=0).stop 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 shutdown_connections 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033015e160 0x7f03300c9b00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03300744d0 0x7f03300c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f031006aed0 0x7f031006d390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 --2- 172.21.15.97:0/3576650127 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f03300afc90 0x7f03300c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 >> 172.21.15.97:0/3576650127 conn(0x7f03300d0bf0 msgr2=0x7f0330071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:42.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 shutdown_connections 2023-11-24T06:09:42.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.097+0000 7f033604d700 1 -- 172.21.15.97:0/3576650127 wait complete. 2023-11-24T06:09:42.111 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:09:42.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.231+0000 7fcac1f3e700 1 Processor -- start 2023-11-24T06:09:42.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.231+0000 7fcac1f3e700 1 -- start start 2023-11-24T06:09:42.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcac1f3e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcabc0746a0 0x7fcabc0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcac1f3e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc0b3c60 con 0x7fcabc15e160 2023-11-24T06:09:42.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcac1f3e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc0b58c0 con 0x7fcabc0afc90 2023-11-24T06:09:42.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcac1f3e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0b4500 con 0x7fcabc0746a0 2023-11-24T06:09:42.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcabb7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcabc15e160 0x7fcabc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38968/0 (socket says 172.21.15.97:38968) 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcabb7fe700 1 -- 172.21.15.97:0/692275296 learned_addr learned my addr 172.21.15.97:0/692275296 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680309499 0 0) 0x7fcabc0b3c60 con 0x7fcabc15e160 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa80035b0 con 0x7fcabc15e160 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.232+0000 7fcabaffd700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcabc0746a0 0x7fcabc0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132600838 0 0) 0x7fcaa80035b0 con 0x7fcabc15e160 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcabc0746a0 msgr2=0x7fcabc0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcabc0746a0 0x7fcabc0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 >> v1:172.21.15.183:6789/0 conn(0x7fcabc0afc90 legacy=0x7fcabc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc0b0140 con 0x7fcabc15e160 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fcaa4001670 con 0x7fcabc15e160 2023-11-24T06:09:42.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 >> v1:172.21.15.97:6789/0 conn(0x7fcabc15e160 legacy=0x7fcabc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa8003a30 0x7fcaa8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcabc0746a0 0x7fcaa8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa8004d90 0x7fcaa8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0b4500 con 0x7fcaa8003a30 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa800bac0 con 0x7fcaa8004d90 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa800bc80 con 0x7fcabc0746a0 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabb7fe700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa8003a30 0x7fcaa8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabaffd700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcabc0746a0 0x7fcaa8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabbfff700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa8004d90 0x7fcaa8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabb7fe700 1 -- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcabc0746a0 msgr2=0x7fcaa8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabb7fe700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcabc0746a0 0x7fcaa8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabb7fe700 1 -- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa8004d90 msgr2=0x7fcaa8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabb7fe700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaa8004d90 0x7fcaa8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabb7fe700 1 -- 172.21.15.97:0/692275296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80035b0 con 0x7fcaa8003a30 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcabb7fe700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa8003a30 0x7fcaa8003e50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400c270 tx=0x7fcaa8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fc50b6ac3d6491 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4007c10 con 0x7fcaa8003a30 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa400cd90 con 0x7fcaa8003a30 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.233+0000 7fcaba7fc700 1 -- 172.21.15.97:0/692275296 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4005ba0 con 0x7fcaa8003a30 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.234+0000 7fcac1f3e700 1 -- 172.21.15.97:0/692275296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa8003a30 msgr2=0x7fcaa8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.234+0000 7fcac1f3e700 1 --2- 172.21.15.97:0/692275296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa8003a30 0x7fcaa8003e50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcaa400c270 tx=0x7fcaa8005880 comp rx=0 tx=0).stop 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.234+0000 7fcabbfff700 1 -- 172.21.15.97:0/692275296 reap_dead start 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.234+0000 7fcac1f3e700 1 -- 172.21.15.97:0/692275296 shutdown_connections 2023-11-24T06:09:42.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.234+0000 7fcac1f3e700 1 -- 172.21.15.97:0/692275296 >> 172.21.15.97:0/692275296 conn(0x7fcabc0d0bf0 msgr2=0x7fcabc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:42.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.234+0000 7fcac1f3e700 1 -- 172.21.15.97:0/692275296 shutdown_connections 2023-11-24T06:09:42.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.234+0000 7fcac1f3e700 1 -- 172.21.15.97:0/692275296 wait complete. 2023-11-24T06:09:42.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.235+0000 7fcac1f3e700 1 Processor -- start 2023-11-24T06:09:42.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.235+0000 7fcac1f3e700 1 -- start start 2023-11-24T06:09:42.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcac1f3e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcaa8003a30 0x7fcabc157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcac1f3e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcabc1526b0 0x7fcabc15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcac1f3e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcabc15b160 0x7fcabc1f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcac1f3e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc15b610 con 0x7fcabc1526b0 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcac1f3e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc1567d0 con 0x7fcabc15b160 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcac1f3e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc1f46e0 con 0x7fcaa8003a30 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcabaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcabc1526b0 0x7fcabc15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcabaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcabc1526b0 0x7fcabc15ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46530/0 (socket says 172.21.15.97:46530) 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcabaffd700 1 -- 172.21.15.97:0/604074488 learned_addr learned my addr 172.21.15.97:0/604074488 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcabbfff700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcabc15b160 0x7fcabc1f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcabb7fe700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcaa8003a30 0x7fcabc157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcabaffd700 1 -- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcaa8003a30 msgr2=0x7fcabc157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.236+0000 7fcabaffd700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcaa8003a30 0x7fcabc157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcabaffd700 1 -- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcabc15b160 msgr2=0x7fcabc1f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcabaffd700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcabc15b160 0x7fcabc1f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcabaffd700 1 -- 172.21.15.97:0/604074488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc156a90 con 0x7fcabc1526b0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcabb7fe700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcaa8003a30 0x7fcabc157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcabaffd700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcabc1526b0 0x7fcabc15ac20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcaac011660 tx=0x7fcaac011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcab3fff700 1 -- 172.21.15.97:0/604074488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac015870 con 0x7fcabc1526b0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcab3fff700 1 -- 172.21.15.97:0/604074488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac0159d0 con 0x7fcabc1526b0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcabc156d70 con 0x7fcabc1526b0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcab3fff700 1 -- 172.21.15.97:0/604074488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac015d40 con 0x7fcabc1526b0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.237+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc152d10 con 0x7fcabc1526b0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.238+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcabc068fb0 con 0x7fcabc1526b0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.238+0000 7fcab3fff700 1 -- 172.21.15.97:0/604074488 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcaac013400 con 0x7fcabc1526b0 2023-11-24T06:09:42.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.239+0000 7fcab3fff700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fca9c06aed0 0x7fca9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.239+0000 7fcab3fff700 1 -- 172.21.15.97:0/604074488 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcaac089750 con 0x7fcabc1526b0 2023-11-24T06:09:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.239+0000 7fcabb7fe700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fca9c06aed0 0x7fca9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.239+0000 7fcabb7fe700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fca9c06aed0 0x7fca9c06d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4007f50 tx=0x7fcaa4007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.243+0000 7fcab3fff700 1 -- 172.21.15.97:0/604074488 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaac054780 con 0x7fcabc1526b0 2023-11-24T06:09:42.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.434+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcabc152d10 con 0x7fcabc1526b0 2023-11-24T06:09:42.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.435+0000 7fcab3fff700 1 -- 172.21.15.97:0/604074488 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcabc152d10 con 0x7fcabc1526b0 2023-11-24T06:09:42.438 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fca9c06aed0 msgr2=0x7fca9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fca9c06aed0 0x7fca9c06d390 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4007f50 tx=0x7fcaa4007000 comp rx=0 tx=0).stop 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcabc1526b0 msgr2=0x7fcabc15ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcabc1526b0 0x7fcabc15ac20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcaac011660 tx=0x7fcaac011970 comp rx=0 tx=0).stop 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 shutdown_connections 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcabc15b160 0x7fcabc1f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fca9c06aed0 0x7fca9c06d390 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcaa8003a30 0x7fcabc157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 --2- 172.21.15.97:0/604074488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcabc1526b0 0x7fcabc15ac20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 >> 172.21.15.97:0/604074488 conn(0x7fcabc0d0bf0 msgr2=0x7fcabc06e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 shutdown_connections 2023-11-24T06:09:42.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.437+0000 7fcac1f3e700 1 -- 172.21.15.97:0/604074488 wait complete. 2023-11-24T06:09:42.452 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:09:42.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.572+0000 7f3748825700 1 Processor -- start 2023-11-24T06:09:42.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.572+0000 7f3748825700 1 -- start start 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.573+0000 7f3748825700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37440746a0 0x7f37440b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.573+0000 7f3748825700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37440b3c60 con 0x7f374415e160 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.573+0000 7f3748825700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37440b58c0 con 0x7f37440afc90 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.573+0000 7f3748825700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37440b4500 con 0x7f37440746a0 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.573+0000 7f3741d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37440746a0 0x7f37440b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.573+0000 7f374259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f374415e160 0x7f3744073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:38988/0 (socket says 172.21.15.97:38988) 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.573+0000 7f3741d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f37440746a0 0x7f37440b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46538/0 (socket says 172.21.15.97:46538) 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.573+0000 7f374259c700 1 -- 172.21.15.97:0/3189051368 learned_addr learned my addr 172.21.15.97:0/3189051368 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f3741d9b700 1 -- 172.21.15.97:0/3189051368 >> v1:172.21.15.183:6789/0 conn(0x7f37440afc90 legacy=0x7f374406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:42.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f3741d9b700 1 -- 172.21.15.97:0/3189051368 >> v1:172.21.15.97:6789/0 conn(0x7f374415e160 legacy=0x7f3744073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f3741d9b700 1 -- 172.21.15.97:0/3189051368 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374406bdc0 con 0x7f37440746a0 2023-11-24T06:09:42.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f3741d9b700 1 --2- 172.21.15.97:0/3189051368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37440746a0 0x7f37440b5da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3734009840 tx=0x7f3734009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60635cc63533ce59 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f374159a700 1 -- 172.21.15.97:0/3189051368 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734004230 con 0x7f37440746a0 2023-11-24T06:09:42.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f374159a700 1 -- 172.21.15.97:0/3189051368 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734004390 con 0x7f37440746a0 2023-11-24T06:09:42.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f374159a700 1 -- 172.21.15.97:0/3189051368 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340036a0 con 0x7f37440746a0 2023-11-24T06:09:42.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f3748825700 1 -- 172.21.15.97:0/3189051368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37440746a0 msgr2=0x7f37440b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.574+0000 7f3748825700 1 --2- 172.21.15.97:0/3189051368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37440746a0 0x7f37440b5da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3734009840 tx=0x7f3734009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:42.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.575+0000 7f3748825700 1 -- 172.21.15.97:0/3189051368 shutdown_connections 2023-11-24T06:09:42.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.575+0000 7f3748825700 1 --2- 172.21.15.97:0/3189051368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37440746a0 0x7f37440b5da0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.575+0000 7f3748825700 1 -- 172.21.15.97:0/3189051368 >> 172.21.15.97:0/3189051368 conn(0x7f37440d0bf0 msgr2=0x7f37440d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:42.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.575+0000 7f3748825700 1 -- 172.21.15.97:0/3189051368 shutdown_connections 2023-11-24T06:09:42.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.575+0000 7f3748825700 1 -- 172.21.15.97:0/3189051368 wait complete. 2023-11-24T06:09:42.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.576+0000 7f3748825700 1 Processor -- start 2023-11-24T06:09:42.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.576+0000 7f3748825700 1 -- start start 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3748825700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37440746a0 0x7f3744155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3748825700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37440afc90 0x7f3744160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3748825700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374415e160 0x7f374415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3748825700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37441594f0 con 0x7f374415e160 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3748825700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3744160ae0 con 0x7f37440746a0 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3748825700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3744155f40 con 0x7f37440afc90 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3742d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374415e160 0x7f374415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3742d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374415e160 0x7f374415cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46546/0 (socket says 172.21.15.97:46546) 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.577+0000 7f3742d9d700 1 -- 172.21.15.97:0/1654814904 learned_addr learned my addr 172.21.15.97:0/1654814904 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:42.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3741d9b700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37440afc90 0x7f3744160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f374259c700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37440746a0 0x7f3744155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3742d9d700 1 -- 172.21.15.97:0/1654814904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37440afc90 msgr2=0x7f3744160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3742d9d700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37440afc90 0x7f3744160c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3742d9d700 1 -- 172.21.15.97:0/1654814904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37440746a0 msgr2=0x7f3744155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3742d9d700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37440746a0 0x7f3744155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3742d9d700 1 -- 172.21.15.97:0/1654814904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37340094a0 con 0x7f374415e160 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3742d9d700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374415e160 0x7f374415cab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f373c00c7d0 tx=0x7f373c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3732ffd700 1 -- 172.21.15.97:0/1654814904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c003e10 con 0x7f374415e160 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3732ffd700 1 -- 172.21.15.97:0/1654814904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c005070 con 0x7f374415e160 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37441562a0 con 0x7f374415e160 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3732ffd700 1 -- 172.21.15.97:0/1654814904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c005420 con 0x7f374415e160 2023-11-24T06:09:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.578+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3744160800 con 0x7f374415e160 2023-11-24T06:09:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.579+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3744156100 con 0x7f374415e160 2023-11-24T06:09:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.580+0000 7f3732ffd700 1 -- 172.21.15.97:0/1654814904 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f373c005580 con 0x7f374415e160 2023-11-24T06:09:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.580+0000 7f3732ffd700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f372406aed0 0x7f372406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.580+0000 7f374259c700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f372406aed0 0x7f372406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.580+0000 7f3732ffd700 1 -- 172.21.15.97:0/1654814904 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f373c088770 con 0x7f374415e160 2023-11-24T06:09:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.580+0000 7f374259c700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f372406aed0 0x7f372406d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3738000f60 tx=0x7f3738007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.585+0000 7f3732ffd700 1 -- 172.21.15.97:0/1654814904 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3744156100 con 0x7f374415e160 2023-11-24T06:09:42.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.774+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3744156100 con 0x7f374415e160 2023-11-24T06:09:42.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.774+0000 7f3732ffd700 1 -- 172.21.15.97:0/1654814904 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3744156100 con 0x7f374415e160 2023-11-24T06:09:42.778 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:09:42.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f372406aed0 msgr2=0x7f372406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f372406aed0 0x7f372406d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3738000f60 tx=0x7f3738007f10 comp rx=0 tx=0).stop 2023-11-24T06:09:42.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374415e160 msgr2=0x7f374415cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374415e160 0x7f374415cab0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f373c00c7d0 tx=0x7f373c00cae0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 shutdown_connections 2023-11-24T06:09:42.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374415e160 0x7f374415cab0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37440746a0 0x7f3744155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f372406aed0 0x7f372406d390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 --2- 172.21.15.97:0/1654814904 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37440afc90 0x7f3744160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 >> 172.21.15.97:0/1654814904 conn(0x7f37440d0bf0 msgr2=0x7f37440ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:42.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 shutdown_connections 2023-11-24T06:09:42.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.776+0000 7f3748825700 1 -- 172.21.15.97:0/1654814904 wait complete. 2023-11-24T06:09:42.792 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:42.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.909+0000 7f2e34741700 1 Processor -- start 2023-11-24T06:09:42.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.909+0000 7f2e34741700 1 -- start start 2023-11-24T06:09:42.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e34741700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e2c15bdd0 0x7f2e2c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e34741700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2e2c0d1c70 0x7f2e2c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e34741700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c0b22d0 con 0x7f2e2c0b3370 2023-11-24T06:09:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e34741700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b2f50 con 0x7f2e2c15bdd0 2023-11-24T06:09:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e34741700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c0d2640 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e324dd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e2c15bdd0 0x7f2e2c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e324dd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2e2c15bdd0 0x7f2e2c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46550/0 (socket says 172.21.15.97:46550) 2023-11-24T06:09:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e324dd700 1 -- 172.21.15.97:0/924208981 learned_addr learned my addr 172.21.15.97:0/924208981 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e32cde700 1 --2- 172.21.15.97:0/924208981 >> v2:172.21.15.183:3300/0 conn(0x7f2e2c0d1c70 0x7f2e2c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e324dd700 1 -- 172.21.15.97:0/924208981 >> v2:172.21.15.183:3300/0 conn(0x7f2e2c0d1c70 msgr2=0x7f2e2c0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e324dd700 1 --2- 172.21.15.97:0/924208981 >> v2:172.21.15.183:3300/0 conn(0x7f2e2c0d1c70 0x7f2e2c0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e324dd700 1 -- 172.21.15.97:0/924208981 >> v1:172.21.15.183:6789/0 conn(0x7f2e2c0b3370 legacy=0x7f2e2c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e324dd700 1 -- 172.21.15.97:0/924208981 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0b25b0 con 0x7f2e2c15bdd0 2023-11-24T06:09:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e32cde700 1 --2- 172.21.15.97:0/924208981 >> v2:172.21.15.183:3300/0 conn(0x7f2e2c0d1c70 0x7f2e2c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e324dd700 1 --2- 172.21.15.97:0/924208981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c15bdd0 0x7f2e2c1604e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c160a50 tx=0x7f2e28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8100c70cb26823 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e314db700 1 -- 172.21.15.97:0/924208981 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28004230 con 0x7f2e2c15bdd0 2023-11-24T06:09:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.910+0000 7f2e314db700 1 -- 172.21.15.97:0/924208981 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e28004390 con 0x7f2e2c15bdd0 2023-11-24T06:09:42.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e314db700 1 -- 172.21.15.97:0/924208981 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e280036a0 con 0x7f2e2c15bdd0 2023-11-24T06:09:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e34741700 1 -- 172.21.15.97:0/924208981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c15bdd0 msgr2=0x7f2e2c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e34741700 1 --2- 172.21.15.97:0/924208981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c15bdd0 0x7f2e2c1604e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c160a50 tx=0x7f2e28009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e34741700 1 -- 172.21.15.97:0/924208981 shutdown_connections 2023-11-24T06:09:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e34741700 1 --2- 172.21.15.97:0/924208981 >> v2:172.21.15.183:3300/0 conn(0x7f2e2c0d1c70 0x7f2e2c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e34741700 1 --2- 172.21.15.97:0/924208981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c15bdd0 0x7f2e2c1604e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e34741700 1 -- 172.21.15.97:0/924208981 >> 172.21.15.97:0/924208981 conn(0x7f2e2c0731a0 msgr2=0x7f2e2c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:42.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e34741700 1 -- 172.21.15.97:0/924208981 shutdown_connections 2023-11-24T06:09:42.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.911+0000 7f2e34741700 1 -- 172.21.15.97:0/924208981 wait complete. 2023-11-24T06:09:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.912+0000 7f2e34741700 1 Processor -- start 2023-11-24T06:09:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.912+0000 7f2e34741700 1 -- start start 2023-11-24T06:09:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e34741700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e2c0b3370 0x7f2e2c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e34741700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d1c70 0x7f2e2c0ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e34741700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e2c15bdd0 0x7f2e2c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e34741700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0c8010 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e34741700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0c9250 con 0x7f2e2c15bdd0 2023-11-24T06:09:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e34741700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0cbc20 con 0x7f2e2c0b3370 2023-11-24T06:09:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e31cdc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d1c70 0x7f2e2c0ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e31cdc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d1c70 0x7f2e2c0ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46558/0 (socket says 172.21.15.97:46558) 2023-11-24T06:09:42.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e31cdc700 1 -- 172.21.15.97:0/2380363273 learned_addr learned my addr 172.21.15.97:0/2380363273 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e32cde700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e2c15bdd0 0x7f2e2c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e324dd700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e2c0b3370 0x7f2e2c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e31cdc700 1 -- 172.21.15.97:0/2380363273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e2c0b3370 msgr2=0x7f2e2c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e31cdc700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e2c0b3370 0x7f2e2c0cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e31cdc700 1 -- 172.21.15.97:0/2380363273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e2c15bdd0 msgr2=0x7f2e2c0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e31cdc700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e2c15bdd0 0x7f2e2c0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:42.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.913+0000 7f2e31cdc700 1 -- 172.21.15.97:0/2380363273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28009550 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.914+0000 7f2e31cdc700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d1c70 0x7f2e2c0ca690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c4a0 tx=0x7f2e1c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.914+0000 7f2e22ffd700 1 -- 172.21.15.97:0/2380363273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c011790 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.914+0000 7f2e22ffd700 1 -- 172.21.15.97:0/2380363273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0118f0 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.914+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c0c7a50 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.914+0000 7f2e22ffd700 1 -- 172.21.15.97:0/2380363273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c011ca0 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.914+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c0c8ee0 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.915+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c0c93c0 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.915+0000 7f2e22ffd700 1 -- 172.21.15.97:0/2380363273 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e1c01c460 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.915+0000 7f2e22ffd700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e1006aed0 0x7f2e1006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:42.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.916+0000 7f2e22ffd700 1 -- 172.21.15.97:0/2380363273 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2e1c0893e0 con 0x7f2e2c0d1c70 2023-11-24T06:09:42.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.916+0000 7f2e324dd700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e1006aed0 0x7f2e1006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:42.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.916+0000 7f2e324dd700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e1006aed0 0x7f2e1006d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000c00 tx=0x7f2e28008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:42.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:42.920+0000 7f2e22ffd700 1 -- 172.21.15.97:0/2380363273 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e1c0544a0 con 0x7f2e2c0d1c70 2023-11-24T06:09:43.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.154+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e2c067f60 con 0x7f2e2c0d1c70 2023-11-24T06:09:43.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.155+0000 7f2e22ffd700 1 -- 172.21.15.97:0/2380363273 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f2e1c057ac0 con 0x7f2e2c0d1c70 2023-11-24T06:09:43.158 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:43.158 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:08:57.902510+0000","modified":"2023-11-24T06:09:00.921422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:09:43.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e1006aed0 msgr2=0x7f2e1006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e1006aed0 0x7f2e1006d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000c00 tx=0x7f2e28008030 comp rx=0 tx=0).stop 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d1c70 msgr2=0x7f2e2c0ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d1c70 0x7f2e2c0ca690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c4a0 tx=0x7f2e1c00c860 comp rx=0 tx=0).stop 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 shutdown_connections 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e2c15bdd0 0x7f2e2c0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e1006aed0 0x7f2e1006d390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e2c0b3370 0x7f2e2c0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 --2- 172.21.15.97:0/2380363273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e2c0d1c70 0x7f2e2c0ca690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 >> 172.21.15.97:0/2380363273 conn(0x7f2e2c0731a0 msgr2=0x7f2e2c0b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 shutdown_connections 2023-11-24T06:09:43.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.157+0000 7f2e34741700 1 -- 172.21.15.97:0/2380363273 wait complete. 2023-11-24T06:09:43.162 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-11-24T06:09:43.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:43.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.290+0000 7fba36c4a700 1 Processor -- start 2023-11-24T06:09:43.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.290+0000 7fba36c4a700 1 -- start start 2023-11-24T06:09:43.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba36c4a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba3015e080 0x7fba300b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba36c4a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fba3006c710 0x7fba300afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba36c4a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba300ae430 con 0x7fba300af4d0 2023-11-24T06:09:43.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba36c4a700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba300af0b0 con 0x7fba3015e080 2023-11-24T06:09:43.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba36c4a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba300b05a0 con 0x7fba3006c710 2023-11-24T06:09:43.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba349e6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba3015e080 0x7fba300b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:43.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba349e6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba3015e080 0x7fba300b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46564/0 (socket says 172.21.15.97:46564) 2023-11-24T06:09:43.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba349e6700 1 -- 172.21.15.97:0/3052166647 learned_addr learned my addr 172.21.15.97:0/3052166647 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:43.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba349e6700 1 -- 172.21.15.97:0/3052166647 >> v2:172.21.15.183:3300/0 conn(0x7fba3006c710 msgr2=0x7fba300afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:43.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba349e6700 1 --2- 172.21.15.97:0/3052166647 >> v2:172.21.15.183:3300/0 conn(0x7fba3006c710 0x7fba300afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba349e6700 1 -- 172.21.15.97:0/3052166647 >> v1:172.21.15.183:6789/0 conn(0x7fba300af4d0 legacy=0x7fba3006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:43.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba349e6700 1 -- 172.21.15.97:0/3052166647 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba300ae710 con 0x7fba3015e080 2023-11-24T06:09:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba349e6700 1 --2- 172.21.15.97:0/3052166647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba3015e080 0x7fba300b5050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fba3006e060 tx=0x7fba24009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf80e702717c738f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba2f7fe700 1 -- 172.21.15.97:0/3052166647 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24004230 con 0x7fba3015e080 2023-11-24T06:09:43.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba2f7fe700 1 -- 172.21.15.97:0/3052166647 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba24004390 con 0x7fba3015e080 2023-11-24T06:09:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.291+0000 7fba2f7fe700 1 -- 172.21.15.97:0/3052166647 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240036a0 con 0x7fba3015e080 2023-11-24T06:09:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.292+0000 7fba36c4a700 1 -- 172.21.15.97:0/3052166647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba3015e080 msgr2=0x7fba300b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.292+0000 7fba36c4a700 1 --2- 172.21.15.97:0/3052166647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba3015e080 0x7fba300b5050 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fba3006e060 tx=0x7fba24009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:43.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.292+0000 7fba36c4a700 1 -- 172.21.15.97:0/3052166647 shutdown_connections 2023-11-24T06:09:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.292+0000 7fba36c4a700 1 --2- 172.21.15.97:0/3052166647 >> v2:172.21.15.183:3300/0 conn(0x7fba3006c710 0x7fba300afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.292+0000 7fba36c4a700 1 --2- 172.21.15.97:0/3052166647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba3015e080 0x7fba300b5050 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.292+0000 7fba36c4a700 1 -- 172.21.15.97:0/3052166647 >> 172.21.15.97:0/3052166647 conn(0x7fba30071b90 msgr2=0x7fba30071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.292+0000 7fba36c4a700 1 -- 172.21.15.97:0/3052166647 shutdown_connections 2023-11-24T06:09:43.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.292+0000 7fba36c4a700 1 -- 172.21.15.97:0/3052166647 wait complete. 2023-11-24T06:09:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.293+0000 7fba36c4a700 1 Processor -- start 2023-11-24T06:09:43.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.293+0000 7fba36c4a700 1 -- start start 2023-11-24T06:09:43.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba36c4a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba3006c710 0x7fba3014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba36c4a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba300af4d0 0x7fba3014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba36c4a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3015e080 0x7fba300cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba36c4a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba300c7660 con 0x7fba300af4d0 2023-11-24T06:09:43.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba36c4a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba300cfa20 con 0x7fba3015e080 2023-11-24T06:09:43.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba36c4a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba300c6cc0 con 0x7fba3006c710 2023-11-24T06:09:43.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba2ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba300af4d0 0x7fba3014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:43.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba2ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba300af4d0 0x7fba3014c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46570/0 (socket says 172.21.15.97:46570) 2023-11-24T06:09:43.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba2ffff700 1 -- 172.21.15.97:0/404067520 learned_addr learned my addr 172.21.15.97:0/404067520 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:43.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba351e7700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3015e080 0x7fba300cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:43.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba349e6700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba3006c710 0x7fba3014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:43.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba2ffff700 1 -- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba3006c710 msgr2=0x7fba3014c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba2ffff700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba3006c710 0x7fba3014c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba2ffff700 1 -- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3015e080 msgr2=0x7fba300cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba2ffff700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3015e080 0x7fba300cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba2ffff700 1 -- 172.21.15.97:0/404067520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24009550 con 0x7fba300af4d0 2023-11-24T06:09:43.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.294+0000 7fba351e7700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3015e080 0x7fba300cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:43.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.295+0000 7fba2ffff700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba300af4d0 0x7fba3014c0f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c4a0 tx=0x7fba1c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:43.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.295+0000 7fba2d7fa700 1 -- 172.21.15.97:0/404067520 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0054e0 con 0x7fba300af4d0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.295+0000 7fba2d7fa700 1 -- 172.21.15.97:0/404067520 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c003680 con 0x7fba300af4d0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.295+0000 7fba2d7fa700 1 -- 172.21.15.97:0/404067520 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c003a10 con 0x7fba300af4d0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.295+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba300c6fa0 con 0x7fba300af4d0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.295+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3014cf30 con 0x7fba300af4d0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.296+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba300753f0 con 0x7fba300af4d0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.296+0000 7fba2d7fa700 1 -- 172.21.15.97:0/404067520 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba1c003b70 con 0x7fba300af4d0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.296+0000 7fba2d7fa700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fba1406aed0 0x7fba1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.296+0000 7fba2d7fa700 1 -- 172.21.15.97:0/404067520 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fba1c0887a0 con 0x7fba300af4d0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.297+0000 7fba349e6700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fba1406aed0 0x7fba1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.297+0000 7fba349e6700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fba1406aed0 0x7fba1406d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba24009130 tx=0x7fba24008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:43.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.301+0000 7fba2d7fa700 1 -- 172.21.15.97:0/404067520 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba1c053860 con 0x7fba300af4d0 2023-11-24T06:09:43.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.490+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba3006e590 con 0x7fba300af4d0 2023-11-24T06:09:43.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.491+0000 7fba2d7fa700 1 -- 172.21.15.97:0/404067520 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7fba1c056e80 con 0x7fba300af4d0 2023-11-24T06:09:43.495 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:43.495 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":22,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:39.752606+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fba1406aed0 msgr2=0x7fba1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fba1406aed0 0x7fba1406d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba24009130 tx=0x7fba24008030 comp rx=0 tx=0).stop 2023-11-24T06:09:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba300af4d0 msgr2=0x7fba3014c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba300af4d0 0x7fba3014c0f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c4a0 tx=0x7fba1c00c860 comp rx=0 tx=0).stop 2023-11-24T06:09:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 shutdown_connections 2023-11-24T06:09:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba3015e080 0x7fba300cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fba1406aed0 0x7fba1406d390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fba3006c710 0x7fba3014c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 --2- 172.21.15.97:0/404067520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba300af4d0 0x7fba3014c0f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 >> 172.21.15.97:0/404067520 conn(0x7fba30071b90 msgr2=0x7fba30160470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:43.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 shutdown_connections 2023-11-24T06:09:43.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.494+0000 7fba36c4a700 1 -- 172.21.15.97:0/404067520 wait complete. 2023-11-24T06:09:43.509 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:09:43.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:43.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.626+0000 7fa866bfb700 1 Processor -- start 2023-11-24T06:09:43.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.626+0000 7fa866bfb700 1 -- start start 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa866bfb700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa86015c160 0x7fa8600622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa866bfb700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86015c540 con 0x7fa8600b2870 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa866bfb700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8600b2ee0 con 0x7fa8600b04b0 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa866bfb700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8600add40 con 0x7fa86015c160 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa864997700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa86015c160 0x7fa8600622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa85ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa8600b2870 0x7fa8600afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39030/0 (socket says 172.21.15.97:39030) 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa85ffff700 1 -- 172.21.15.97:0/4217119038 learned_addr learned my addr 172.21.15.97:0/4217119038 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa864997700 1 -- 172.21.15.97:0/4217119038 >> v1:172.21.15.183:6789/0 conn(0x7fa8600b04b0 legacy=0x7fa860165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa864997700 1 -- 172.21.15.97:0/4217119038 >> v1:172.21.15.97:6789/0 conn(0x7fa8600b2870 legacy=0x7fa8600afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa864997700 1 -- 172.21.15.97:0/4217119038 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600ad410 con 0x7fa86015c160 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa85f7fe700 1 -- 172.21.15.97:0/4217119038 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595618826 0 0) 0x7fa86015c540 con 0x7fa8600b2870 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa864997700 1 --2- 172.21.15.97:0/4217119038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86015c160 0x7fa8600622a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa854009840 tx=0x7fa854009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0b966d15557ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa85f7fe700 1 -- 172.21.15.97:0/4217119038 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854004230 con 0x7fa86015c160 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa85f7fe700 1 -- 172.21.15.97:0/4217119038 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854004390 con 0x7fa86015c160 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.627+0000 7fa85f7fe700 1 -- 172.21.15.97:0/4217119038 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540036a0 con 0x7fa86015c160 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.628+0000 7fa866bfb700 1 -- 172.21.15.97:0/4217119038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86015c160 msgr2=0x7fa8600622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.628+0000 7fa866bfb700 1 --2- 172.21.15.97:0/4217119038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86015c160 0x7fa8600622a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa854009840 tx=0x7fa854009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.628+0000 7fa866bfb700 1 -- 172.21.15.97:0/4217119038 shutdown_connections 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.628+0000 7fa866bfb700 1 --2- 172.21.15.97:0/4217119038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa86015c160 0x7fa8600622a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.628+0000 7fa866bfb700 1 -- 172.21.15.97:0/4217119038 >> 172.21.15.97:0/4217119038 conn(0x7fa8600b57b0 msgr2=0x7fa8600b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.628+0000 7fa866bfb700 1 -- 172.21.15.97:0/4217119038 shutdown_connections 2023-11-24T06:09:43.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.628+0000 7fa866bfb700 1 -- 172.21.15.97:0/4217119038 wait complete. 2023-11-24T06:09:43.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.629+0000 7fa866bfb700 1 Processor -- start 2023-11-24T06:09:43.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.629+0000 7fa866bfb700 1 -- start start 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.630+0000 7fa866bfb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8600b04b0 0x7fa860158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.630+0000 7fa866bfb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8600b2870 0x7fa86015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.630+0000 7fa866bfb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa86015c160 0x7fa860156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.630+0000 7fa866bfb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86015d590 con 0x7fa8600b04b0 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.630+0000 7fa866bfb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601f1ae0 con 0x7fa86015c160 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.630+0000 7fa866bfb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601f1dc0 con 0x7fa8600b2870 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8600b04b0 0x7fa860158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8600b04b0 0x7fa860158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46582/0 (socket says 172.21.15.97:46582) 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 -- 172.21.15.97:0/247439434 learned_addr learned my addr 172.21.15.97:0/247439434 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 -- 172.21.15.97:0/247439434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8600b2870 msgr2=0x7fa86015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8600b2870 0x7fa86015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 -- 172.21.15.97:0/247439434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa86015c160 msgr2=0x7fa860156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa86015c160 0x7fa860156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 -- 172.21.15.97:0/247439434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540094a0 con 0x7fa8600b04b0 2023-11-24T06:09:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa864997700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8600b04b0 0x7fa860158290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa854009b90 tx=0x7fa854004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa85d7fa700 1 -- 172.21.15.97:0/247439434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854011630 con 0x7fa8600b04b0 2023-11-24T06:09:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa85d7fa700 1 -- 172.21.15.97:0/247439434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85401e070 con 0x7fa8600b04b0 2023-11-24T06:09:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa85d7fa700 1 -- 172.21.15.97:0/247439434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540119b0 con 0x7fa8600b04b0 2023-11-24T06:09:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8601f20a0 con 0x7fa8600b04b0 2023-11-24T06:09:43.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.631+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8601587d0 con 0x7fa8600b04b0 2023-11-24T06:09:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.632+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8600cf330 con 0x7fa8600b04b0 2023-11-24T06:09:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.632+0000 7fa85d7fa700 1 -- 172.21.15.97:0/247439434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa854011b10 con 0x7fa8600b04b0 2023-11-24T06:09:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.633+0000 7fa85d7fa700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa84406aed0 0x7fa84406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.633+0000 7fa85d7fa700 1 -- 172.21.15.97:0/247439434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa85408a850 con 0x7fa8600b04b0 2023-11-24T06:09:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.633+0000 7fa85ffff700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa84406aed0 0x7fa84406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:43.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.633+0000 7fa85ffff700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa84406aed0 0x7fa84406d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0011e0 tx=0x7fa84c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:43.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.637+0000 7fa85d7fa700 1 -- 172.21.15.97:0/247439434 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa854055910 con 0x7fa8600b04b0 2023-11-24T06:09:43.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.870+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8600b2ee0 con 0x7fa8600b04b0 2023-11-24T06:09:43.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.871+0000 7fa85d7fa700 1 -- 172.21.15.97:0/247439434 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7fa8600b2ee0 con 0x7fa8600b04b0 2023-11-24T06:09:43.873 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:43.874 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:08:57.902510+0000","modified":"2023-11-24T06:09:00.921422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.872+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa84406aed0 msgr2=0x7fa84406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa84406aed0 0x7fa84406d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa84c0011e0 tx=0x7fa84c0079b0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8600b04b0 msgr2=0x7fa860158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8600b04b0 0x7fa860158290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa854009b90 tx=0x7fa854004750 comp rx=0 tx=0).stop 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 shutdown_connections 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa86015c160 0x7fa860156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8600b04b0 0x7fa860158290 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa84406aed0 0x7fa84406d390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 --2- 172.21.15.97:0/247439434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa8600b2870 0x7fa86015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 >> 172.21.15.97:0/247439434 conn(0x7fa8600b57b0 msgr2=0x7fa8600b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 shutdown_connections 2023-11-24T06:09:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:43.873+0000 7fa866bfb700 1 -- 172.21.15.97:0/247439434 wait complete. 2023-11-24T06:09:43.877 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-11-24T06:09:43.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:44.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.004+0000 7f4398f2a700 1 Processor -- start 2023-11-24T06:09:44.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.004+0000 7f4398f2a700 1 -- start start 2023-11-24T06:09:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4398f2a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43940b3370 0x7f43941604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4398f2a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f43940d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4398f2a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43940b22d0 con 0x7f439415bdd0 2023-11-24T06:09:44.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4398f2a700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43940b2f50 con 0x7f43940b3370 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4398f2a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43940d2640 con 0x7f43940d1c70 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4391d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43940b3370 0x7f43941604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f439259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f439415bdd0 0x7f43940b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39042/0 (socket says 172.21.15.97:39042) 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f439259c700 1 -- 172.21.15.97:0/3031682684 learned_addr learned my addr 172.21.15.97:0/3031682684 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4392d9d700 1 --2- 172.21.15.97:0/3031682684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f43940d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4391d9b700 1 -- 172.21.15.97:0/3031682684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 msgr2=0x7f43940d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4391d9b700 1 --2- 172.21.15.97:0/3031682684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f43940d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4391d9b700 1 -- 172.21.15.97:0/3031682684 >> v1:172.21.15.97:6789/0 conn(0x7f439415bdd0 legacy=0x7f43940b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4391d9b700 1 -- 172.21.15.97:0/3031682684 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43940b25b0 con 0x7f43940b3370 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f439159a700 1 -- 172.21.15.97:0/3031682684 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745373325 0 0) 0x7f43940b22d0 con 0x7f439415bdd0 2023-11-24T06:09:44.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4392d9d700 1 --2- 172.21.15.97:0/3031682684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f43940d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f4391d9b700 1 --2- 172.21.15.97:0/3031682684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43940b3370 0x7f43941604e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4388009840 tx=0x7f4388009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921e2f662efd46e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f439159a700 1 -- 172.21.15.97:0/3031682684 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388011830 con 0x7f43940b3370 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f439159a700 1 -- 172.21.15.97:0/3031682684 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4388011990 con 0x7f43940b3370 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.005+0000 7f439159a700 1 -- 172.21.15.97:0/3031682684 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388011d20 con 0x7f43940b3370 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.006+0000 7f4398f2a700 1 -- 172.21.15.97:0/3031682684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43940b3370 msgr2=0x7f43941604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.006+0000 7f4398f2a700 1 --2- 172.21.15.97:0/3031682684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43940b3370 0x7f43941604e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4388009840 tx=0x7f4388009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.006+0000 7f4398f2a700 1 -- 172.21.15.97:0/3031682684 shutdown_connections 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.006+0000 7f4398f2a700 1 --2- 172.21.15.97:0/3031682684 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f43940d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.006+0000 7f4398f2a700 1 --2- 172.21.15.97:0/3031682684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43940b3370 0x7f43941604e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.006+0000 7f4398f2a700 1 -- 172.21.15.97:0/3031682684 >> 172.21.15.97:0/3031682684 conn(0x7f43940731a0 msgr2=0x7f439409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:44.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.006+0000 7f4398f2a700 1 -- 172.21.15.97:0/3031682684 shutdown_connections 2023-11-24T06:09:44.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.006+0000 7f4398f2a700 1 -- 172.21.15.97:0/3031682684 wait complete. 2023-11-24T06:09:44.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.007+0000 7f4398f2a700 1 Processor -- start 2023-11-24T06:09:44.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.007+0000 7f4398f2a700 1 -- start start 2023-11-24T06:09:44.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4398f2a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43940b3370 0x7f43941f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4398f2a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f4394159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4398f2a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439415bdd0 0x7f43941596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4398f2a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43941f4750 con 0x7f439415bdd0 2023-11-24T06:09:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4398f2a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940cca20 con 0x7f43940b3370 2023-11-24T06:09:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4398f2a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43940ccd00 con 0x7f43940d1c70 2023-11-24T06:09:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4392d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439415bdd0 0x7f43941596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4392d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439415bdd0 0x7f43941596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46600/0 (socket says 172.21.15.97:46600) 2023-11-24T06:09:44.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4392d9d700 1 -- 172.21.15.97:0/364340443 learned_addr learned my addr 172.21.15.97:0/364340443 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:44.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f439259c700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43940b3370 0x7f43941f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4391d9b700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f4394159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4392d9d700 1 -- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 msgr2=0x7f4394159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.008+0000 7f4392d9d700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f4394159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4392d9d700 1 -- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43940b3370 msgr2=0x7f43941f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4392d9d700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43940b3370 0x7f43941f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4392d9d700 1 -- 172.21.15.97:0/364340443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43880094a0 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4391d9b700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f4394159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4392d9d700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439415bdd0 0x7f43941596b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f438c00cef0 tx=0x7f438c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4382ffd700 1 -- 172.21.15.97:0/364340443 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c00adb0 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4382ffd700 1 -- 172.21.15.97:0/364340443 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c015440 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43940ccfe0 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4382ffd700 1 -- 172.21.15.97:0/364340443 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c0157f0 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.009+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43940cd5a0 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.010+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43940d01b0 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.010+0000 7f4382ffd700 1 -- 172.21.15.97:0/364340443 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f438c00d430 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.011+0000 7f4382ffd700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f437406aed0 0x7f437406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.011+0000 7f4382ffd700 1 -- 172.21.15.97:0/364340443 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f438c089bc0 con 0x7f439415bdd0 2023-11-24T06:09:44.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.011+0000 7f439259c700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f437406aed0 0x7f437406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.011+0000 7f439259c700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f437406aed0 0x7f437406d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4384000f60 tx=0x7f4384007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.015+0000 7f4382ffd700 1 -- 172.21.15.97:0/364340443 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f438c054be0 con 0x7f439415bdd0 2023-11-24T06:09:44.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.248+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4394067f60 con 0x7f439415bdd0 2023-11-24T06:09:44.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.248+0000 7f4382ffd700 1 -- 172.21.15.97:0/364340443 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4910 (secure 0 0 0) 0x7f438c01a020 con 0x7f439415bdd0 2023-11-24T06:09:44.251 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:44.251 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2474257915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2474257915},{"type":"v1","addr":"172.21.15.97:6836","nonce":2474257915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3614666313","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3614666313},{"type":"v1","addr":"172.21.15.183:6835","nonce":3614666313}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2082091700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2082091700},{"type":"v1","addr":"172.21.15.97:6837","nonce":2082091700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:08:57.902510+0000","modified":"2023-11-24T06:09:00.921422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6834/401436267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":401436267},{"type":"v1","addr":"172.21.15.183:6834","nonce":401436267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:09:44.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.250+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f437406aed0 msgr2=0x7f437406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.250+0000 7f4398f2a700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f437406aed0 0x7f437406d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4384000f60 tx=0x7f4384007e70 comp rx=0 tx=0).stop 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.250+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439415bdd0 msgr2=0x7f43941596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.250+0000 7f4398f2a700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439415bdd0 0x7f43941596b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f438c00cef0 tx=0x7f438c00cf20 comp rx=0 tx=0).stop 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.250+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 shutdown_connections 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.251+0000 7f4398f2a700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f439415bdd0 0x7f43941596b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.251+0000 7f4398f2a700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43940b3370 0x7f43941f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.251+0000 7f4398f2a700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f437406aed0 0x7f437406d390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.251+0000 7f4398f2a700 1 --2- 172.21.15.97:0/364340443 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f43940d1c70 0x7f4394159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.251+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 >> 172.21.15.97:0/364340443 conn(0x7f43940731a0 msgr2=0x7f4394159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.251+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 shutdown_connections 2023-11-24T06:09:44.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.251+0000 7f4398f2a700 1 -- 172.21.15.97:0/364340443 wait complete. 2023-11-24T06:09:44.255 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2023-11-24T06:09:44.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:44.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.384+0000 7fbc74c7c700 1 Processor -- start 2023-11-24T06:09:44.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.385+0000 7fbc74c7c700 1 -- start start 2023-11-24T06:09:44.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.385+0000 7fbc74c7c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc700b04b0 0x7fbc700622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.385+0000 7fbc74c7c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc700af1e0 0x7fbc700b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.385+0000 7fbc74c7c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7015c540 con 0x7fbc7015c160 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.385+0000 7fbc74c7c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc700b2ee0 con 0x7fbc700b04b0 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.385+0000 7fbc74c7c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700add40 con 0x7fbc700af1e0 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc700b04b0 0x7fbc700622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6e59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbc7015c160 0x7fbc700afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39056/0 (socket says 172.21.15.97:39056) 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc700b04b0 0x7fbc700622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46606/0 (socket says 172.21.15.97:46606) 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6e59c700 1 -- 172.21.15.97:0/4191648969 learned_addr learned my addr 172.21.15.97:0/4191648969 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6ed9d700 1 --2- 172.21.15.97:0/4191648969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc700af1e0 0x7fbc700b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6dd9b700 1 -- 172.21.15.97:0/4191648969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc700af1e0 msgr2=0x7fbc700b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6dd9b700 1 --2- 172.21.15.97:0/4191648969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc700af1e0 0x7fbc700b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6dd9b700 1 -- 172.21.15.97:0/4191648969 >> v1:172.21.15.97:6789/0 conn(0x7fbc7015c160 legacy=0x7fbc700afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6dd9b700 1 -- 172.21.15.97:0/4191648969 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc700ad410 con 0x7fbc700b04b0 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6d59a700 1 -- 172.21.15.97:0/4191648969 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434954195 0 0) 0x7fbc7015c540 con 0x7fbc7015c160 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6dd9b700 1 --2- 172.21.15.97:0/4191648969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700b04b0 0x7fbc700622a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009840 tx=0x7fbc60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd94ceb9f2f19a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6d59a700 1 -- 172.21.15.97:0/4191648969 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60011830 con 0x7fbc700b04b0 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6d59a700 1 -- 172.21.15.97:0/4191648969 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc60011990 con 0x7fbc700b04b0 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.386+0000 7fbc6d59a700 1 -- 172.21.15.97:0/4191648969 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60011d20 con 0x7fbc700b04b0 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.387+0000 7fbc74c7c700 1 -- 172.21.15.97:0/4191648969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700b04b0 msgr2=0x7fbc700622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.387+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/4191648969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700b04b0 0x7fbc700622a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbc60009840 tx=0x7fbc60009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.387+0000 7fbc74c7c700 1 -- 172.21.15.97:0/4191648969 shutdown_connections 2023-11-24T06:09:44.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.387+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/4191648969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc700af1e0 0x7fbc700b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.387+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/4191648969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700b04b0 0x7fbc700622a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.387+0000 7fbc74c7c700 1 -- 172.21.15.97:0/4191648969 >> 172.21.15.97:0/4191648969 conn(0x7fbc700b57b0 msgr2=0x7fbc700b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.387+0000 7fbc74c7c700 1 -- 172.21.15.97:0/4191648969 shutdown_connections 2023-11-24T06:09:44.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.387+0000 7fbc74c7c700 1 -- 172.21.15.97:0/4191648969 wait complete. 2023-11-24T06:09:44.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.388+0000 7fbc74c7c700 1 Processor -- start 2023-11-24T06:09:44.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.388+0000 7fbc74c7c700 1 -- start start 2023-11-24T06:09:44.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc74c7c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700af1e0 0x7fbc700cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc74c7c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc700b04b0 0x7fbc7015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc74c7c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc7015c160 0x7fbc70151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc74c7c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700ccba0 con 0x7fbc700af1e0 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc74c7c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700cce80 con 0x7fbc700b04b0 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc74c7c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700cd160 con 0x7fbc7015c160 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700af1e0 0x7fbc700cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700af1e0 0x7fbc700cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46610/0 (socket says 172.21.15.97:46610) 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 -- 172.21.15.97:0/3661204647 learned_addr learned my addr 172.21.15.97:0/3661204647 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6dd9b700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc700b04b0 0x7fbc7015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6ed9d700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc7015c160 0x7fbc70151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 -- 172.21.15.97:0/3661204647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc7015c160 msgr2=0x7fbc70151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc7015c160 0x7fbc70151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 -- 172.21.15.97:0/3661204647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc700b04b0 msgr2=0x7fbc7015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc700b04b0 0x7fbc7015dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 -- 172.21.15.97:0/3661204647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600094a0 con 0x7fbc700af1e0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc6e59c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700af1e0 0x7fbc700cc370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbc640011e0 tx=0x7fbc6400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.389+0000 7fbc5effd700 1 -- 172.21.15.97:0/3661204647 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64004c80 con 0x7fbc700af1e0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.390+0000 7fbc5effd700 1 -- 172.21.15.97:0/3661204647 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc64004e00 con 0x7fbc700af1e0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.390+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7015d590 con 0x7fbc700af1e0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.390+0000 7fbc5effd700 1 -- 172.21.15.97:0/3661204647 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64007310 con 0x7fbc700af1e0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.390+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc700cc8b0 con 0x7fbc700af1e0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.391+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc70067f60 con 0x7fbc700af1e0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.391+0000 7fbc5effd700 1 -- 172.21.15.97:0/3661204647 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc64007470 con 0x7fbc700af1e0 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.391+0000 7fbc5effd700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbc5006aed0 0x7fbc5006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.391+0000 7fbc5effd700 1 -- 172.21.15.97:0/3661204647 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbc64089430 con 0x7fbc700af1e0 2023-11-24T06:09:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.391+0000 7fbc6dd9b700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbc5006aed0 0x7fbc5006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.392+0000 7fbc6dd9b700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbc5006aed0 0x7fbc5006d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbc600090b0 tx=0x7fbc6001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.396+0000 7fbc5effd700 1 -- 172.21.15.97:0/3661204647 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc64054460 con 0x7fbc700af1e0 2023-11-24T06:09:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.587+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc70073190 con 0x7fbc700af1e0 2023-11-24T06:09:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.588+0000 7fbc5effd700 1 -- 172.21.15.97:0/3661204647 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16766 (secure 0 0 0) 0x7fbc64057a80 con 0x7fbc700af1e0 2023-11-24T06:09:44.591 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:44.591 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":22,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:39.752606+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:08:55.666878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:08:56.875018+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.13","mappings":[{"from":7,"to":1}]},{"pgid":"3.38","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:44.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbc5006aed0 msgr2=0x7fbc5006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbc5006aed0 0x7fbc5006d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbc600090b0 tx=0x7fbc6001c040 comp rx=0 tx=0).stop 2023-11-24T06:09:44.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700af1e0 msgr2=0x7fbc700cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700af1e0 0x7fbc700cc370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbc640011e0 tx=0x7fbc6400cae0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 shutdown_connections 2023-11-24T06:09:44.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbc5006aed0 0x7fbc5006d390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc7015c160 0x7fbc70151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700af1e0 0x7fbc700cc370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 --2- 172.21.15.97:0/3661204647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc700b04b0 0x7fbc7015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 >> 172.21.15.97:0/3661204647 conn(0x7fbc700b57b0 msgr2=0x7fbc700cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 shutdown_connections 2023-11-24T06:09:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.590+0000 7fbc74c7c700 1 -- 172.21.15.97:0/3661204647 wait complete. 2023-11-24T06:09:44.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:09:44.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.723+0000 7febc0117700 1 Processor -- start 2023-11-24T06:09:44.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.723+0000 7febc0117700 1 -- start start 2023-11-24T06:09:44.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febc0117700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7febb80d2380 0x7febb81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febc0117700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb80b22d0 con 0x7febb80b3370 2023-11-24T06:09:44.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febc0117700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb80b25b0 con 0x7febb815bdd0 2023-11-24T06:09:44.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febc0117700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb80b2f50 con 0x7febb80d2380 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbe6b4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7febb80d2380 0x7febb81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbd6b2700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7febb80b3370 0x7febb80d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39072/0 (socket says 172.21.15.97:39072) 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbd6b2700 1 -- 172.21.15.97:0/2508503773 learned_addr learned my addr 172.21.15.97:0/2508503773 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbe6b4700 1 -- 172.21.15.97:0/2508503773 >> v1:172.21.15.183:6789/0 conn(0x7febb815bdd0 legacy=0x7febb80b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbe6b4700 1 -- 172.21.15.97:0/2508503773 >> v1:172.21.15.97:6789/0 conn(0x7febb80b3370 legacy=0x7febb80d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbe6b4700 1 -- 172.21.15.97:0/2508503773 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb80b2890 con 0x7febb80d2380 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbceb1700 1 -- 172.21.15.97:0/2508503773 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461264644 0 0) 0x7febb80b22d0 con 0x7febb80b3370 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbe6b4700 1 --2- 172.21.15.97:0/2508503773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb81604e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7febb0009d90 tx=0x7febb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f0ca17f10901a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbceb1700 1 -- 172.21.15.97:0/2508503773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb000e070 con 0x7febb80d2380 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbceb1700 1 -- 172.21.15.97:0/2508503773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb000bb90 con 0x7febb80d2380 2023-11-24T06:09:44.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.724+0000 7febbceb1700 1 -- 172.21.15.97:0/2508503773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0003a20 con 0x7febb80d2380 2023-11-24T06:09:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.725+0000 7febc0117700 1 -- 172.21.15.97:0/2508503773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 msgr2=0x7febb81604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.725+0000 7febc0117700 1 --2- 172.21.15.97:0/2508503773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb81604e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7febb0009d90 tx=0x7febb000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.725+0000 7febc0117700 1 -- 172.21.15.97:0/2508503773 shutdown_connections 2023-11-24T06:09:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.725+0000 7febc0117700 1 --2- 172.21.15.97:0/2508503773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb81604e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.725+0000 7febc0117700 1 -- 172.21.15.97:0/2508503773 >> 172.21.15.97:0/2508503773 conn(0x7febb80731a0 msgr2=0x7febb809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.725+0000 7febc0117700 1 -- 172.21.15.97:0/2508503773 shutdown_connections 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.725+0000 7febc0117700 1 -- 172.21.15.97:0/2508503773 wait complete. 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.726+0000 7febc0117700 1 Processor -- start 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.726+0000 7febc0117700 1 -- start start 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febc0117700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febb80b3370 0x7febb80c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febc0117700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb80c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febc0117700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7febb815bdd0 0x7febb80c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febc0117700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80ce0d0 con 0x7febb80d2380 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febc0117700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb80c8740 con 0x7febb80b3370 2023-11-24T06:09:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febc0117700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb80c6b70 con 0x7febb815bdd0 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb80c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb80c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46626/0 (socket says 172.21.15.97:46626) 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 -- 172.21.15.97:0/2392105341 learned_addr learned my addr 172.21.15.97:0/2392105341 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 -- 172.21.15.97:0/2392105341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7febb815bdd0 msgr2=0x7febb80c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7febb815bdd0 0x7febb80c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 -- 172.21.15.97:0/2392105341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febb80b3370 msgr2=0x7febb80c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbdeb3700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febb80b3370 0x7febb80c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febb80b3370 0x7febb80c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 -- 172.21.15.97:0/2392105341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb0009a70 con 0x7febb80d2380 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbdeb3700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febb80b3370 0x7febb80c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febbd6b2700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb80c8200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feba80011e0 tx=0x7feba800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febae7fc700 1 -- 172.21.15.97:0/2392105341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8004c80 con 0x7febb80d2380 2023-11-24T06:09:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febae7fc700 1 -- 172.21.15.97:0/2392105341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba8004e00 con 0x7febb80d2380 2023-11-24T06:09:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.727+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb80c5ef0 con 0x7febb80d2380 2023-11-24T06:09:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.728+0000 7febae7fc700 1 -- 172.21.15.97:0/2392105341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8007310 con 0x7febb80d2380 2023-11-24T06:09:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.728+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb80cda10 con 0x7febb80d2380 2023-11-24T06:09:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.728+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb8166ca0 con 0x7febb80d2380 2023-11-24T06:09:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.729+0000 7febae7fc700 1 -- 172.21.15.97:0/2392105341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feba8007470 con 0x7febb80d2380 2023-11-24T06:09:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.729+0000 7febae7fc700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feb9c06aed0 0x7feb9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.729+0000 7febbdeb3700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feb9c06aed0 0x7feb9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.729+0000 7febae7fc700 1 -- 172.21.15.97:0/2392105341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7feba8089370 con 0x7febb80d2380 2023-11-24T06:09:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.730+0000 7febbdeb3700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feb9c06aed0 0x7feb9c06d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7febb4006fd0 tx=0x7febb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.734+0000 7febae7fc700 1 -- 172.21.15.97:0/2392105341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feba8054430 con 0x7febb80d2380 2023-11-24T06:09:44.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.969+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7febb80ce210 con 0x7febb80d2380 2023-11-24T06:09:44.972 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-11-24T06:09:44.970+0000 7fbb14a83700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:09:44.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.971+0000 7febae7fc700 1 -- 172.21.15.97:0/2392105341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7febb80ce210 con 0x7febb80d2380 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.973+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feb9c06aed0 msgr2=0x7feb9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.973+0000 7febc0117700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feb9c06aed0 0x7feb9c06d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7febb4006fd0 tx=0x7febb4008040 comp rx=0 tx=0).stop 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.973+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 msgr2=0x7febb80c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.973+0000 7febc0117700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb80c8200 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7feba80011e0 tx=0x7feba800cae0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.974+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 shutdown_connections 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.974+0000 7febc0117700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feb9c06aed0 0x7feb9c06d390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.974+0000 7febc0117700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7febb815bdd0 0x7febb80c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.974+0000 7febc0117700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febb80b3370 0x7febb80c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.974+0000 7febc0117700 1 --2- 172.21.15.97:0/2392105341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb80d2380 0x7febb80c8200 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:44.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.974+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 >> 172.21.15.97:0/2392105341 conn(0x7febb80731a0 msgr2=0x7febb8153ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:44.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.974+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 shutdown_connections 2023-11-24T06:09:44.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:44.974+0000 7febc0117700 1 -- 172.21.15.97:0/2392105341 wait complete. 2023-11-24T06:09:44.978 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:09:44.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:09:45.019 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-11-24T06:09:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.109+0000 7f934b1cd700 1 Processor -- start 2023-11-24T06:09:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.109+0000 7f934b1cd700 1 -- start start 2023-11-24T06:09:45.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f934b1cd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f934415e160 0x7f93440b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:45.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f934b1cd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93440746a0 0x7f934406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f934b1cd700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440b3c60 con 0x7f93440afc90 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f934b1cd700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440b4500 con 0x7f934415e160 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f934b1cd700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440b58c0 con 0x7f93440746a0 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f9348f69700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f934415e160 0x7f93440b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f9343fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f93440afc90 0x7f9344073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39084/0 (socket says 172.21.15.97:39084) 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f9343fff700 1 -- 172.21.15.97:0/3870812587 learned_addr learned my addr 172.21.15.97:0/3870812587 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f9348f69700 1 -- 172.21.15.97:0/3870812587 >> v2:172.21.15.183:3300/0 conn(0x7f93440746a0 msgr2=0x7f934406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f9348f69700 1 --2- 172.21.15.97:0/3870812587 >> v2:172.21.15.183:3300/0 conn(0x7f93440746a0 0x7f934406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f9348f69700 1 -- 172.21.15.97:0/3870812587 >> v1:172.21.15.97:6789/0 conn(0x7f93440afc90 legacy=0x7f9344073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f9348f69700 1 -- 172.21.15.97:0/3870812587 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440b0140 con 0x7f934415e160 2023-11-24T06:09:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f93437fe700 1 -- 172.21.15.97:0/3870812587 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371536783 0 0) 0x7f93440b3c60 con 0x7f93440afc90 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f9348f69700 1 --2- 172.21.15.97:0/3870812587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f93440b5da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93440d16e0 tx=0x7f9338009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd16757c66b4e45 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f93437fe700 1 -- 172.21.15.97:0/3870812587 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338011830 con 0x7f934415e160 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.110+0000 7f93437fe700 1 -- 172.21.15.97:0/3870812587 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338011990 con 0x7f934415e160 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f93437fe700 1 -- 172.21.15.97:0/3870812587 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338011d20 con 0x7f934415e160 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f934b1cd700 1 -- 172.21.15.97:0/3870812587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 msgr2=0x7f93440b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3870812587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f93440b5da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93440d16e0 tx=0x7f9338009b20 comp rx=0 tx=0).stop 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f934b1cd700 1 -- 172.21.15.97:0/3870812587 shutdown_connections 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3870812587 >> v2:172.21.15.183:3300/0 conn(0x7f93440746a0 0x7f934406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3870812587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f93440b5da0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f934b1cd700 1 -- 172.21.15.97:0/3870812587 >> 172.21.15.97:0/3870812587 conn(0x7f93440d0bf0 msgr2=0x7f93440d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f934b1cd700 1 -- 172.21.15.97:0/3870812587 shutdown_connections 2023-11-24T06:09:45.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.111+0000 7f934b1cd700 1 -- 172.21.15.97:0/3870812587 wait complete. 2023-11-24T06:09:45.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.112+0000 7f934b1cd700 1 Processor -- start 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.112+0000 7f934b1cd700 1 -- start start 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934b1cd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93440746a0 0x7f934414f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934b1cd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93440afc90 0x7f9344160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934b1cd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f934406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934b1cd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934414fc30 con 0x7f934415e160 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934b1cd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934415f840 con 0x7f93440afc90 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934b1cd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93441614f0 con 0x7f93440746a0 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934976a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f934406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934976a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f934406e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46642/0 (socket says 172.21.15.97:46642) 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934976a700 1 -- 172.21.15.97:0/3056069279 learned_addr learned my addr 172.21.15.97:0/3056069279 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:45.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f9348f69700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93440746a0 0x7f934414f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f9343fff700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93440afc90 0x7f9344160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934976a700 1 -- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93440746a0 msgr2=0x7f934414f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.113+0000 7f934976a700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93440746a0 0x7f934414f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f934976a700 1 -- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93440afc90 msgr2=0x7f9344160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f934976a700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93440afc90 0x7f9344160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f934976a700 1 -- 172.21.15.97:0/3056069279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380094a0 con 0x7f934415e160 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f9343fff700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93440afc90 0x7f9344160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f934976a700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f934406e870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f933c00b9e0 tx=0x7f933c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f93417fa700 1 -- 172.21.15.97:0/3056069279 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00c9b0 con 0x7f934415e160 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f93417fa700 1 -- 172.21.15.97:0/3056069279 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c00cb10 con 0x7f934415e160 2023-11-24T06:09:45.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934415fae0 con 0x7f934415e160 2023-11-24T06:09:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f93417fa700 1 -- 172.21.15.97:0/3056069279 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c014440 con 0x7f934415e160 2023-11-24T06:09:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.114+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93441600a0 con 0x7f934415e160 2023-11-24T06:09:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.115+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9344068fb0 con 0x7f934415e160 2023-11-24T06:09:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.115+0000 7f93417fa700 1 -- 172.21.15.97:0/3056069279 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f933c01d3b0 con 0x7f934415e160 2023-11-24T06:09:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.116+0000 7f93417fa700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f932806aed0 0x7f932806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.116+0000 7f9348f69700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f932806aed0 0x7f932806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.116+0000 7f93417fa700 1 -- 172.21.15.97:0/3056069279 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f933c006e40 con 0x7f934415e160 2023-11-24T06:09:45.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.116+0000 7f9348f69700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f932806aed0 0x7f932806d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f93380090b0 tx=0x7f933801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:45.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.120+0000 7f93417fa700 1 -- 172.21.15.97:0/3056069279 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f933c054350 con 0x7f934415e160 2023-11-24T06:09:45.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.353+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f934406c140 con 0x7f934415e160 2023-11-24T06:09:45.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.972+0000 7f93417fa700 1 -- 172.21.15.97:0/3056069279 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f933c057970 con 0x7f934415e160 2023-11-24T06:09:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f932806aed0 msgr2=0x7f932806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f932806aed0 0x7f932806d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f93380090b0 tx=0x7f933801c040 comp rx=0 tx=0).stop 2023-11-24T06:09:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 msgr2=0x7f934406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f934406e870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f933c00b9e0 tx=0x7f933c00bda0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 shutdown_connections 2023-11-24T06:09:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f934415e160 0x7f934406e870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f932806aed0 0x7f932806d390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93440746a0 0x7f934414f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 --2- 172.21.15.97:0/3056069279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93440afc90 0x7f9344160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 >> 172.21.15.97:0/3056069279 conn(0x7f93440d0bf0 msgr2=0x7f9344162690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 shutdown_connections 2023-11-24T06:09:45.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:45.975+0000 7f934b1cd700 1 -- 172.21.15.97:0/3056069279 wait complete. 2023-11-24T06:09:45.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:09:46.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.114+0000 7f0699b9c700 1 Processor -- start 2023-11-24T06:09:46.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.115+0000 7f0699b9c700 1 -- start start 2023-11-24T06:09:46.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.115+0000 7f0699b9c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f06940ce5a0 0x7f06940ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:46.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.115+0000 7f0699b9c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06940ad720 0x7f06940afb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.115+0000 7f0699b9c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06940aeff0 con 0x7f069415a510 2023-11-24T06:09:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.115+0000 7f0699b9c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06940aebd0 con 0x7f06940ce5a0 2023-11-24T06:09:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.115+0000 7f0699b9c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06940adf50 con 0x7f06940ad720 2023-11-24T06:09:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0692ffd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f06940ce5a0 0x7f06940ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0693fff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06940ad720 0x7f06940afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0693fff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06940ad720 0x7f06940afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:33686/0 (socket says 172.21.15.97:33686) 2023-11-24T06:09:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f06937fe700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f069415a510 0x7f06940cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:48420/0 (socket says 172.21.15.97:48420) 2023-11-24T06:09:46.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0693fff700 1 -- 172.21.15.97:0/1776403173 learned_addr learned my addr 172.21.15.97:0/1776403173 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0693fff700 1 -- 172.21.15.97:0/1776403173 >> v2:172.21.15.183:3300/0 conn(0x7f06940ce5a0 msgr2=0x7f06940ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0693fff700 1 --2- 172.21.15.97:0/1776403173 >> v2:172.21.15.183:3300/0 conn(0x7f06940ce5a0 0x7f06940ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0693fff700 1 -- 172.21.15.97:0/1776403173 >> v1:172.21.15.183:6789/0 conn(0x7f069415a510 legacy=0x7f06940cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0693fff700 1 -- 172.21.15.97:0/1776403173 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06940ae230 con 0x7f06940ad720 2023-11-24T06:09:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0692ffd700 1 --2- 172.21.15.97:0/1776403173 >> v2:172.21.15.183:3300/0 conn(0x7f06940ce5a0 0x7f06940ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f0693fff700 1 --2- 172.21.15.97:0/1776403173 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06940ad720 0x7f06940afb80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f068c009e50 tx=0x7f068c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96550ddafae69a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f06927fc700 1 -- 172.21.15.97:0/1776403173 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00e070 con 0x7f06940ad720 2023-11-24T06:09:46.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f06927fc700 1 -- 172.21.15.97:0/1776403173 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c00b910 con 0x7f06940ad720 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.116+0000 7f06927fc700 1 -- 172.21.15.97:0/1776403173 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00bcc0 con 0x7f06940ad720 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.117+0000 7f0699b9c700 1 -- 172.21.15.97:0/1776403173 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06940ad720 msgr2=0x7f06940afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.117+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1776403173 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06940ad720 0x7f06940afb80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f068c009e50 tx=0x7f068c00b330 comp rx=0 tx=0).stop 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.117+0000 7f0699b9c700 1 -- 172.21.15.97:0/1776403173 shutdown_connections 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.117+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1776403173 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06940ad720 0x7f06940afb80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.117+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1776403173 >> v2:172.21.15.183:3300/0 conn(0x7f06940ce5a0 0x7f06940ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.117+0000 7f0699b9c700 1 -- 172.21.15.97:0/1776403173 >> 172.21.15.97:0/1776403173 conn(0x7f0694072710 msgr2=0x7f0694072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.117+0000 7f0699b9c700 1 -- 172.21.15.97:0/1776403173 shutdown_connections 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.117+0000 7f0699b9c700 1 -- 172.21.15.97:0/1776403173 wait complete. 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.118+0000 7f0699b9c700 1 Processor -- start 2023-11-24T06:09:46.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.118+0000 7f0699b9c700 1 -- start start 2023-11-24T06:09:46.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0699b9c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06940ad720 0x7f0694156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:46.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0699b9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06940ce5a0 0x7f069415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:46.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0699b9c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f069415a510 0x7f069415c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:46.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0699b9c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06941f4260 con 0x7f06940ce5a0 2023-11-24T06:09:46.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0699b9c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0694158b60 con 0x7f06940ad720 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0699b9c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0694158e40 con 0x7f069415a510 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0692ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06940ce5a0 0x7f069415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0692ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06940ce5a0 0x7f069415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46654/0 (socket says 172.21.15.97:46654) 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0692ffd700 1 -- 172.21.15.97:0/1323125637 learned_addr learned my addr 172.21.15.97:0/1323125637 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0693fff700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f069415a510 0x7f069415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f06937fe700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06940ad720 0x7f0694156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0692ffd700 1 -- 172.21.15.97:0/1323125637 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f069415a510 msgr2=0x7f069415c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.119+0000 7f0692ffd700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f069415a510 0x7f069415c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f0692ffd700 1 -- 172.21.15.97:0/1323125637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06940ad720 msgr2=0x7f0694156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f0692ffd700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06940ad720 0x7f0694156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f0692ffd700 1 -- 172.21.15.97:0/1323125637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840094a0 con 0x7f06940ce5a0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f0692ffd700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06940ce5a0 0x7f069415aca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f068400ebe0 tx=0x7f068400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f067bfff700 1 -- 172.21.15.97:0/1323125637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068400c850 con 0x7f06940ce5a0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f067bfff700 1 -- 172.21.15.97:0/1323125637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068400c9b0 con 0x7f06940ce5a0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c009b30 con 0x7f06940ce5a0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f067bfff700 1 -- 172.21.15.97:0/1323125637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068400cd40 con 0x7f06940ce5a0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.120+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0694154d20 con 0x7f06940ce5a0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.121+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06940cf8e0 con 0x7f06940ce5a0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.121+0000 7f067bfff700 1 -- 172.21.15.97:0/1323125637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0684013470 con 0x7f06940ce5a0 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.122+0000 7f067bfff700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f068006aed0 0x7f068006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:46.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.122+0000 7f067bfff700 1 -- 172.21.15.97:0/1323125637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f0684089850 con 0x7f06940ce5a0 2023-11-24T06:09:46.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.122+0000 7f06937fe700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f068006aed0 0x7f068006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:46.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.122+0000 7f06937fe700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f068006aed0 0x7f068006d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0688000f60 tx=0x7f0688007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:46.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.126+0000 7f067bfff700 1 -- 172.21.15.97:0/1323125637 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0684054700 con 0x7f06940ce5a0 2023-11-24T06:09:46.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.315+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06940744d0 con 0x7f06940ce5a0 2023-11-24T06:09:46.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.976+0000 7f067bfff700 1 -- 172.21.15.97:0/1323125637 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f06840170d0 con 0x7f06940ce5a0 2023-11-24T06:09:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f068006aed0 msgr2=0x7f068006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f068006aed0 0x7f068006d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0688000f60 tx=0x7f0688007ed0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06940ce5a0 msgr2=0x7f069415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06940ce5a0 0x7f069415aca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f068400ebe0 tx=0x7f068400eef0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 shutdown_connections 2023-11-24T06:09:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f068006aed0 0x7f068006d390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f069415a510 0x7f069415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06940ad720 0x7f0694156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 --2- 172.21.15.97:0/1323125637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06940ce5a0 0x7f069415aca0 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f068400ebe0 tx=0x7f068400eef0 comp rx=0 tx=0).stop 2023-11-24T06:09:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.979+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 >> 172.21.15.97:0/1323125637 conn(0x7f0694072710 msgr2=0x7f06940c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.980+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 shutdown_connections 2023-11-24T06:09:46.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:46.980+0000 7f0699b9c700 1 -- 172.21.15.97:0/1323125637 wait complete. 2023-11-24T06:09:46.984 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:09:46.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:09:47.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.121+0000 7f380e1a0700 1 Processor -- start 2023-11-24T06:09:47.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.121+0000 7f380e1a0700 1 -- start start 2023-11-24T06:09:47.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f380e1a0700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f380806c710 0x7f38081658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:47.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f380e1a0700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38080b3a30 con 0x7f38080cecd0 2023-11-24T06:09:47.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f380e1a0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38080b3d10 con 0x7f3808144550 2023-11-24T06:09:47.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f380e1a0700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38080b3ff0 con 0x7f380806c710 2023-11-24T06:09:47.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f380c99d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f38080cecd0 0x7f380806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39114/0 (socket says 172.21.15.97:39114) 2023-11-24T06:09:47.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f380c99d700 1 -- 172.21.15.97:0/847502994 learned_addr learned my addr 172.21.15.97:0/847502994 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536665520 0 0) 0x7f38080b3a30 con 0x7f38080cecd0 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f40035b0 con 0x7f38080cecd0 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f380d99f700 1 --2- 172.21.15.97:0/847502994 >> v2:172.21.15.183:3300/0 conn(0x7f380806c710 0x7f38081658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.122+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1312875682 0 0) 0x7f37f40035b0 con 0x7f38080cecd0 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 >> v2:172.21.15.183:3300/0 conn(0x7f380806c710 msgr2=0x7f38081658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 --2- 172.21.15.97:0/847502994 >> v2:172.21.15.183:3300/0 conn(0x7f380806c710 0x7f38081658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 >> v1:172.21.15.183:6789/0 conn(0x7f3808144550 legacy=0x7f380806f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080b42d0 con 0x7f38080cecd0 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f3804001670 con 0x7f38080cecd0 2023-11-24T06:09:47.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 >> v1:172.21.15.97:6789/0 conn(0x7f38080cecd0 legacy=0x7f380806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4003a00 0x7f37f4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f4005410 0x7f37f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080b3ff0 con 0x7f37f4004850 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400c2a0 con 0x7f37f4005410 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400c460 con 0x7f37f4003a00 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380c99d700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380d19e700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4003a00 0x7f37f4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380d99f700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f4005410 0x7f37f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:47.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380c99d700 1 -- 172.21.15.97:0/847502994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4003a00 msgr2=0x7f37f4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380c99d700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4003a00 0x7f37f4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380c99d700 1 -- 172.21.15.97:0/847502994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f4005410 msgr2=0x7f37f400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380c99d700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37f4005410 0x7f37f400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380d99f700 1 -- 172.21.15.97:0/847502994 reap_dead start 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380c99d700 1 -- 172.21.15.97:0/847502994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40035b0 con 0x7f37f4004850 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f380c99d700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f380400c240 tx=0x7f37f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfcd64b78bef2af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804007c10 con 0x7f37f4004850 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380400cde0 con 0x7f37f4004850 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.123+0000 7f37fffff700 1 -- 172.21.15.97:0/847502994 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804005ba0 con 0x7f37f4004850 2023-11-24T06:09:47.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.124+0000 7f380e1a0700 1 -- 172.21.15.97:0/847502994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f4004850 msgr2=0x7f37f4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.124+0000 7f380e1a0700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f380400c240 tx=0x7f37f4005880 comp rx=0 tx=0).stop 2023-11-24T06:09:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.124+0000 7f380e1a0700 1 -- 172.21.15.97:0/847502994 shutdown_connections 2023-11-24T06:09:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.124+0000 7f380e1a0700 1 --2- 172.21.15.97:0/847502994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37f4004850 0x7f37f4004cd0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.124+0000 7f380e1a0700 1 -- 172.21.15.97:0/847502994 >> 172.21.15.97:0/847502994 conn(0x7f3808071af0 msgr2=0x7f3808071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.124+0000 7f380e1a0700 1 -- 172.21.15.97:0/847502994 shutdown_connections 2023-11-24T06:09:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.124+0000 7f380e1a0700 1 -- 172.21.15.97:0/847502994 wait complete. 2023-11-24T06:09:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.125+0000 7f380e1a0700 1 Processor -- start 2023-11-24T06:09:47.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.125+0000 7f380e1a0700 1 -- start start 2023-11-24T06:09:47.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380e1a0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4004850 0x7f380813a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380e1a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3808138830 0x7f3808153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380e1a0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380815e070 0x7f380815e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380e1a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380815ea30 con 0x7f3808138830 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380e1a0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808138cb0 con 0x7f380815e070 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380e1a0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38081540f0 con 0x7f37f4004850 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3808138830 0x7f3808153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3808138830 0x7f3808153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46674/0 (socket says 172.21.15.97:46674) 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 -- 172.21.15.97:0/2714743179 learned_addr learned my addr 172.21.15.97:0/2714743179 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380d19e700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4004850 0x7f380813a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380d99f700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380815e070 0x7f380815e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 -- 172.21.15.97:0/2714743179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4004850 msgr2=0x7f380813a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4004850 0x7f380813a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 -- 172.21.15.97:0/2714743179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380815e070 msgr2=0x7f380815e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380815e070 0x7f380815e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 -- 172.21.15.97:0/2714743179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38081543d0 con 0x7f3808138830 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.126+0000 7f380c99d700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3808138830 0x7f3808153810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f37f400c270 tx=0x7f3804001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.127+0000 7f37fdffb700 1 -- 172.21.15.97:0/2714743179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804004c80 con 0x7f3808138830 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.127+0000 7f37fdffb700 1 -- 172.21.15.97:0/2714743179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3804004de0 con 0x7f3808138830 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.127+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38081546b0 con 0x7f3808138830 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.127+0000 7f37fdffb700 1 -- 172.21.15.97:0/2714743179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804007310 con 0x7f3808138830 2023-11-24T06:09:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.127+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3808154c70 con 0x7f3808138830 2023-11-24T06:09:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.128+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3808154fe0 con 0x7f3808138830 2023-11-24T06:09:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.128+0000 7f37fdffb700 1 -- 172.21.15.97:0/2714743179 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3804007470 con 0x7f3808138830 2023-11-24T06:09:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.128+0000 7f37fdffb700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37ec06aed0 0x7f37ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.128+0000 7f37fdffb700 1 -- 172.21.15.97:0/2714743179 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f3804089d00 con 0x7f3808138830 2023-11-24T06:09:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.129+0000 7f380d19e700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37ec06aed0 0x7f37ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.129+0000 7f380d19e700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37ec06aed0 0x7f37ec06d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37f8007fd0 tx=0x7f37f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.133+0000 7f37fdffb700 1 -- 172.21.15.97:0/2714743179 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3804054ea0 con 0x7f3808138830 2023-11-24T06:09:47.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.322+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f380815a670 con 0x7f3808138830 2023-11-24T06:09:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.978+0000 7f37fdffb700 1 -- 172.21.15.97:0/2714743179 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f38040584c0 con 0x7f3808138830 2023-11-24T06:09:47.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.981+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37ec06aed0 msgr2=0x7f37ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:47.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.981+0000 7f380e1a0700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37ec06aed0 0x7f37ec06d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f37f8007fd0 tx=0x7f37f8009040 comp rx=0 tx=0).stop 2023-11-24T06:09:47.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.981+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3808138830 msgr2=0x7f3808153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.981+0000 7f380e1a0700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3808138830 0x7f3808153810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f37f400c270 tx=0x7f3804001a00 comp rx=0 tx=0).stop 2023-11-24T06:09:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.982+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 shutdown_connections 2023-11-24T06:09:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.982+0000 7f380e1a0700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f380815e070 0x7f380815e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.982+0000 7f380e1a0700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f37ec06aed0 0x7f37ec06d390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.982+0000 7f380e1a0700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37f4004850 0x7f380813a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.982+0000 7f380e1a0700 1 --2- 172.21.15.97:0/2714743179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3808138830 0x7f3808153810 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.982+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 >> 172.21.15.97:0/2714743179 conn(0x7f3808071af0 msgr2=0x7f3808159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:47.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.983+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 shutdown_connections 2023-11-24T06:09:47.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:47.983+0000 7f380e1a0700 1 -- 172.21.15.97:0/2714743179 wait complete. 2023-11-24T06:09:47.988 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-11-24T06:09:48.003 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:09:48.004 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:09:48.004 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:09:48.004 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:09:48.004 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:09:48.004 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:09:48.004 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:09:48.005 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:09:48.005 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:09:48.005 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:09:48.005 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:09:48.005 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:09:48.005 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:09:48.005 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:09:48.006 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:09:48.006 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:09:48.015 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:09:48.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:09:48.016 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:09:48.016 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:09:48.017 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:09:48.018 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:09:48.018 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:09:48.018 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:09:48.058 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:09:48.059 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:48.060 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:09:48.060 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:48.061 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:09:48.061 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:48.061 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:09:48.061 DEBUG:teuthology.parallel:result is None 2023-11-24T06:09:48.062 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:09:48.147 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-11-24T06:09:48.148 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-11-24T06:09:48.148 INFO:tasks.ceph.mds.a.smithi097.stderr:2023-11-24T06:09:48.145+0000 7f7c7168d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-24T06:09:48.151 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-11-24T06:09:48.152 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-11-24T06:09:48.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.197+0000 7f0ce69b7700 1 Processor -- start 2023-11-24T06:09:48.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.197+0000 7f0ce69b7700 1 -- start start 2023-11-24T06:09:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.197+0000 7f0ce69b7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0ce00b0fa0 0x7f0ce00afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.197+0000 7f0ce69b7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0ce00ca190 0x7f0ce00ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.197+0000 7f0ce69b7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce00ca620 con 0x7f0ce00d0c40 2023-11-24T06:09:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.197+0000 7f0ce69b7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce00d10a0 con 0x7f0ce00ca190 2023-11-24T06:09:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.197+0000 7f0ce69b7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce00622a0 con 0x7f0ce00b0fa0 2023-11-24T06:09:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0ce4f54700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0ce00ca190 0x7f0ce00ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0ce4f54700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0ce00ca190 0x7f0ce00ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46704/0 (socket says 172.21.15.97:46704) 2023-11-24T06:09:48.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0ce4f54700 1 -- 172.21.15.97:0/271814224 learned_addr learned my addr 172.21.15.97:0/271814224 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0cdffff700 1 --2- 172.21.15.97:0/271814224 >> v2:172.21.15.183:3300/0 conn(0x7f0ce00b0fa0 0x7f0ce00afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0ce4f54700 1 -- 172.21.15.97:0/271814224 >> v2:172.21.15.183:3300/0 conn(0x7f0ce00b0fa0 msgr2=0x7f0ce00afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0ce4f54700 1 --2- 172.21.15.97:0/271814224 >> v2:172.21.15.183:3300/0 conn(0x7f0ce00b0fa0 0x7f0ce00afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0ce4f54700 1 -- 172.21.15.97:0/271814224 >> v1:172.21.15.183:6789/0 conn(0x7f0ce00d0c40 legacy=0x7f0ce00b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0ce4f54700 1 -- 172.21.15.97:0/271814224 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0062580 con 0x7f0ce00ca190 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0ce4f54700 1 --2- 172.21.15.97:0/271814224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce00ce1a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ce006fc90 tx=0x7f0cd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd757fcc24600a44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0cdeffd700 1 -- 172.21.15.97:0/271814224 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800e070 con 0x7f0ce00ca190 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0cdeffd700 1 -- 172.21.15.97:0/271814224 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd800b9c0 con 0x7f0ce00ca190 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.198+0000 7f0cdeffd700 1 -- 172.21.15.97:0/271814224 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800bd70 con 0x7f0ce00ca190 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.199+0000 7f0ce69b7700 1 -- 172.21.15.97:0/271814224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 msgr2=0x7f0ce00ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.199+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/271814224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce00ce1a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ce006fc90 tx=0x7f0cd800b330 comp rx=0 tx=0).stop 2023-11-24T06:09:48.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.199+0000 7f0ce69b7700 1 -- 172.21.15.97:0/271814224 shutdown_connections 2023-11-24T06:09:48.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.199+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/271814224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce00ce1a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:48.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.199+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/271814224 >> v2:172.21.15.183:3300/0 conn(0x7f0ce00b0fa0 0x7f0ce00afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:48.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.199+0000 7f0ce69b7700 1 -- 172.21.15.97:0/271814224 >> 172.21.15.97:0/271814224 conn(0x7f0ce00b57b0 msgr2=0x7f0ce00b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:48.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.199+0000 7f0ce69b7700 1 -- 172.21.15.97:0/271814224 shutdown_connections 2023-11-24T06:09:48.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.199+0000 7f0ce69b7700 1 -- 172.21.15.97:0/271814224 wait complete. 2023-11-24T06:09:48.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.200+0000 7f0ce69b7700 1 Processor -- start 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.201+0000 7f0ce69b7700 1 -- start start 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.201+0000 7f0ce69b7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00b0fa0 0x7f0ce0158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.201+0000 7f0ce69b7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce0156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.201+0000 7f0ce69b7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ce00d0c40 0x7f0ce0156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.201+0000 7f0ce69b7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce015c3e0 con 0x7f0ce00ca190 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.201+0000 7f0ce69b7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce015b340 con 0x7f0ce00b0fa0 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.201+0000 7f0ce69b7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce015b620 con 0x7f0ce00d0c40 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.201+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce0156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce0156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46708/0 (socket says 172.21.15.97:46708) 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdf7fe700 1 -- 172.21.15.97:0/1126755306 learned_addr learned my addr 172.21.15.97:0/1126755306 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdffff700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00b0fa0 0x7f0ce0158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:48.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0ce4f54700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ce00d0c40 0x7f0ce0156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdf7fe700 1 -- 172.21.15.97:0/1126755306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ce00d0c40 msgr2=0x7f0ce0156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdf7fe700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ce00d0c40 0x7f0ce0156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdf7fe700 1 -- 172.21.15.97:0/1126755306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00b0fa0 msgr2=0x7f0ce0158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdf7fe700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00b0fa0 0x7f0ce0158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdf7fe700 1 -- 172.21.15.97:0/1126755306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd8009b30 con 0x7f0ce00ca190 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdf7fe700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce0156040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c450 tx=0x7f0cd400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdcff9700 1 -- 172.21.15.97:0/1126755306 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40052d0 con 0x7f0ce00ca190 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdcff9700 1 -- 172.21.15.97:0/1126755306 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd4005430 con 0x7f0ce00ca190 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce015b900 con 0x7f0ce00ca190 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0cdcff9700 1 -- 172.21.15.97:0/1126755306 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40038c0 con 0x7f0ce00ca190 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.202+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce015bec0 con 0x7f0ce00ca190 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.203+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce015c080 con 0x7f0ce00ca190 2023-11-24T06:09:48.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.203+0000 7f0cdcff9700 1 -- 172.21.15.97:0/1126755306 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0cd4003a20 con 0x7f0ce00ca190 2023-11-24T06:09:48.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.204+0000 7f0cdcff9700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ccc06aed0 0x7f0ccc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:48.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.204+0000 7f0cdcff9700 1 -- 172.21.15.97:0/1126755306 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0cd4088480 con 0x7f0ce00ca190 2023-11-24T06:09:48.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.204+0000 7f0cdffff700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ccc06aed0 0x7f0ccc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:48.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.204+0000 7f0cdffff700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ccc06aed0 0x7f0ccc06d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00cf180 tx=0x7f0cd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:48.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.208+0000 7f0cdcff9700 1 -- 172.21.15.97:0/1126755306 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cd4053830 con 0x7f0ce00ca190 2023-11-24T06:09:48.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-24T06:09:48.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-24T06:09:48.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:48.419+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0ce0062840 con 0x7f0ce00ca190 2023-11-24T06:09:49.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.006+0000 7f0cdcff9700 1 -- 172.21.15.97:0/1126755306 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0ce0062840 con 0x7f0ce00ca190 2023-11-24T06:09:49.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ccc06aed0 msgr2=0x7f0ccc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ccc06aed0 0x7f0ccc06d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00cf180 tx=0x7f0cd0008040 comp rx=0 tx=0).stop 2023-11-24T06:09:49.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 msgr2=0x7f0ce0156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce0156040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400c450 tx=0x7f0cd400c760 comp rx=0 tx=0).stop 2023-11-24T06:09:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 shutdown_connections 2023-11-24T06:09:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ccc06aed0 0x7f0ccc06d390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0ce00d0c40 0x7f0ce0156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ce00b0fa0 0x7f0ce0158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 --2- 172.21.15.97:0/1126755306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ce00ca190 0x7f0ce0156040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 >> 172.21.15.97:0/1126755306 conn(0x7f0ce00b57b0 msgr2=0x7f0ce00b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 shutdown_connections 2023-11-24T06:09:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.010+0000 7f0ce69b7700 1 -- 172.21.15.97:0/1126755306 wait complete. 2023-11-24T06:09:49.015 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-11-24T06:09:49.029 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:09:49.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.150+0000 7fc072e8b700 1 Processor -- start 2023-11-24T06:09:49.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.150+0000 7fc072e8b700 1 -- start start 2023-11-24T06:09:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc072e8b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc06c15bc90 0x7fc06c0d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc072e8b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc06c06f770 0x7fc06c0d06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc072e8b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c06fdd0 con 0x7fc06c15c070 2023-11-24T06:09:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc072e8b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06c0700b0 con 0x7fc06c06f770 2023-11-24T06:09:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc072e8b700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06c070390 con 0x7fc06c15bc90 2023-11-24T06:09:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc071428700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc06c06f770 0x7fc06c0d06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc06bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc06c15c070 0x7fc06c06f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39168/0 (socket says 172.21.15.97:39168) 2023-11-24T06:09:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc06bfff700 1 -- 172.21.15.97:0/1928132778 learned_addr learned my addr 172.21.15.97:0/1928132778 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:49.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.151+0000 7fc071428700 1 -- 172.21.15.97:0/1928132778 >> v2:172.21.15.183:3300/0 conn(0x7fc06c15bc90 msgr2=0x7fc06c0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc071428700 1 --2- 172.21.15.97:0/1928132778 >> v2:172.21.15.183:3300/0 conn(0x7fc06c15bc90 0x7fc06c0d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc071428700 1 -- 172.21.15.97:0/1928132778 >> v1:172.21.15.97:6789/0 conn(0x7fc06c15c070 legacy=0x7fc06c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc071428700 1 -- 172.21.15.97:0/1928132778 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c070670 con 0x7fc06c06f770 2023-11-24T06:09:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc06b7fe700 1 -- 172.21.15.97:0/1928132778 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656324889 0 0) 0x7fc06c06fdd0 con 0x7fc06c15c070 2023-11-24T06:09:49.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc071428700 1 --2- 172.21.15.97:0/1928132778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c06f770 0x7fc06c0d06e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc06c09fcd0 tx=0x7fc06400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a9540ed0f8c4f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc06b7fe700 1 -- 172.21.15.97:0/1928132778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400e070 con 0x7fc06c06f770 2023-11-24T06:09:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc06b7fe700 1 -- 172.21.15.97:0/1928132778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06400bbd0 con 0x7fc06c06f770 2023-11-24T06:09:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc06b7fe700 1 -- 172.21.15.97:0/1928132778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064012430 con 0x7fc06c06f770 2023-11-24T06:09:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc072e8b700 1 -- 172.21.15.97:0/1928132778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c06f770 msgr2=0x7fc06c0d06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc072e8b700 1 --2- 172.21.15.97:0/1928132778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c06f770 0x7fc06c0d06e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc06c09fcd0 tx=0x7fc06400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:49.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc072e8b700 1 -- 172.21.15.97:0/1928132778 shutdown_connections 2023-11-24T06:09:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc072e8b700 1 --2- 172.21.15.97:0/1928132778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c06f770 0x7fc06c0d06e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc072e8b700 1 --2- 172.21.15.97:0/1928132778 >> v2:172.21.15.183:3300/0 conn(0x7fc06c15bc90 0x7fc06c0d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc072e8b700 1 -- 172.21.15.97:0/1928132778 >> 172.21.15.97:0/1928132778 conn(0x7fc06c073130 msgr2=0x7fc06c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc072e8b700 1 -- 172.21.15.97:0/1928132778 shutdown_connections 2023-11-24T06:09:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.152+0000 7fc072e8b700 1 -- 172.21.15.97:0/1928132778 wait complete. 2023-11-24T06:09:49.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.153+0000 7fc072e8b700 1 Processor -- start 2023-11-24T06:09:49.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.154+0000 7fc072e8b700 1 -- start start 2023-11-24T06:09:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.154+0000 7fc072e8b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc06c06f770 0x7fc06c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.154+0000 7fc072e8b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc06c15bc90 0x7fc06c09de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.154+0000 7fc072e8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c15c070 0x7fc06c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.154+0000 7fc072e8b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0c9d50 con 0x7fc06c15c070 2023-11-24T06:09:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.154+0000 7fc072e8b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c09e370 con 0x7fc06c06f770 2023-11-24T06:09:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.154+0000 7fc072e8b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0c9fa0 con 0x7fc06c15bc90 2023-11-24T06:09:49.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.154+0000 7fc071428700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c15c070 0x7fc06c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc071428700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c15c070 0x7fc06c15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46726/0 (socket says 172.21.15.97:46726) 2023-11-24T06:09:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc071428700 1 -- 172.21.15.97:0/3101575910 learned_addr learned my addr 172.21.15.97:0/3101575910 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc071428700 1 -- 172.21.15.97:0/3101575910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc06c15bc90 msgr2=0x7fc06c09de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc071428700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc06c15bc90 0x7fc06c09de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc071428700 1 -- 172.21.15.97:0/3101575910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc06c06f770 msgr2=0x7fc06c15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc070c27700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc06c06f770 0x7fc06c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc071428700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc06c06f770 0x7fc06c15a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc071428700 1 -- 172.21.15.97:0/3101575910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc064009a70 con 0x7fc06c15c070 2023-11-24T06:09:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc071428700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c15c070 0x7fc06c15cb20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc064014040 tx=0x7fc064004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc0697fa700 1 -- 172.21.15.97:0/3101575910 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064012430 con 0x7fc06c15c070 2023-11-24T06:09:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc0697fa700 1 -- 172.21.15.97:0/3101575910 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc064020070 con 0x7fc06c15c070 2023-11-24T06:09:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc0697fa700 1 -- 172.21.15.97:0/3101575910 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064004280 con 0x7fc06c15c070 2023-11-24T06:09:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c0ca280 con 0x7fc06c15c070 2023-11-24T06:09:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.155+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c0ca840 con 0x7fc06c15c070 2023-11-24T06:09:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.156+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c067f60 con 0x7fc06c15c070 2023-11-24T06:09:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.157+0000 7fc0697fa700 1 -- 172.21.15.97:0/3101575910 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0640043e0 con 0x7fc06c15c070 2023-11-24T06:09:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.157+0000 7fc0697fa700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc05006aed0 0x7fc05006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.157+0000 7fc0697fa700 1 -- 172.21.15.97:0/3101575910 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc064022070 con 0x7fc06c15c070 2023-11-24T06:09:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.157+0000 7fc070c27700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc05006aed0 0x7fc05006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.157+0000 7fc070c27700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc05006aed0 0x7fc05006d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc060006fd0 tx=0x7fc060008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:49.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.161+0000 7fc0697fa700 1 -- 172.21.15.97:0/3101575910 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0640565b0 con 0x7fc06c15c070 2023-11-24T06:09:49.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.436+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc06c0b6d80 con 0x7fc06c15c070 2023-11-24T06:09:49.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.437+0000 7fc0697fa700 1 -- 172.21.15.97:0/3101575910 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc064059bd0 con 0x7fc06c15c070 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlijU/IxAAp5/+zEd6ER/JFqum0ptk0Q==", 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:09:49.440 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLftpLxAAH+Ew2GQv73YtKLzZJjSBew==", 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:09:49.441 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlGtxpKhAAiQm5ore3YYETyG+sIKzEcw==", 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:09:49.442 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBl05iyNhAAHZYZ5pGghTwe55n84iXXQw==", 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:09:49.443 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDAPWBlwipJFBAAGLCD8wGcYoSy4yM4RXb0/Q==", 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDDPWBlgW7fExAAQuUSGJn+ZfHSMwSi5r1pQg==", 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-11-24T06:09:49.445 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDGPWBl3E8uExAAJKjzbRFyYie6dmnRrP7VVw==", 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJPWBl5gnRExAAbKbgF6DcdTSv8M84GXyONw==", 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.446 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDPPWBlmCfaFRAAsIQWHUnlgDQcRXLndlUZKg==", 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.447 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSPWBlPxfqFRAARyMzgivfTfgmFxZRjp54JQ==", 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:09:49.448 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVPWBl8cjTFBAAsNfkNY4gGVTJwh+mMtADgg==", 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDYPWBljh8EFBAA4nVPoyEXigIDTUB6enXsXA==", 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.450 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA==", 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.451 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlOXAsBhAABOT6CW2RVj7Iv5XMPvQTgQ==", 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlJRe4AhAAGQVWmqQr/urD7AefOfmoZQ==", 2023-11-24T06:09:49.452 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlwX2YCRAAIM9eB39aohf1xUdEvHr/tA==", 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:09:49.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlVBcHBhAAbu6IOCKCLUA+5APAYwbRUw==", 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.454 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mds", 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDgPWBlaxxnKxAANtmg28s9Rx5Ty6HcLtUBww==", 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T06:09:49.455 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDgPWBlbyVnKxAAPgtb/jbbYcPFxBtBf8ghbA==", 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-osd", 2023-11-24T06:09:49.456 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDgPWBlWy1nKxAATDemZXnFfi0KkKKZ1/+ZRg==", 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDgPWBleDZnKxAAmmsXcG8BdddawOCVbSGkwg==", 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDgPWBlaD5nKxAAHck8ZJH/ygjXkYDetMA8qA==", 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDgPWBlr0ZnKxAAjVMNoZiwxJp2BlbugWcrKw==", 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.458 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlOezbHxAAlNbpLHDzRHvk2SXFvQoTXQ==", 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBla7myGxAAHEPHxXHfBzCeqi2nLjhK6g==", 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-11-24T06:09:49.460 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:09:49.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc05006aed0 msgr2=0x7fc05006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc05006aed0 0x7fc05006d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc060006fd0 tx=0x7fc060008040 comp rx=0 tx=0).stop 2023-11-24T06:09:49.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c15c070 msgr2=0x7fc06c15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c15c070 0x7fc06c15cb20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc064014040 tx=0x7fc064004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 shutdown_connections 2023-11-24T06:09:49.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc06c15c070 0x7fc06c15cb20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc06c06f770 0x7fc06c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc05006aed0 0x7fc05006d390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 --2- 172.21.15.97:0/3101575910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc06c15bc90 0x7fc06c09de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 >> 172.21.15.97:0/3101575910 conn(0x7fc06c073130 msgr2=0x7fc06c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:49.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 shutdown_connections 2023-11-24T06:09:49.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.439+0000 7fc072e8b700 1 -- 172.21.15.97:0/3101575910 wait complete. 2023-11-24T06:09:49.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T06:09:49.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.581+0000 7f956434d700 1 Processor -- start 2023-11-24T06:09:49.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.581+0000 7f956434d700 1 -- start start 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.582+0000 7f956434d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c0af4d0 0x7f955c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.582+0000 7f956434d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0ae430 con 0x7f955c15e080 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.582+0000 7f956434d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0ae710 con 0x7f955c06c710 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.582+0000 7f956434d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0af0b0 con 0x7f955c0af4d0 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.582+0000 7f95620e9700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f955c15e080 0x7f955c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39182/0 (socket says 172.21.15.97:39182) 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.582+0000 7f95620e9700 1 -- 172.21.15.97:0/772546 learned_addr learned my addr 172.21.15.97:0/772546 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95628ea700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c0af4d0 0x7f955c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667895207 0 0) 0x7f955c0ae430 con 0x7f955c15e080 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95480035b0 con 0x7f955c15e080 2023-11-24T06:09:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1537826036 0 0) 0x7f95480035b0 con 0x7f955c15e080 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c0af4d0 msgr2=0x7f955c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c0af4d0 0x7f955c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 >> v1:172.21.15.183:6789/0 conn(0x7f955c06c710 legacy=0x7f955c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0ae9f0 con 0x7f955c15e080 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f954c001670 con 0x7f955c15e080 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 >> v1:172.21.15.97:6789/0 conn(0x7f955c15e080 legacy=0x7f955c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c0af4d0 0x7f9548003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9548003fb0 0x7f9548004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548004d90 0x7f9548005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0af0b0 con 0x7f9548003fb0 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954800baf0 con 0x7f9548004d90 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954800bcb0 con 0x7f955c0af4d0 2023-11-24T06:09:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95618e8700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9548003fb0 0x7f9548004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95620e9700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c0af4d0 0x7f9548003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95628ea700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548004d90 0x7f9548005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95618e8700 1 -- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c0af4d0 msgr2=0x7f9548003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95618e8700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c0af4d0 0x7f9548003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95618e8700 1 -- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548004d90 msgr2=0x7f9548005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95618e8700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548004d90 0x7f9548005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.583+0000 7f95618e8700 1 -- 172.21.15.97:0/772546 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480035b0 con 0x7f9548003fb0 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f95618e8700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9548003fb0 0x7f9548004850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f95580017c0 tx=0x7f9548005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64932974e0c24895 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95580052d0 con 0x7f9548003fb0 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9558005430 con 0x7f9548003fb0 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f95610e7700 1 -- 172.21.15.97:0/772546 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95580038c0 con 0x7f9548003fb0 2023-11-24T06:09:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f956434d700 1 -- 172.21.15.97:0/772546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9548003fb0 msgr2=0x7f9548004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f956434d700 1 --2- 172.21.15.97:0/772546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9548003fb0 0x7f9548004850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f95580017c0 tx=0x7f9548005880 comp rx=0 tx=0).stop 2023-11-24T06:09:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f95628ea700 1 -- 172.21.15.97:0/772546 reap_dead start 2023-11-24T06:09:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f956434d700 1 -- 172.21.15.97:0/772546 shutdown_connections 2023-11-24T06:09:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f956434d700 1 -- 172.21.15.97:0/772546 >> 172.21.15.97:0/772546 conn(0x7f955c071b90 msgr2=0x7f955c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f956434d700 1 -- 172.21.15.97:0/772546 shutdown_connections 2023-11-24T06:09:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.584+0000 7f956434d700 1 -- 172.21.15.97:0/772546 wait complete. 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f956434d700 1 Processor -- start 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f956434d700 1 -- start start 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f956434d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548003fb0 0x7f955c1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f956434d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c1521c0 0x7f955c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f956434d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c15ee70 0x7f955c0cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f956434d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c153b30 con 0x7f955c15ee70 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f956434d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1e67b0 con 0x7f9548003fb0 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f956434d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1e7050 con 0x7f955c1521c0 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.586+0000 7f95628ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c15ee70 0x7f955c0cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95628ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c15ee70 0x7f955c0cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46746/0 (socket says 172.21.15.97:46746) 2023-11-24T06:09:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95628ea700 1 -- 172.21.15.97:0/624415187 learned_addr learned my addr 172.21.15.97:0/624415187 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95618e8700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c1521c0 0x7f955c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95620e9700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548003fb0 0x7f955c1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95628ea700 1 -- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c1521c0 msgr2=0x7f955c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95628ea700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c1521c0 0x7f955c0cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95628ea700 1 -- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548003fb0 msgr2=0x7f955c1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95628ea700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548003fb0 0x7f955c1519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95628ea700 1 -- 172.21.15.97:0/624415187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c15f410 con 0x7f955c15ee70 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95620e9700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548003fb0 0x7f955c1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f95628ea700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c15ee70 0x7f955c0cfd30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9554007d10 tx=0x7f95540114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f9552ffd700 1 -- 172.21.15.97:0/624415187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400e070 con 0x7f955c15ee70 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f9552ffd700 1 -- 172.21.15.97:0/624415187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9554003c30 con 0x7f955c15ee70 2023-11-24T06:09:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c1e6a00 con 0x7f955c15ee70 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f9552ffd700 1 -- 172.21.15.97:0/624415187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554003e20 con 0x7f955c15ee70 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.587+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c152950 con 0x7f955c15ee70 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.588+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c0753f0 con 0x7f955c15ee70 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.588+0000 7f9552ffd700 1 -- 172.21.15.97:0/624415187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9554011a40 con 0x7f955c15ee70 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.589+0000 7f9552ffd700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954006aed0 0x7f954006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.589+0000 7f9552ffd700 1 -- 172.21.15.97:0/624415187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9554089670 con 0x7f955c15ee70 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.589+0000 7f95620e9700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954006aed0 0x7f954006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.589+0000 7f95620e9700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954006aed0 0x7f954006d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f954c000f60 tx=0x7f954c000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.593+0000 7f9552ffd700 1 -- 172.21.15.97:0/624415187 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9554054b00 con 0x7f955c15ee70 2023-11-24T06:09:49.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.861+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f955c068670 con 0x7f955c15ee70 2023-11-24T06:09:49.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.863+0000 7f9552ffd700 1 -- 172.21.15.97:0/624415187 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9554011cf0 con 0x7f955c15ee70 2023-11-24T06:09:49.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.865+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954006aed0 msgr2=0x7f954006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.865+0000 7f956434d700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954006aed0 0x7f954006d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f954c000f60 tx=0x7f954c000ed0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.865+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c15ee70 msgr2=0x7f955c0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.865+0000 7f956434d700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c15ee70 0x7f955c0cfd30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9554007d10 tx=0x7f95540114b0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.866+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 shutdown_connections 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.866+0000 7f956434d700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f955c15ee70 0x7f955c0cfd30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.866+0000 7f956434d700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9548003fb0 0x7f955c1519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.866+0000 7f956434d700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f954006aed0 0x7f954006d390 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.866+0000 7f956434d700 1 --2- 172.21.15.97:0/624415187 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f955c1521c0 0x7f955c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.866+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 >> 172.21.15.97:0/624415187 conn(0x7f955c071b90 msgr2=0x7f955c0b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.866+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 shutdown_connections 2023-11-24T06:09:49.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.866+0000 7f956434d700 1 -- 172.21.15.97:0/624415187 wait complete. 2023-11-24T06:09:49.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T06:09:50.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.999+0000 7f6b05c53700 1 Processor -- start 2023-11-24T06:09:50.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:49.999+0000 7f6b05c53700 1 -- start start 2023-11-24T06:09:50.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6b05c53700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b000d2240 0x7f6b000b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6b05c53700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6b0015e1b0 0x7f6b000cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6b05c53700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b000b1d40 con 0x7f6b0015ddd0 2023-11-24T06:09:50.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6b05c53700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b000b28c0 con 0x7f6b000d2240 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6b05c53700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0006c000 con 0x7f6b0015e1b0 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6afeffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b000d2240 0x7f6b000b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6aff7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6b0015ddd0 0x7f6b000d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39202/0 (socket says 172.21.15.97:39202) 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6aff7fe700 1 -- 172.21.15.97:0/104407976 learned_addr learned my addr 172.21.15.97:0/104407976 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6afffff700 1 --2- 172.21.15.97:0/104407976 >> v2:172.21.15.183:3300/0 conn(0x7f6b0015e1b0 0x7f6b000cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6afeffd700 1 -- 172.21.15.97:0/104407976 >> v2:172.21.15.183:3300/0 conn(0x7f6b0015e1b0 msgr2=0x7f6b000cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6afeffd700 1 --2- 172.21.15.97:0/104407976 >> v2:172.21.15.183:3300/0 conn(0x7f6b0015e1b0 0x7f6b000cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6afeffd700 1 -- 172.21.15.97:0/104407976 >> v1:172.21.15.97:6789/0 conn(0x7f6b0015ddd0 legacy=0x7f6b000d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6afeffd700 1 -- 172.21.15.97:0/104407976 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b000b1f90 con 0x7f6b000d2240 2023-11-24T06:09:50.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.000+0000 7f6afeffd700 1 --2- 172.21.15.97:0/104407976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b000d2240 0x7f6b000b1680 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009840 tx=0x7f6ae8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c8864916aed107 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.001+0000 7f6afe7fc700 1 -- 172.21.15.97:0/104407976 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8011830 con 0x7f6b000d2240 2023-11-24T06:09:50.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.001+0000 7f6afe7fc700 1 -- 172.21.15.97:0/104407976 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae8011990 con 0x7f6b000d2240 2023-11-24T06:09:50.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.001+0000 7f6afe7fc700 1 -- 172.21.15.97:0/104407976 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8011d20 con 0x7f6b000d2240 2023-11-24T06:09:50.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.002+0000 7f6b05c53700 1 -- 172.21.15.97:0/104407976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b000d2240 msgr2=0x7f6b000b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.002+0000 7f6b05c53700 1 --2- 172.21.15.97:0/104407976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b000d2240 0x7f6b000b1680 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009840 tx=0x7f6ae8009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:50.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.002+0000 7f6b05c53700 1 -- 172.21.15.97:0/104407976 shutdown_connections 2023-11-24T06:09:50.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.002+0000 7f6b05c53700 1 --2- 172.21.15.97:0/104407976 >> v2:172.21.15.183:3300/0 conn(0x7f6b0015e1b0 0x7f6b000cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.002+0000 7f6b05c53700 1 --2- 172.21.15.97:0/104407976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b000d2240 0x7f6b000b1680 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.002+0000 7f6b05c53700 1 -- 172.21.15.97:0/104407976 >> 172.21.15.97:0/104407976 conn(0x7f6b000731a0 msgr2=0x7f6b0009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:50.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.002+0000 7f6b05c53700 1 -- 172.21.15.97:0/104407976 shutdown_connections 2023-11-24T06:09:50.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.002+0000 7f6b05c53700 1 -- 172.21.15.97:0/104407976 wait complete. 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.004+0000 7f6b05c53700 1 Processor -- start 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.004+0000 7f6b05c53700 1 -- start start 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6b05c53700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b000d2240 0x7f6b000b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6b05c53700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0015ddd0 0x7f6b000b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6b05c53700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b0015e1b0 0x7f6b00151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6b05c53700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0009dbf0 con 0x7f6b0015ddd0 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6b05c53700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0009e230 con 0x7f6b000d2240 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6b05c53700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0009e510 con 0x7f6b0015e1b0 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0015ddd0 0x7f6b000b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0015ddd0 0x7f6b000b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46756/0 (socket says 172.21.15.97:46756) 2023-11-24T06:09:50.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 -- 172.21.15.97:0/3938486424 learned_addr learned my addr 172.21.15.97:0/3938486424 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6aff7fe700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b000d2240 0x7f6b000b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afffff700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b0015e1b0 0x7f6b00151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 -- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b0015e1b0 msgr2=0x7f6b00151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b0015e1b0 0x7f6b00151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 -- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b000d2240 msgr2=0x7f6b000b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b000d2240 0x7f6b000b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 -- 172.21.15.97:0/3938486424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae80094a0 con 0x7f6b0015ddd0 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afffff700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b0015e1b0 0x7f6b00151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6afeffd700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0015ddd0 0x7f6b000b5ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8000c00 tx=0x7f6ae8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6af7fff700 1 -- 172.21.15.97:0/3938486424 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80036a0 con 0x7f6b0015ddd0 2023-11-24T06:09:50.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6af7fff700 1 -- 172.21.15.97:0/3938486424 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae801e070 con 0x7f6b0015ddd0 2023-11-24T06:09:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b000b2ed0 con 0x7f6b0015ddd0 2023-11-24T06:09:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6af7fff700 1 -- 172.21.15.97:0/3938486424 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8003a60 con 0x7f6b0015ddd0 2023-11-24T06:09:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.005+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0009ead0 con 0x7f6b0015ddd0 2023-11-24T06:09:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.007+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b00067f60 con 0x7f6b0015ddd0 2023-11-24T06:09:50.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.007+0000 7f6af7fff700 1 -- 172.21.15.97:0/3938486424 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ae8003bc0 con 0x7f6b0015ddd0 2023-11-24T06:09:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.008+0000 7f6af7fff700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ae006aed0 0x7f6ae006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.008+0000 7f6af7fff700 1 -- 172.21.15.97:0/3938486424 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6ae8089d80 con 0x7f6b0015ddd0 2023-11-24T06:09:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.008+0000 7f6aff7fe700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ae006aed0 0x7f6ae006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.008+0000 7f6aff7fe700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ae006aed0 0x7f6ae006d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6af0005c90 tx=0x7f6af0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.012+0000 7f6af7fff700 1 -- 172.21.15.97:0/3938486424 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ae8055210 con 0x7f6b0015ddd0 2023-11-24T06:09:50.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.280+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f6b000b7c60 con 0x7f6b0015ddd0 2023-11-24T06:09:50.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.282+0000 7f6af7fff700 1 -- 172.21.15.97:0/3938486424 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6ae80200a0 con 0x7f6b0015ddd0 2023-11-24T06:09:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ae006aed0 msgr2=0x7f6ae006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ae006aed0 0x7f6ae006d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6af0005c90 tx=0x7f6af0009040 comp rx=0 tx=0).stop 2023-11-24T06:09:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0015ddd0 msgr2=0x7f6b000b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0015ddd0 0x7f6b000b5ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8000c00 tx=0x7f6ae8004750 comp rx=0 tx=0).stop 2023-11-24T06:09:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 shutdown_connections 2023-11-24T06:09:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ae006aed0 0x7f6ae006d390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b0015e1b0 0x7f6b00151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b000d2240 0x7f6b000b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 --2- 172.21.15.97:0/3938486424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0015ddd0 0x7f6b000b5ed0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 >> 172.21.15.97:0/3938486424 conn(0x7f6b000731a0 msgr2=0x7f6b000cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:50.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 shutdown_connections 2023-11-24T06:09:50.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.285+0000 7f6b05c53700 1 -- 172.21.15.97:0/3938486424 wait complete. 2023-11-24T06:09:50.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T06:09:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.418+0000 7f9ed7610700 1 Processor -- start 2023-11-24T06:09:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.419+0000 7f9ed7610700 1 -- start start 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.419+0000 7f9ed7610700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9ed015e080 0x7f9ed00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.419+0000 7f9ed7610700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ed00af4d0 0x7f9ed00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.419+0000 7f9ed7610700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00ae430 con 0x7f9ed006c000 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.419+0000 7f9ed7610700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00af0b0 con 0x7f9ed00af4d0 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.419+0000 7f9ed7610700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00b05a0 con 0x7f9ed015e080 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed4bab700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ed00af4d0 0x7f9ed00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed5bad700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9ed006c000 0x7f9ed0073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39218/0 (socket says 172.21.15.97:39218) 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed5bad700 1 -- 172.21.15.97:0/2213473372 learned_addr learned my addr 172.21.15.97:0/2213473372 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed53ac700 1 --2- 172.21.15.97:0/2213473372 >> v2:172.21.15.183:3300/0 conn(0x7f9ed015e080 0x7f9ed00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed4bab700 1 -- 172.21.15.97:0/2213473372 >> v2:172.21.15.183:3300/0 conn(0x7f9ed015e080 msgr2=0x7f9ed00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed4bab700 1 --2- 172.21.15.97:0/2213473372 >> v2:172.21.15.183:3300/0 conn(0x7f9ed015e080 0x7f9ed00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed4bab700 1 -- 172.21.15.97:0/2213473372 >> v1:172.21.15.97:6789/0 conn(0x7f9ed006c000 legacy=0x7f9ed0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed4bab700 1 -- 172.21.15.97:0/2213473372 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00ae710 con 0x7f9ed00af4d0 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed53ac700 1 --2- 172.21.15.97:0/2213473372 >> v2:172.21.15.183:3300/0 conn(0x7f9ed015e080 0x7f9ed00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ed4bab700 1 --2- 172.21.15.97:0/2213473372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00af4d0 0x7f9ed00afb90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0072a90 tx=0x7f9ecc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8042bea93d7550 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ec7fff700 1 -- 172.21.15.97:0/2213473372 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc011830 con 0x7f9ed00af4d0 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ec7fff700 1 -- 172.21.15.97:0/2213473372 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc011990 con 0x7f9ed00af4d0 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.420+0000 7f9ec7fff700 1 -- 172.21.15.97:0/2213473372 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc011d20 con 0x7f9ed00af4d0 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.421+0000 7f9ed7610700 1 -- 172.21.15.97:0/2213473372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00af4d0 msgr2=0x7f9ed00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.421+0000 7f9ed7610700 1 --2- 172.21.15.97:0/2213473372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00af4d0 0x7f9ed00afb90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0072a90 tx=0x7f9ecc009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.421+0000 7f9ed7610700 1 -- 172.21.15.97:0/2213473372 shutdown_connections 2023-11-24T06:09:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.421+0000 7f9ed7610700 1 --2- 172.21.15.97:0/2213473372 >> v2:172.21.15.183:3300/0 conn(0x7f9ed015e080 0x7f9ed00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.421+0000 7f9ed7610700 1 --2- 172.21.15.97:0/2213473372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00af4d0 0x7f9ed00afb90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.421+0000 7f9ed7610700 1 -- 172.21.15.97:0/2213473372 >> 172.21.15.97:0/2213473372 conn(0x7f9ed0071b90 msgr2=0x7f9ed0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.421+0000 7f9ed7610700 1 -- 172.21.15.97:0/2213473372 shutdown_connections 2023-11-24T06:09:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.421+0000 7f9ed7610700 1 -- 172.21.15.97:0/2213473372 wait complete. 2023-11-24T06:09:50.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.422+0000 7f9ed7610700 1 Processor -- start 2023-11-24T06:09:50.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.422+0000 7f9ed7610700 1 -- start start 2023-11-24T06:09:50.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed7610700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ed006c000 0x7f9ed014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed7610700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ed00af4d0 0x7f9ed0153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed7610700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed015e080 0x7f9ed0158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed7610700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01574e0 con 0x7f9ed015e080 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed7610700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01f4ce0 con 0x7f9ed006c000 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed7610700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0157780 con 0x7f9ed00af4d0 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed015e080 0x7f9ed0158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed015e080 0x7f9ed0158f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46772/0 (socket says 172.21.15.97:46772) 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 -- 172.21.15.97:0/3209049678 learned_addr learned my addr 172.21.15.97:0/3209049678 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed53ac700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ed006c000 0x7f9ed014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 -- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ed00af4d0 msgr2=0x7f9ed0153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed4bab700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ed00af4d0 0x7f9ed0153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ed00af4d0 0x7f9ed0153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 -- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ed006c000 msgr2=0x7f9ed014a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ed006c000 0x7f9ed014a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 -- 172.21.15.97:0/3209049678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0094a0 con 0x7f9ed015e080 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed53ac700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ed006c000 0x7f9ed014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed5bad700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed015e080 0x7f9ed0158f90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800cbc0 tx=0x7f9ec800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ec5ffb700 1 -- 172.21.15.97:0/3209049678 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec800ab50 con 0x7f9ed015e080 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ec5ffb700 1 -- 172.21.15.97:0/3209049678 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec800acb0 con 0x7f9ed015e080 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.423+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed0157a60 con 0x7f9ed015e080 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.424+0000 7f9ec5ffb700 1 -- 172.21.15.97:0/3209049678 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8012600 con 0x7f9ed015e080 2023-11-24T06:09:50.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.424+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed01544a0 con 0x7f9ed015e080 2023-11-24T06:09:50.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.425+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00753f0 con 0x7f9ed015e080 2023-11-24T06:09:50.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.425+0000 7f9ec5ffb700 1 -- 172.21.15.97:0/3209049678 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ec801b3b0 con 0x7f9ed015e080 2023-11-24T06:09:50.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.425+0000 7f9ec5ffb700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9eb406aed0 0x7f9eb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.426+0000 7f9ec5ffb700 1 -- 172.21.15.97:0/3209049678 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9ec8088ea0 con 0x7f9ed015e080 2023-11-24T06:09:50.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.426+0000 7f9ed53ac700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9eb406aed0 0x7f9eb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.426+0000 7f9ed53ac700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9eb406aed0 0x7f9eb406d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0008910 tx=0x7f9ec0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.430+0000 7f9ec5ffb700 1 -- 172.21.15.97:0/3209049678 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ec80543e0 con 0x7f9ed015e080 2023-11-24T06:09:50.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.697+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9ed0067f60 con 0x7f9ed015e080 2023-11-24T06:09:50.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.699+0000 7f9ec5ffb700 1 -- 172.21.15.97:0/3209049678 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9ec8057a00 con 0x7f9ed015e080 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9eb406aed0 msgr2=0x7f9eb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9eb406aed0 0x7f9eb406d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0008910 tx=0x7f9ec0008040 comp rx=0 tx=0).stop 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed015e080 msgr2=0x7f9ed0158f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed015e080 0x7f9ed0158f90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800cbc0 tx=0x7f9ec800cf80 comp rx=0 tx=0).stop 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 shutdown_connections 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed015e080 0x7f9ed0158f90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ed006c000 0x7f9ed014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9eb406aed0 0x7f9eb406d390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 --2- 172.21.15.97:0/3209049678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ed00af4d0 0x7f9ed0153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 >> 172.21.15.97:0/3209049678 conn(0x7f9ed0071b90 msgr2=0x7f9ed006e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 shutdown_connections 2023-11-24T06:09:50.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.701+0000 7f9ed7610700 1 -- 172.21.15.97:0/3209049678 wait complete. 2023-11-24T06:09:50.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T06:09:50.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.834+0000 7fed3ca51700 1 Processor -- start 2023-11-24T06:09:50.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.834+0000 7fed3ca51700 1 -- start start 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.835+0000 7fed3ca51700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed3806c710 0x7fed380b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.835+0000 7fed3ca51700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed380af4d0 0x7fed380afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.835+0000 7fed3ca51700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed380ae430 con 0x7fed3815e080 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.835+0000 7fed3ca51700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed380af0b0 con 0x7fed3806c710 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.835+0000 7fed3ca51700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed380b05a0 con 0x7fed380af4d0 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.835+0000 7fed35d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fed3806c710 0x7fed380b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.835+0000 7fed3659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fed3815e080 0x7fed3806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39228/0 (socket says 172.21.15.97:39228) 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.835+0000 7fed3659c700 1 -- 172.21.15.97:0/2285897873 learned_addr learned my addr 172.21.15.97:0/2285897873 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed36d9d700 1 --2- 172.21.15.97:0/2285897873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed380af4d0 0x7fed380afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed35d9b700 1 -- 172.21.15.97:0/2285897873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed380af4d0 msgr2=0x7fed380afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed35d9b700 1 --2- 172.21.15.97:0/2285897873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed380af4d0 0x7fed380afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed35d9b700 1 -- 172.21.15.97:0/2285897873 >> v1:172.21.15.97:6789/0 conn(0x7fed3815e080 legacy=0x7fed3806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed35d9b700 1 -- 172.21.15.97:0/2285897873 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed380ae710 con 0x7fed3806c710 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed3559a700 1 -- 172.21.15.97:0/2285897873 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371321105 0 0) 0x7fed380ae430 con 0x7fed3815e080 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed35d9b700 1 --2- 172.21.15.97:0/2285897873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3806c710 0x7fed380b5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed28009840 tx=0x7fed28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7df8f506d16176 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed3559a700 1 -- 172.21.15.97:0/2285897873 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28011830 con 0x7fed3806c710 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed3559a700 1 -- 172.21.15.97:0/2285897873 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed28011990 con 0x7fed3806c710 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.836+0000 7fed3559a700 1 -- 172.21.15.97:0/2285897873 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28011d20 con 0x7fed3806c710 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.837+0000 7fed3ca51700 1 -- 172.21.15.97:0/2285897873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3806c710 msgr2=0x7fed380b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.837+0000 7fed3ca51700 1 --2- 172.21.15.97:0/2285897873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3806c710 0x7fed380b5050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fed28009840 tx=0x7fed28009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:50.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.837+0000 7fed3ca51700 1 -- 172.21.15.97:0/2285897873 shutdown_connections 2023-11-24T06:09:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.837+0000 7fed3ca51700 1 --2- 172.21.15.97:0/2285897873 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed380af4d0 0x7fed380afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.837+0000 7fed3ca51700 1 --2- 172.21.15.97:0/2285897873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed3806c710 0x7fed380b5050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.837+0000 7fed3ca51700 1 -- 172.21.15.97:0/2285897873 >> 172.21.15.97:0/2285897873 conn(0x7fed38071b90 msgr2=0x7fed38071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.837+0000 7fed3ca51700 1 -- 172.21.15.97:0/2285897873 shutdown_connections 2023-11-24T06:09:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.837+0000 7fed3ca51700 1 -- 172.21.15.97:0/2285897873 wait complete. 2023-11-24T06:09:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.838+0000 7fed3ca51700 1 Processor -- start 2023-11-24T06:09:50.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.838+0000 7fed3ca51700 1 -- start start 2023-11-24T06:09:50.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed3ca51700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed3806c710 0x7fed381f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed3ca51700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed380af4d0 0x7fed38155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed3ca51700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3815e080 0x7fed380b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed3ca51700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38156ad0 con 0x7fed380af4d0 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed3ca51700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed381567f0 con 0x7fed3815e080 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed3ca51700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3815a8e0 con 0x7fed3806c710 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed380af4d0 0x7fed38155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed380af4d0 0x7fed38155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46784/0 (socket says 172.21.15.97:46784) 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 -- 172.21.15.97:0/1119800207 learned_addr learned my addr 172.21.15.97:0/1119800207 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed3659c700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed3806c710 0x7fed381f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed36d9d700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3815e080 0x7fed380b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 -- 172.21.15.97:0/1119800207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed3806c710 msgr2=0x7fed381f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed3806c710 0x7fed381f3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 -- 172.21.15.97:0/1119800207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3815e080 msgr2=0x7fed380b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3815e080 0x7fed380b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 -- 172.21.15.97:0/1119800207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed280094a0 con 0x7fed380af4d0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.839+0000 7fed35d9b700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed380af4d0 0x7fed38155da0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fed28000c00 tx=0x7fed28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.840+0000 7fed26ffd700 1 -- 172.21.15.97:0/1119800207 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280045f0 con 0x7fed380af4d0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.840+0000 7fed26ffd700 1 -- 172.21.15.97:0/1119800207 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2801f070 con 0x7fed380af4d0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.840+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3815abc0 con 0x7fed380af4d0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.840+0000 7fed26ffd700 1 -- 172.21.15.97:0/1119800207 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28003ae0 con 0x7fed380af4d0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.840+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed381f3580 con 0x7fed380af4d0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.841+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed380753f0 con 0x7fed380af4d0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.841+0000 7fed26ffd700 1 -- 172.21.15.97:0/1119800207 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed28003c40 con 0x7fed380af4d0 2023-11-24T06:09:50.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.841+0000 7fed26ffd700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fed1806aed0 0x7fed1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:50.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.842+0000 7fed26ffd700 1 -- 172.21.15.97:0/1119800207 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fed28021070 con 0x7fed380af4d0 2023-11-24T06:09:50.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.842+0000 7fed3659c700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fed1806aed0 0x7fed1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:50.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.842+0000 7fed3659c700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fed1806aed0 0x7fed1806d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed2c000f60 tx=0x7fed2c007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:50.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:50.846+0000 7fed26ffd700 1 -- 172.21.15.97:0/1119800207 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed28056100 con 0x7fed380af4d0 2023-11-24T06:09:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.111+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fed3815a740 con 0x7fed380af4d0 2023-11-24T06:09:51.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.113+0000 7fed26ffd700 1 -- 172.21.15.97:0/1119800207 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fed3815a740 con 0x7fed380af4d0 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fed1806aed0 msgr2=0x7fed1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fed1806aed0 0x7fed1806d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed2c000f60 tx=0x7fed2c007c90 comp rx=0 tx=0).stop 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed380af4d0 msgr2=0x7fed38155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed380af4d0 0x7fed38155da0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fed28000c00 tx=0x7fed28004750 comp rx=0 tx=0).stop 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 shutdown_connections 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3815e080 0x7fed380b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fed1806aed0 0x7fed1806d390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fed3806c710 0x7fed381f3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 --2- 172.21.15.97:0/1119800207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fed380af4d0 0x7fed38155da0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 >> 172.21.15.97:0/1119800207 conn(0x7fed38071b90 msgr2=0x7fed380b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:51.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 shutdown_connections 2023-11-24T06:09:51.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.116+0000 7fed3ca51700 1 -- 172.21.15.97:0/1119800207 wait complete. 2023-11-24T06:09:51.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T06:09:51.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.249+0000 7fdf52faa700 1 Processor -- start 2023-11-24T06:09:51.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.249+0000 7fdf52faa700 1 -- start start 2023-11-24T06:09:51.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.249+0000 7fdf52faa700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c0d2240 0x7fdf4c0b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.249+0000 7fdf52faa700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf4c15e1b0 0x7fdf4c0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.249+0000 7fdf52faa700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4c0b1d40 con 0x7fdf4c15ddd0 2023-11-24T06:09:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.249+0000 7fdf52faa700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c0b28c0 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.249+0000 7fdf52faa700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c06c000 con 0x7fdf4c0d2240 2023-11-24T06:09:51.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf51547700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf4c15e1b0 0x7fdf4c0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf51547700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf4c15e1b0 0x7fdf4c0cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46794/0 (socket says 172.21.15.97:46794) 2023-11-24T06:09:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf51547700 1 -- 172.21.15.97:0/1956439587 learned_addr learned my addr 172.21.15.97:0/1956439587 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf4bfff700 1 --2- 172.21.15.97:0/1956439587 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c0d2240 0x7fdf4c0b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf51547700 1 -- 172.21.15.97:0/1956439587 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c0d2240 msgr2=0x7fdf4c0b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf51547700 1 --2- 172.21.15.97:0/1956439587 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c0d2240 0x7fdf4c0b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf51547700 1 -- 172.21.15.97:0/1956439587 >> v1:172.21.15.183:6789/0 conn(0x7fdf4c15ddd0 legacy=0x7fdf4c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf51547700 1 -- 172.21.15.97:0/1956439587 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c0b1f90 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf51547700 1 --2- 172.21.15.97:0/1956439587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c0cf000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009e50 tx=0x7fdf4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decf1efae4bcda97 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:51.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf4b7fe700 1 -- 172.21.15.97:0/1956439587 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400e070 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf4b7fe700 1 -- 172.21.15.97:0/1956439587 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4400b910 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.250+0000 7fdf4b7fe700 1 -- 172.21.15.97:0/1956439587 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400bcc0 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.251+0000 7fdf52faa700 1 -- 172.21.15.97:0/1956439587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 msgr2=0x7fdf4c0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.251+0000 7fdf52faa700 1 --2- 172.21.15.97:0/1956439587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c0cf000 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009e50 tx=0x7fdf4400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.251+0000 7fdf52faa700 1 -- 172.21.15.97:0/1956439587 shutdown_connections 2023-11-24T06:09:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.251+0000 7fdf52faa700 1 --2- 172.21.15.97:0/1956439587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c0cf000 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.251+0000 7fdf52faa700 1 --2- 172.21.15.97:0/1956439587 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c0d2240 0x7fdf4c0b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.251+0000 7fdf52faa700 1 -- 172.21.15.97:0/1956439587 >> 172.21.15.97:0/1956439587 conn(0x7fdf4c0731a0 msgr2=0x7fdf4c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.251+0000 7fdf52faa700 1 -- 172.21.15.97:0/1956439587 shutdown_connections 2023-11-24T06:09:51.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.251+0000 7fdf52faa700 1 -- 172.21.15.97:0/1956439587 wait complete. 2023-11-24T06:09:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.252+0000 7fdf52faa700 1 Processor -- start 2023-11-24T06:09:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.252+0000 7fdf52faa700 1 -- start start 2023-11-24T06:09:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.253+0000 7fdf52faa700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c0d2240 0x7fdf4c09bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.253+0000 7fdf52faa700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c15ddd0 0x7fdf4c0b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.253+0000 7fdf52faa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c09f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.253+0000 7fdf52faa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0b74b0 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.253+0000 7fdf52faa700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c09c250 con 0x7fdf4c0d2240 2023-11-24T06:09:51.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.253+0000 7fdf52faa700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c09c530 con 0x7fdf4c15ddd0 2023-11-24T06:09:51.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.253+0000 7fdf51547700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c09f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf51547700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c09f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46800/0 (socket says 172.21.15.97:46800) 2023-11-24T06:09:51.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf51547700 1 -- 172.21.15.97:0/3440532789 learned_addr learned my addr 172.21.15.97:0/3440532789 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:51.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf50d46700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c0d2240 0x7fdf4c09bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf51547700 1 -- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c15ddd0 msgr2=0x7fdf4c0b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf4bfff700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c15ddd0 0x7fdf4c0b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf51547700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c15ddd0 0x7fdf4c0b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf51547700 1 -- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c0d2240 msgr2=0x7fdf4c09bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf51547700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c0d2240 0x7fdf4c09bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf51547700 1 -- 172.21.15.97:0/3440532789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44009b30 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf50d46700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c0d2240 0x7fdf4c09bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:51.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf51547700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c09f4b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0716d0 tx=0x7fdf44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:51.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf497fa700 1 -- 172.21.15.97:0/3440532789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44004960 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf497fa700 1 -- 172.21.15.97:0/3440532789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf440099b0 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c0b2ed0 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf497fa700 1 -- 172.21.15.97:0/3440532789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4401c680 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.254+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c09caf0 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.255+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c067f60 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.256+0000 7fdf497fa700 1 -- 172.21.15.97:0/3440532789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf4401c7e0 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.256+0000 7fdf497fa700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdf3006aed0 0x7fdf3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.256+0000 7fdf497fa700 1 -- 172.21.15.97:0/3440532789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdf4408af70 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.256+0000 7fdf50d46700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdf3006aed0 0x7fdf3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.256+0000 7fdf50d46700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdf3006aed0 0x7fdf3006d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf40000f60 tx=0x7fdf40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:51.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.261+0000 7fdf497fa700 1 -- 172.21.15.97:0/3440532789 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf440bde90 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.530+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fdf4c0b7650 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.532+0000 7fdf497fa700 1 -- 172.21.15.97:0/3440532789 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdf44056400 con 0x7fdf4c15e1b0 2023-11-24T06:09:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdf3006aed0 msgr2=0x7fdf3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdf3006aed0 0x7fdf3006d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf40000f60 tx=0x7fdf40009040 comp rx=0 tx=0).stop 2023-11-24T06:09:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 msgr2=0x7fdf4c09f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c09f4b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0716d0 tx=0x7fdf44004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 shutdown_connections 2023-11-24T06:09:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf4c15e1b0 0x7fdf4c09f4b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf4c0d2240 0x7fdf4c09bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdf3006aed0 0x7fdf3006d390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 --2- 172.21.15.97:0/3440532789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf4c15ddd0 0x7fdf4c0b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 >> 172.21.15.97:0/3440532789 conn(0x7fdf4c0731a0 msgr2=0x7fdf4c070410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 shutdown_connections 2023-11-24T06:09:51.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.534+0000 7fdf52faa700 1 -- 172.21.15.97:0/3440532789 wait complete. 2023-11-24T06:09:51.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T06:09:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.667+0000 7f43120f4700 1 Processor -- start 2023-11-24T06:09:51.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.667+0000 7f43120f4700 1 -- start start 2023-11-24T06:09:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.667+0000 7f43120f4700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f430c15e080 0x7f430c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.667+0000 7f43120f4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c06c710 0x7f430c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f43120f4700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f430c0ae430 con 0x7f430c0af4d0 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f43120f4700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430c0af0b0 con 0x7f430c15e080 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f43120f4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430c0b05a0 con 0x7f430c06c710 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430bfff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c06c710 0x7f430c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430b7fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f430c15e080 0x7f430c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430affd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f430c0af4d0 0x7f430c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:48580/0 (socket says 172.21.15.97:48580) 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430bfff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c06c710 0x7f430c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:33844/0 (socket says 172.21.15.97:33844) 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430b7fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f430c15e080 0x7f430c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:40964/0 (socket says 172.21.15.97:40964) 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430bfff700 1 -- 172.21.15.97:0/4122013631 learned_addr learned my addr 172.21.15.97:0/4122013631 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430bfff700 1 -- 172.21.15.97:0/4122013631 >> v2:172.21.15.183:3300/0 conn(0x7f430c15e080 msgr2=0x7f430c0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430bfff700 1 --2- 172.21.15.97:0/4122013631 >> v2:172.21.15.183:3300/0 conn(0x7f430c15e080 0x7f430c0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430bfff700 1 -- 172.21.15.97:0/4122013631 >> v1:172.21.15.183:6789/0 conn(0x7f430c0af4d0 legacy=0x7f430c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430a7fc700 1 -- 172.21.15.97:0/4122013631 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199506032 0 0) 0x7f430c0ae430 con 0x7f430c0af4d0 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430bfff700 1 -- 172.21.15.97:0/4122013631 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0ae710 con 0x7f430c06c710 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.668+0000 7f430b7fe700 1 --2- 172.21.15.97:0/4122013631 >> v2:172.21.15.183:3300/0 conn(0x7f430c15e080 0x7f430c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f430bfff700 1 --2- 172.21.15.97:0/4122013631 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c06c710 0x7f430c0afb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4304009e50 tx=0x7f430400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3934b5468231b61 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f430a7fc700 1 -- 172.21.15.97:0/4122013631 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430400e070 con 0x7f430c06c710 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f430a7fc700 1 -- 172.21.15.97:0/4122013631 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430400b910 con 0x7f430c06c710 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f430a7fc700 1 -- 172.21.15.97:0/4122013631 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430400bcc0 con 0x7f430c06c710 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f43120f4700 1 -- 172.21.15.97:0/4122013631 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c06c710 msgr2=0x7f430c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f43120f4700 1 --2- 172.21.15.97:0/4122013631 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c06c710 0x7f430c0afb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4304009e50 tx=0x7f430400b330 comp rx=0 tx=0).stop 2023-11-24T06:09:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f43120f4700 1 -- 172.21.15.97:0/4122013631 shutdown_connections 2023-11-24T06:09:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f43120f4700 1 --2- 172.21.15.97:0/4122013631 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c06c710 0x7f430c0afb90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f43120f4700 1 --2- 172.21.15.97:0/4122013631 >> v2:172.21.15.183:3300/0 conn(0x7f430c15e080 0x7f430c0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f43120f4700 1 -- 172.21.15.97:0/4122013631 >> 172.21.15.97:0/4122013631 conn(0x7f430c071b90 msgr2=0x7f430c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f43120f4700 1 -- 172.21.15.97:0/4122013631 shutdown_connections 2023-11-24T06:09:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.669+0000 7f43120f4700 1 -- 172.21.15.97:0/4122013631 wait complete. 2023-11-24T06:09:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.670+0000 7f43120f4700 1 Processor -- start 2023-11-24T06:09:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f43120f4700 1 -- start start 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f43120f4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f430c06c710 0x7f430c14a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f43120f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430c0af4d0 0x7f430c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f43120f4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c15e080 0x7f430c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f43120f4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c15f0c0 con 0x7f430c0af4d0 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f43120f4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c14af00 con 0x7f430c06c710 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f43120f4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430c15e460 con 0x7f430c15e080 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f430affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430c0af4d0 0x7f430c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f430affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430c0af4d0 0x7f430c156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46812/0 (socket says 172.21.15.97:46812) 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.671+0000 7f430affd700 1 -- 172.21.15.97:0/1453438285 learned_addr learned my addr 172.21.15.97:0/1453438285 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f430b7fe700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f430c06c710 0x7f430c14a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f430affd700 1 -- 172.21.15.97:0/1453438285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c15e080 msgr2=0x7f430c1565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f430affd700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c15e080 0x7f430c1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f430affd700 1 -- 172.21.15.97:0/1453438285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f430c06c710 msgr2=0x7f430c14a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f430affd700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f430c06c710 0x7f430c14a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f430affd700 1 -- 172.21.15.97:0/1453438285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0094a0 con 0x7f430c0af4d0 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f430affd700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430c0af4d0 0x7f430c156070 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f430000c540 tx=0x7f430000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f42f3fff700 1 -- 172.21.15.97:0/1453438285 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300004c80 con 0x7f430c0af4d0 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f42f3fff700 1 -- 172.21.15.97:0/1453438285 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4300004de0 con 0x7f430c0af4d0 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4304009b30 con 0x7f430c0af4d0 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f42f3fff700 1 -- 172.21.15.97:0/1453438285 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300007310 con 0x7f430c0af4d0 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.672+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c15ea00 con 0x7f430c0af4d0 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.673+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430c0753f0 con 0x7f430c0af4d0 2023-11-24T06:09:51.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.673+0000 7f42f3fff700 1 -- 172.21.15.97:0/1453438285 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4300007470 con 0x7f430c0af4d0 2023-11-24T06:09:51.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.674+0000 7f42f3fff700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42f806aed0 0x7f42f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:51.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.674+0000 7f42f3fff700 1 -- 172.21.15.97:0/1453438285 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4300088ff0 con 0x7f430c0af4d0 2023-11-24T06:09:51.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.674+0000 7f430b7fe700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42f806aed0 0x7f42f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:51.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.674+0000 7f430b7fe700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42f806aed0 0x7f42f806d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009b80 tx=0x7f42fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:51.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.678+0000 7f42f3fff700 1 -- 172.21.15.97:0/1453438285 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4300054480 con 0x7f430c0af4d0 2023-11-24T06:09:51.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.943+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f430c067f60 con 0x7f430c0af4d0 2023-11-24T06:09:51.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.945+0000 7f42f3fff700 1 -- 172.21.15.97:0/1453438285 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4300057aa0 con 0x7f430c0af4d0 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.947+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42f806aed0 msgr2=0x7f42f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.947+0000 7f43120f4700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42f806aed0 0x7f42f806d390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009b80 tx=0x7f42fc008040 comp rx=0 tx=0).stop 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.947+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430c0af4d0 msgr2=0x7f430c156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.947+0000 7f43120f4700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430c0af4d0 0x7f430c156070 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f430000c540 tx=0x7f430000c900 comp rx=0 tx=0).stop 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.948+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 shutdown_connections 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.948+0000 7f43120f4700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42f806aed0 0x7f42f806d390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.948+0000 7f43120f4700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f430c15e080 0x7f430c1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.948+0000 7f43120f4700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f430c06c710 0x7f430c14a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.948+0000 7f43120f4700 1 --2- 172.21.15.97:0/1453438285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430c0af4d0 0x7f430c156070 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:51.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.948+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 >> 172.21.15.97:0/1453438285 conn(0x7f430c071b90 msgr2=0x7f430c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.948+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 shutdown_connections 2023-11-24T06:09:51.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:51.948+0000 7f43120f4700 1 -- 172.21.15.97:0/1453438285 wait complete. 2023-11-24T06:09:51.963 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.080+0000 7fd3c6fc6700 1 Processor -- start 2023-11-24T06:09:52.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.080+0000 7fd3c6fc6700 1 -- start start 2023-11-24T06:09:52.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c6fc6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c00d2240 0x7fd3c00b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c6fc6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3c015e1b0 0x7fd3c00cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c6fc6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c00b1d40 con 0x7fd3c015ddd0 2023-11-24T06:09:52.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c6fc6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c00b28c0 con 0x7fd3c015e1b0 2023-11-24T06:09:52.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c6fc6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c006c000 con 0x7fd3c00d2240 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c5563700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3c015e1b0 0x7fd3c00cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c5563700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3c015e1b0 0x7fd3c00cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46822/0 (socket says 172.21.15.97:46822) 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c5563700 1 -- 172.21.15.97:0/273606948 learned_addr learned my addr 172.21.15.97:0/273606948 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3bffff700 1 --2- 172.21.15.97:0/273606948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c00d2240 0x7fd3c00b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c5563700 1 -- 172.21.15.97:0/273606948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c00d2240 msgr2=0x7fd3c00b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c5563700 1 --2- 172.21.15.97:0/273606948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c00d2240 0x7fd3c00b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c5563700 1 -- 172.21.15.97:0/273606948 >> v1:172.21.15.183:6789/0 conn(0x7fd3c015ddd0 legacy=0x7fd3c00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c5563700 1 -- 172.21.15.97:0/273606948 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c00b1f90 con 0x7fd3c015e1b0 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.081+0000 7fd3c5563700 1 --2- 172.21.15.97:0/273606948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015e1b0 0x7fd3c00cf000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009e50 tx=0x7fd3b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f60c49bf6ca94 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3bf7fe700 1 -- 172.21.15.97:0/273606948 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800e070 con 0x7fd3c015e1b0 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3bf7fe700 1 -- 172.21.15.97:0/273606948 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b800b9c0 con 0x7fd3c015e1b0 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3bf7fe700 1 -- 172.21.15.97:0/273606948 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b800bd70 con 0x7fd3c015e1b0 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/273606948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015e1b0 msgr2=0x7fd3c00cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/273606948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015e1b0 0x7fd3c00cf000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009e50 tx=0x7fd3b800b330 comp rx=0 tx=0).stop 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/273606948 shutdown_connections 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/273606948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015e1b0 0x7fd3c00cf000 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/273606948 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c00d2240 0x7fd3c00b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/273606948 >> 172.21.15.97:0/273606948 conn(0x7fd3c00731a0 msgr2=0x7fd3c009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/273606948 shutdown_connections 2023-11-24T06:09:52.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.082+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/273606948 wait complete. 2023-11-24T06:09:52.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.083+0000 7fd3c6fc6700 1 Processor -- start 2023-11-24T06:09:52.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.083+0000 7fd3c6fc6700 1 -- start start 2023-11-24T06:09:52.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3c6fc6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3c00d2240 0x7fd3c009bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3c6fc6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015ddd0 0x7fd3c00b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3c6fc6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c015e1b0 0x7fd3c009f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3c6fc6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00b74b0 con 0x7fd3c015ddd0 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3c6fc6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c009c250 con 0x7fd3c00d2240 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3c6fc6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c009c530 con 0x7fd3c015e1b0 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015ddd0 0x7fd3c00b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015ddd0 0x7fd3c00b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46826/0 (socket says 172.21.15.97:46826) 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.084+0000 7fd3bffff700 1 -- 172.21.15.97:0/1785683634 learned_addr learned my addr 172.21.15.97:0/1785683634 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3c4d62700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3c00d2240 0x7fd3c009bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3c5563700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c015e1b0 0x7fd3c009f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bffff700 1 -- 172.21.15.97:0/1785683634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c015e1b0 msgr2=0x7fd3c009f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bffff700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c015e1b0 0x7fd3c009f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bffff700 1 -- 172.21.15.97:0/1785683634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3c00d2240 msgr2=0x7fd3c009bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bffff700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3c00d2240 0x7fd3c009bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bffff700 1 -- 172.21.15.97:0/1785683634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8009b30 con 0x7fd3c015ddd0 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bffff700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015ddd0 0x7fd3c00b6f70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00d800 tx=0x7fd3ac00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bd7fa700 1 -- 172.21.15.97:0/1785683634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0125f0 con 0x7fd3c015ddd0 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bd7fa700 1 -- 172.21.15.97:0/1785683634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac012750 con 0x7fd3c015ddd0 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c00b2ed0 con 0x7fd3c015ddd0 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3bd7fa700 1 -- 172.21.15.97:0/1785683634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac012b00 con 0x7fd3c015ddd0 2023-11-24T06:09:52.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.085+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c009caf0 con 0x7fd3c015ddd0 2023-11-24T06:09:52.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.086+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c0067f60 con 0x7fd3c015ddd0 2023-11-24T06:09:52.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.086+0000 7fd3bd7fa700 1 -- 172.21.15.97:0/1785683634 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3ac012c60 con 0x7fd3c015ddd0 2023-11-24T06:09:52.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.087+0000 7fd3bd7fa700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3a406aed0 0x7fd3a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.087+0000 7fd3bd7fa700 1 -- 172.21.15.97:0/1785683634 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd3ac089630 con 0x7fd3c015ddd0 2023-11-24T06:09:52.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.087+0000 7fd3c4d62700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3a406aed0 0x7fd3a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.087+0000 7fd3c4d62700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3a406aed0 0x7fd3a406d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4000f60 tx=0x7fd3b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.091+0000 7fd3bd7fa700 1 -- 172.21.15.97:0/1785683634 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3ac054a30 con 0x7fd3c015ddd0 2023-11-24T06:09:52.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.281+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3c00b7650 con 0x7fd3c015ddd0 2023-11-24T06:09:52.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.282+0000 7fd3bd7fa700 1 -- 172.21.15.97:0/1785683634 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12869 (secure 0 0 0) 0x7fd3ac058050 con 0x7fd3c015ddd0 2023-11-24T06:09:52.284 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:52.285 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":29,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:49.023015+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3a406aed0 msgr2=0x7fd3a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3a406aed0 0x7fd3a406d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4000f60 tx=0x7fd3b4009040 comp rx=0 tx=0).stop 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015ddd0 msgr2=0x7fd3c00b6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015ddd0 0x7fd3c00b6f70 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00d800 tx=0x7fd3ac00dbc0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 shutdown_connections 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd3a406aed0 0x7fd3a406d390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3c015e1b0 0x7fd3c009f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3c00d2240 0x7fd3c009bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 --2- 172.21.15.97:0/1785683634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3c015ddd0 0x7fd3c00b6f70 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 >> 172.21.15.97:0/1785683634 conn(0x7fd3c00731a0 msgr2=0x7fd3c0070410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 shutdown_connections 2023-11-24T06:09:52.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.284+0000 7fd3c6fc6700 1 -- 172.21.15.97:0/1785683634 wait complete. 2023-11-24T06:09:52.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:09:52.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:09:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.419+0000 7f213ff37700 1 Processor -- start 2023-11-24T06:09:52.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213ff37700 1 -- start start 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213ff37700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f21380c6c30 0x7f21380cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213ff37700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21380cef40 0x7f21380cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213ff37700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21380d0140 con 0x7f213815bf50 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213ff37700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213815c330 con 0x7f21380cef40 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213ff37700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21380cf3d0 con 0x7f21380c6c30 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213e4d4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21380cef40 0x7f21380cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213dcd3700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f213815bf50 0x7f21380ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39284/0 (socket says 172.21.15.97:39284) 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.420+0000 7f213dcd3700 1 -- 172.21.15.97:0/3450000923 learned_addr learned my addr 172.21.15.97:0/3450000923 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213d4d2700 1 --2- 172.21.15.97:0/3450000923 >> v2:172.21.15.183:3300/0 conn(0x7f21380c6c30 0x7f21380cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213e4d4700 1 -- 172.21.15.97:0/3450000923 >> v2:172.21.15.183:3300/0 conn(0x7f21380c6c30 msgr2=0x7f21380cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213e4d4700 1 --2- 172.21.15.97:0/3450000923 >> v2:172.21.15.183:3300/0 conn(0x7f21380c6c30 0x7f21380cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213e4d4700 1 -- 172.21.15.97:0/3450000923 >> v1:172.21.15.97:6789/0 conn(0x7f213815bf50 legacy=0x7f21380ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213e4d4700 1 -- 172.21.15.97:0/3450000923 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21380d03e0 con 0x7f21380cef40 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213ccd1700 1 -- 172.21.15.97:0/3450000923 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618933466 0 0) 0x7f21380d0140 con 0x7f213815bf50 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213e4d4700 1 --2- 172.21.15.97:0/3450000923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380cef40 0x7f21380cc6c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2138072420 tx=0x7f213000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d45d5a5b2851a42 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213ccd1700 1 -- 172.21.15.97:0/3450000923 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213000e070 con 0x7f21380cef40 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213ccd1700 1 -- 172.21.15.97:0/3450000923 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213000bbd0 con 0x7f21380cef40 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.421+0000 7f213ccd1700 1 -- 172.21.15.97:0/3450000923 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130012430 con 0x7f21380cef40 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.422+0000 7f213ff37700 1 -- 172.21.15.97:0/3450000923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380cef40 msgr2=0x7f21380cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.422+0000 7f213ff37700 1 --2- 172.21.15.97:0/3450000923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380cef40 0x7f21380cc6c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2138072420 tx=0x7f213000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.422+0000 7f213ff37700 1 -- 172.21.15.97:0/3450000923 shutdown_connections 2023-11-24T06:09:52.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.422+0000 7f213ff37700 1 --2- 172.21.15.97:0/3450000923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380cef40 0x7f21380cc6c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.422+0000 7f213ff37700 1 --2- 172.21.15.97:0/3450000923 >> v2:172.21.15.183:3300/0 conn(0x7f21380c6c30 0x7f21380cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.422+0000 7f213ff37700 1 -- 172.21.15.97:0/3450000923 >> 172.21.15.97:0/3450000923 conn(0x7f21380b3780 msgr2=0x7f21380b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.422+0000 7f213ff37700 1 -- 172.21.15.97:0/3450000923 shutdown_connections 2023-11-24T06:09:52.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.422+0000 7f213ff37700 1 -- 172.21.15.97:0/3450000923 wait complete. 2023-11-24T06:09:52.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.423+0000 7f213ff37700 1 Processor -- start 2023-11-24T06:09:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.423+0000 7f213ff37700 1 -- start start 2023-11-24T06:09:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213ff37700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380c6c30 0x7f21381f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213ff37700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21380cef40 0x7f21381457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213ff37700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213815bf50 0x7f21381590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213ff37700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21381576a0 con 0x7f21380c6c30 2023-11-24T06:09:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213ff37700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21380ca480 con 0x7f213815bf50 2023-11-24T06:09:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213ff37700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21381f1ab0 con 0x7f21380cef40 2023-11-24T06:09:52.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380c6c30 0x7f21381f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380c6c30 0x7f21381f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46838/0 (socket says 172.21.15.97:46838) 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 -- 172.21.15.97:0/3618334876 learned_addr learned my addr 172.21.15.97:0/3618334876 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213e4d4700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213815bf50 0x7f21381590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213d4d2700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21380cef40 0x7f21381457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 -- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21380cef40 msgr2=0x7f21381457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21380cef40 0x7f21381457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 -- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213815bf50 msgr2=0x7f21381590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213815bf50 0x7f21381590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 -- 172.21.15.97:0/3618334876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130009a70 con 0x7f21380c6c30 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213e4d4700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213815bf50 0x7f21381590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:52.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213dcd3700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380c6c30 0x7f21381f1570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f212800c550 tx=0x7f212800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f212e7fc700 1 -- 172.21.15.97:0/3618334876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128004c80 con 0x7f21380c6c30 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f212e7fc700 1 -- 172.21.15.97:0/3618334876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2128004e00 con 0x7f21380c6c30 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21380ca690 con 0x7f21380c6c30 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f212e7fc700 1 -- 172.21.15.97:0/3618334876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2128007310 con 0x7f21380c6c30 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.424+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21380cac50 con 0x7f21380c6c30 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.425+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21380cb2a0 con 0x7f21380c6c30 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.426+0000 7f212e7fc700 1 -- 172.21.15.97:0/3618334876 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2128007470 con 0x7f21380c6c30 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.426+0000 7f212e7fc700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f211c06aed0 0x7f211c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.426+0000 7f212e7fc700 1 -- 172.21.15.97:0/3618334876 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2128088fa0 con 0x7f21380c6c30 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.426+0000 7f213d4d2700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f211c06aed0 0x7f211c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.426+0000 7f213d4d2700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f211c06aed0 0x7f211c06d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f213806f1b0 tx=0x7f2134008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.430+0000 7f212e7fc700 1 -- 172.21.15.97:0/3618334876 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21280bbe90 con 0x7f21380c6c30 2023-11-24T06:09:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.619+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21380cdd00 con 0x7f21380c6c30 2023-11-24T06:09:52.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.619+0000 7f212e7fc700 1 -- 172.21.15.97:0/3618334876 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21380cdd00 con 0x7f21380c6c30 2023-11-24T06:09:52.622 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:09:52.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f211c06aed0 msgr2=0x7f211c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f211c06aed0 0x7f211c06d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f213806f1b0 tx=0x7f2134008040 comp rx=0 tx=0).stop 2023-11-24T06:09:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380c6c30 msgr2=0x7f21381f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380c6c30 0x7f21381f1570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f212800c550 tx=0x7f212800c910 comp rx=0 tx=0).stop 2023-11-24T06:09:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 shutdown_connections 2023-11-24T06:09:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f213815bf50 0x7f21381590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21380c6c30 0x7f21381f1570 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f211c06aed0 0x7f211c06d390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 --2- 172.21.15.97:0/3618334876 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f21380cef40 0x7f21381457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 >> 172.21.15.97:0/3618334876 conn(0x7f21380b3780 msgr2=0x7f21380ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 shutdown_connections 2023-11-24T06:09:52.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.621+0000 7f213ff37700 1 -- 172.21.15.97:0/3618334876 wait complete. 2023-11-24T06:09:52.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:09:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.754+0000 7fe741ee4700 1 Processor -- start 2023-11-24T06:09:52.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.754+0000 7fe741ee4700 1 -- start start 2023-11-24T06:09:52.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe741ee4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe73c158a50 0x7fe73c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe741ee4700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0cfff0 con 0x7fe73c162a30 2023-11-24T06:09:52.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe741ee4700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0cfd10 con 0x7fe73c158670 2023-11-24T06:09:52.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe741ee4700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c0629b0 con 0x7fe73c158a50 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe73c158a50 0x7fe73c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe73c158a50 0x7fe73c0c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46848/0 (socket says 172.21.15.97:46848) 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73affd700 1 -- 172.21.15.97:0/701484760 learned_addr learned my addr 172.21.15.97:0/701484760 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73affd700 1 -- 172.21.15.97:0/701484760 >> v1:172.21.15.183:6789/0 conn(0x7fe73c158670 legacy=0x7fe73c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73affd700 1 -- 172.21.15.97:0/701484760 >> v1:172.21.15.97:6789/0 conn(0x7fe73c162a30 legacy=0x7fe73c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73affd700 1 -- 172.21.15.97:0/701484760 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c161f50 con 0x7fe73c158a50 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73a7fc700 1 -- 172.21.15.97:0/701484760 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091455101 0 0) 0x7fe73c0cfff0 con 0x7fe73c162a30 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73affd700 1 --2- 172.21.15.97:0/701484760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0c98b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0098f0 tx=0x7fe72c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d108419f25fabbdd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.755+0000 7fe73a7fc700 1 -- 172.21.15.97:0/701484760 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00b8a0 con 0x7fe73c158a50 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe73a7fc700 1 -- 172.21.15.97:0/701484760 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c00ba00 con 0x7fe73c158a50 2023-11-24T06:09:52.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe73a7fc700 1 -- 172.21.15.97:0/701484760 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c00bd90 con 0x7fe73c158a50 2023-11-24T06:09:52.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe741ee4700 1 -- 172.21.15.97:0/701484760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 msgr2=0x7fe73c0c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe741ee4700 1 --2- 172.21.15.97:0/701484760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0c98b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0098f0 tx=0x7fe72c009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:52.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe741ee4700 1 -- 172.21.15.97:0/701484760 shutdown_connections 2023-11-24T06:09:52.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe741ee4700 1 --2- 172.21.15.97:0/701484760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0c98b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe741ee4700 1 -- 172.21.15.97:0/701484760 >> 172.21.15.97:0/701484760 conn(0x7fe73c0b57b0 msgr2=0x7fe73c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:52.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe741ee4700 1 -- 172.21.15.97:0/701484760 shutdown_connections 2023-11-24T06:09:52.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.756+0000 7fe741ee4700 1 -- 172.21.15.97:0/701484760 wait complete. 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.757+0000 7fe741ee4700 1 Processor -- start 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.757+0000 7fe741ee4700 1 -- start start 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe741ee4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe73c158670 0x7fe73c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe741ee4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe741ee4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe73c162a30 0x7fe73c161590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe741ee4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0b54a0 con 0x7fe73c158a50 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe741ee4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c1607f0 con 0x7fe73c162a30 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe741ee4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c160ab0 con 0x7fe73c158670 2023-11-24T06:09:52.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe73affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe73affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46854/0 (socket says 172.21.15.97:46854) 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.758+0000 7fe73affd700 1 -- 172.21.15.97:0/3304230738 learned_addr learned my addr 172.21.15.97:0/3304230738 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe73affd700 1 -- 172.21.15.97:0/3304230738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe73c158670 msgr2=0x7fe73c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe73affd700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe73c158670 0x7fe73c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe73affd700 1 -- 172.21.15.97:0/3304230738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe73c162a30 msgr2=0x7fe73c161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe73affd700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe73c162a30 0x7fe73c161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe73affd700 1 -- 172.21.15.97:0/3304230738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c009550 con 0x7fe73c158a50 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe73affd700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0b4f60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe72c013000 tx=0x7fe72c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe723fff700 1 -- 172.21.15.97:0/3304230738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c011a70 con 0x7fe73c158a50 2023-11-24T06:09:52.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe723fff700 1 -- 172.21.15.97:0/3304230738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c01f070 con 0x7fe73c158a50 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c160d90 con 0x7fe73c158a50 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe723fff700 1 -- 172.21.15.97:0/3304230738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c011dd0 con 0x7fe73c158a50 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.759+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c161ad0 con 0x7fe73c158a50 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.760+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c067f60 con 0x7fe73c158a50 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.760+0000 7fe723fff700 1 -- 172.21.15.97:0/3304230738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe72c01a430 con 0x7fe73c158a50 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.761+0000 7fe723fff700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe72806aed0 0x7fe72806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.761+0000 7fe723fff700 1 -- 172.21.15.97:0/3304230738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe72c021070 con 0x7fe73c158a50 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.761+0000 7fe73b7fe700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe72806aed0 0x7fe72806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:52.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.761+0000 7fe73b7fe700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe72806aed0 0x7fe72806d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe730000f60 tx=0x7fe730007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:52.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.765+0000 7fe723fff700 1 -- 172.21.15.97:0/3304230738 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe72c056190 con 0x7fe73c158a50 2023-11-24T06:09:52.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.958+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe73c0b0680 con 0x7fe73c158a50 2023-11-24T06:09:52.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.958+0000 7fe723fff700 1 -- 172.21.15.97:0/3304230738 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe72c015090 con 0x7fe73c158a50 2023-11-24T06:09:52.961 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:52.962 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:09:52.962 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:09:52.962 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-11-24T06:09:52.962 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-11-24T06:09:52.962 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:09:52.962 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:09:52.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe72806aed0 msgr2=0x7fe72806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe72806aed0 0x7fe72806d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe730000f60 tx=0x7fe730007a40 comp rx=0 tx=0).stop 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 msgr2=0x7fe73c0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0b4f60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe72c013000 tx=0x7fe72c004750 comp rx=0 tx=0).stop 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 shutdown_connections 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe73c162a30 0x7fe73c161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe72806aed0 0x7fe72806d390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe73c158670 0x7fe73c15b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 --2- 172.21.15.97:0/3304230738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe73c158a50 0x7fe73c0b4f60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:52.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 >> 172.21.15.97:0/3304230738 conn(0x7fe73c0b57b0 msgr2=0x7fe73c0c98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:52.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 shutdown_connections 2023-11-24T06:09:52.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:52.960+0000 7fe741ee4700 1 -- 172.21.15.97:0/3304230738 wait complete. 2023-11-24T06:09:52.975 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:09:52.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:09:53.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.097+0000 7fe314ba9700 1 Processor -- start 2023-11-24T06:09:53.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.097+0000 7fe314ba9700 1 -- start start 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.098+0000 7fe314ba9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe3100b5b60 0x7fe31006e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.098+0000 7fe314ba9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3101624c0 con 0x7fe31015e0b0 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.098+0000 7fe314ba9700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe310161b90 con 0x7fe31006c710 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.098+0000 7fe314ba9700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31015e490 con 0x7fe3100b5b60 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30e59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe31015e0b0 0x7fe31006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39312/0 (socket says 172.21.15.97:39312) 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30e59c700 1 -- 172.21.15.97:0/1362684118 learned_addr learned my addr 172.21.15.97:0/1362684118 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30ed9d700 1 --2- 172.21.15.97:0/1362684118 >> v2:172.21.15.183:3300/0 conn(0x7fe3100b5b60 0x7fe31006e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278274468 0 0) 0x7fe3101624c0 con 0x7fe31015e0b0 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f80035b0 con 0x7fe31015e0b0 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3058050635 0 0) 0x7fe2f80035b0 con 0x7fe31015e0b0 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 >> v2:172.21.15.183:3300/0 conn(0x7fe3100b5b60 msgr2=0x7fe31006e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:53.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 --2- 172.21.15.97:0/1362684118 >> v2:172.21.15.183:3300/0 conn(0x7fe3100b5b60 0x7fe31006e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 >> v1:172.21.15.183:6789/0 conn(0x7fe31006c710 legacy=0x7fe31006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310161e70 con 0x7fe31015e0b0 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fe304001670 con 0x7fe31015e0b0 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 >> v1:172.21.15.97:6789/0 conn(0x7fe31015e0b0 legacy=0x7fe31006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2f8003a00 0x7fe2f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2f8004850 0x7fe2f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f8005210 0x7fe2f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31015e490 con 0x7fe2f8005210 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f800c1b0 con 0x7fe2f8004850 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f800c370 con 0x7fe2f8003a00 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30ed9d700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f8005210 0x7fe2f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30e59c700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2f8003a00 0x7fe2f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:53.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.099+0000 7fe30dd9b700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2f8004850 0x7fe2f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30ed9d700 1 -- 172.21.15.97:0/1362684118 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2f8003a00 msgr2=0x7fe2f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30ed9d700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2f8003a00 0x7fe2f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30ed9d700 1 -- 172.21.15.97:0/1362684118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2f8004850 msgr2=0x7fe2f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30ed9d700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2f8004850 0x7fe2f8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30ed9d700 1 -- 172.21.15.97:0/1362684118 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f80035b0 con 0x7fe2f8005210 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30ed9d700 1 -- 172.21.15.97:0/1362684118 reap_dead start 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30ed9d700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f8005210 0x7fe2f800ba10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe308009f90 tx=0x7fe2f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a234f297fb749b83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308003a20 con 0x7fe2f8005210 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30800f040 con 0x7fe2f8005210 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe30d59a700 1 -- 172.21.15.97:0/1362684118 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308003de0 con 0x7fe2f8005210 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe314ba9700 1 -- 172.21.15.97:0/1362684118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f8005210 msgr2=0x7fe2f800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe314ba9700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f8005210 0x7fe2f800ba10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe308009f90 tx=0x7fe2f8005880 comp rx=0 tx=0).stop 2023-11-24T06:09:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe314ba9700 1 -- 172.21.15.97:0/1362684118 shutdown_connections 2023-11-24T06:09:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe314ba9700 1 --2- 172.21.15.97:0/1362684118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2f8005210 0x7fe2f800ba10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.100+0000 7fe314ba9700 1 -- 172.21.15.97:0/1362684118 >> 172.21.15.97:0/1362684118 conn(0x7fe310071b50 msgr2=0x7fe310071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.101+0000 7fe314ba9700 1 -- 172.21.15.97:0/1362684118 shutdown_connections 2023-11-24T06:09:53.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.101+0000 7fe314ba9700 1 -- 172.21.15.97:0/1362684118 wait complete. 2023-11-24T06:09:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.102+0000 7fe314ba9700 1 Processor -- start 2023-11-24T06:09:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.102+0000 7fe314ba9700 1 -- start start 2023-11-24T06:09:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe314ba9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2f8005210 0x7fe3101f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe314ba9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe310160fd0 0x7fe3100b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe314ba9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3101440a0 0x7fe310144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe314ba9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe310144a90 con 0x7fe310160fd0 2023-11-24T06:09:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe314ba9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3101f73e0 con 0x7fe2f8005210 2023-11-24T06:09:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe314ba9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31015e990 con 0x7fe3101440a0 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe30dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe310160fd0 0x7fe3100b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe30dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe310160fd0 0x7fe3100b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46874/0 (socket says 172.21.15.97:46874) 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe30dd9b700 1 -- 172.21.15.97:0/1020895386 learned_addr learned my addr 172.21.15.97:0/1020895386 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe30dd9b700 1 -- 172.21.15.97:0/1020895386 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3101440a0 msgr2=0x7fe310144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe30dd9b700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3101440a0 0x7fe310144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe30dd9b700 1 -- 172.21.15.97:0/1020895386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2f8005210 msgr2=0x7fe3101f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe30dd9b700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2f8005210 0x7fe3101f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.103+0000 7fe30dd9b700 1 -- 172.21.15.97:0/1020895386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3101f7680 con 0x7fe310160fd0 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.104+0000 7fe30dd9b700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe310160fd0 0x7fe3100b11b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe30000f610 tx=0x7fe30000f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.104+0000 7fe2feffd700 1 -- 172.21.15.97:0/1020895386 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000052e0 con 0x7fe310160fd0 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.104+0000 7fe2feffd700 1 -- 172.21.15.97:0/1020895386 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe300005440 con 0x7fe310160fd0 2023-11-24T06:09:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.104+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3101f7960 con 0x7fe310160fd0 2023-11-24T06:09:53.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.104+0000 7fe2feffd700 1 -- 172.21.15.97:0/1020895386 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3000038a0 con 0x7fe310160fd0 2023-11-24T06:09:53.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.104+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31014e650 con 0x7fe310160fd0 2023-11-24T06:09:53.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.105+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3100624c0 con 0x7fe310160fd0 2023-11-24T06:09:53.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.105+0000 7fe2feffd700 1 -- 172.21.15.97:0/1020895386 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe300003a00 con 0x7fe310160fd0 2023-11-24T06:09:53.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.105+0000 7fe2feffd700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe2f006aed0 0x7fe2f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:53.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.106+0000 7fe30e59c700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe2f006aed0 0x7fe2f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:53.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.106+0000 7fe2feffd700 1 -- 172.21.15.97:0/1020895386 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe300088f80 con 0x7fe310160fd0 2023-11-24T06:09:53.111 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.106+0000 7fe30e59c700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe2f006aed0 0x7fe2f006d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe3040011e0 tx=0x7fe304001170 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:53.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.110+0000 7fe2feffd700 1 -- 172.21.15.97:0/1020895386 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3000bbe90 con 0x7fe310160fd0 2023-11-24T06:09:53.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:53.300+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe310067f60 con 0x7fe310160fd0 2023-11-24T06:09:54.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.021+0000 7fe2feffd700 1 -- 172.21.15.97:0/1020895386 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe300054410 con 0x7fe310160fd0 2023-11-24T06:09:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe2f006aed0 msgr2=0x7fe2f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe2f006aed0 0x7fe2f006d390 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe3040011e0 tx=0x7fe304001170 comp rx=0 tx=0).stop 2023-11-24T06:09:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe310160fd0 msgr2=0x7fe3100b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:54.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe310160fd0 0x7fe3100b11b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe30000f610 tx=0x7fe30000f9d0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 shutdown_connections 2023-11-24T06:09:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe2f006aed0 0x7fe2f006d390 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3101440a0 0x7fe310144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2f8005210 0x7fe3101f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 --2- 172.21.15.97:0/1020895386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe310160fd0 0x7fe3100b11b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.024+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 >> 172.21.15.97:0/1020895386 conn(0x7fe310071b50 msgr2=0x7fe31006bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.025+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 shutdown_connections 2023-11-24T06:09:54.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.025+0000 7fe314ba9700 1 -- 172.21.15.97:0/1020895386 wait complete. 2023-11-24T06:09:54.029 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-11-24T06:09:54.045 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:09:54.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:54.092 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:09:54.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.179+0000 7f25b386a700 1 Processor -- start 2023-11-24T06:09:54.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.179+0000 7f25b386a700 1 -- start start 2023-11-24T06:09:54.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.180+0000 7f25b386a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f25ac15bf50 0x7f25ac0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:54.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.180+0000 7f25b386a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0d0140 con 0x7f25ac0ce830 2023-11-24T06:09:54.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.180+0000 7f25b386a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0cf3c0 con 0x7f25ac0c6c30 2023-11-24T06:09:54.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.180+0000 7f25b386a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac15c330 con 0x7f25ac15bf50 2023-11-24T06:09:54.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.180+0000 7f25b0e05700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f25ac0ce830 0x7f25ac0cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39334/0 (socket says 172.21.15.97:39334) 2023-11-24T06:09:54.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25b0e05700 1 -- 172.21.15.97:0/1428767119 learned_addr learned my addr 172.21.15.97:0/1428767119 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81378107 0 0) 0x7f25ac0d0140 con 0x7f25ac0ce830 2023-11-24T06:09:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25980035b0 con 0x7f25ac0ce830 2023-11-24T06:09:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25b1606700 1 --2- 172.21.15.97:0/1428767119 >> v2:172.21.15.183:3300/0 conn(0x7f25ac15bf50 0x7f25ac0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1905937203 0 0) 0x7f25980035b0 con 0x7f25ac0ce830 2023-11-24T06:09:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 >> v2:172.21.15.183:3300/0 conn(0x7f25ac15bf50 msgr2=0x7f25ac0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:54.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 --2- 172.21.15.97:0/1428767119 >> v2:172.21.15.183:3300/0 conn(0x7f25ac15bf50 0x7f25ac0ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 >> v1:172.21.15.183:6789/0 conn(0x7f25ac0c6c30 legacy=0x7f25ac0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0d03e0 con 0x7f25ac0ce830 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f25a8001670 con 0x7f25ac0ce830 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 >> v1:172.21.15.97:6789/0 conn(0x7f25ac0ce830 legacy=0x7f25ac0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2598003a00 0x7f2598003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2598004850 0x7f2598004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598005210 0x7f259800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac15c330 con 0x7f2598003a00 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259800c160 con 0x7f2598004850 2023-11-24T06:09:54.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259800c320 con 0x7f2598005210 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25b1606700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2598003a00 0x7f2598003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25b0e05700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2598004850 0x7f2598004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25b1e07700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598005210 0x7f259800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.181+0000 7f25b1606700 1 -- 172.21.15.97:0/1428767119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598005210 msgr2=0x7f259800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b1606700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598005210 0x7f259800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b1606700 1 -- 172.21.15.97:0/1428767119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2598004850 msgr2=0x7f2598004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b1606700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2598004850 0x7f2598004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b1606700 1 -- 172.21.15.97:0/1428767119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980035b0 con 0x7f2598003a00 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b1e07700 1 -- 172.21.15.97:0/1428767119 reap_dead start 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b1606700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2598003a00 0x7f2598003e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f259c009890 tx=0x7f2598005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209c0f0248cb0153 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c00c070 con 0x7f2598003a00 2023-11-24T06:09:54.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c013610 con 0x7f2598003a00 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25a3fff700 1 -- 172.21.15.97:0/1428767119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0139c0 con 0x7f2598003a00 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b386a700 1 -- 172.21.15.97:0/1428767119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2598003a00 msgr2=0x7f2598003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b386a700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2598003a00 0x7f2598003e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f259c009890 tx=0x7f2598005880 comp rx=0 tx=0).stop 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b386a700 1 -- 172.21.15.97:0/1428767119 shutdown_connections 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b386a700 1 --2- 172.21.15.97:0/1428767119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2598003a00 0x7f2598003e20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.182+0000 7f25b386a700 1 -- 172.21.15.97:0/1428767119 >> 172.21.15.97:0/1428767119 conn(0x7f25ac0b3780 msgr2=0x7f25ac0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.183+0000 7f25b386a700 1 -- 172.21.15.97:0/1428767119 shutdown_connections 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.183+0000 7f25b386a700 1 -- 172.21.15.97:0/1428767119 wait complete. 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.184+0000 7f25b386a700 1 Processor -- start 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.184+0000 7f25b386a700 1 -- start start 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b386a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598003a00 0x7f25ac0b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b386a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac15a450 0x7f25ac0b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:54.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b386a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ac149f60 0x7f25ac14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b386a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0b5470 con 0x7f25ac149f60 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b386a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0b6290 con 0x7f25ac15a450 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b386a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0b7870 con 0x7f2598003a00 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1e07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ac149f60 0x7f25ac14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1e07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ac149f60 0x7f25ac14a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46896/0 (socket says 172.21.15.97:46896) 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1e07700 1 -- 172.21.15.97:0/2080911545 learned_addr learned my addr 172.21.15.97:0/2080911545 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b0e05700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac15a450 0x7f25ac0b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1606700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598003a00 0x7f25ac0b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1e07700 1 -- 172.21.15.97:0/2080911545 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598003a00 msgr2=0x7f25ac0b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1e07700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598003a00 0x7f25ac0b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1e07700 1 -- 172.21.15.97:0/2080911545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac15a450 msgr2=0x7f25ac0b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:54.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1e07700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac15a450 0x7f25ac0b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.185+0000 7f25b1e07700 1 -- 172.21.15.97:0/2080911545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0b6c60 con 0x7f25ac149f60 2023-11-24T06:09:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.186+0000 7f25b1e07700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ac149f60 0x7f25ac14a7f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0c70c0 tx=0x7f25a400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.186+0000 7f25a1ffb700 1 -- 172.21.15.97:0/2080911545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4002090 con 0x7f25ac149f60 2023-11-24T06:09:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.186+0000 7f25a1ffb700 1 -- 172.21.15.97:0/2080911545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a4003c10 con 0x7f25ac149f60 2023-11-24T06:09:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.186+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac0b5680 con 0x7f25ac149f60 2023-11-24T06:09:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.186+0000 7f25a1ffb700 1 -- 172.21.15.97:0/2080911545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a400d930 con 0x7f25ac149f60 2023-11-24T06:09:54.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.186+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac0b5c40 con 0x7f25ac149f60 2023-11-24T06:09:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.187+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ac068fb0 con 0x7f25ac149f60 2023-11-24T06:09:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.187+0000 7f25a1ffb700 1 -- 172.21.15.97:0/2080911545 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f25a400da90 con 0x7f25ac149f60 2023-11-24T06:09:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.188+0000 7f25a1ffb700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f259006af00 0x7f259006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.188+0000 7f25a1ffb700 1 -- 172.21.15.97:0/2080911545 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f25a4089580 con 0x7f25ac149f60 2023-11-24T06:09:54.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.188+0000 7f25b1606700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f259006af00 0x7f259006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.188+0000 7f25b1606700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f259006af00 0x7f259006d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f259c016040 tx=0x7f259c01e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:54.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.192+0000 7f25a1ffb700 1 -- 172.21.15.97:0/2080911545 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25a4054780 con 0x7f25ac149f60 2023-11-24T06:09:54.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:54.386+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f25ac0b63d0 con 0x7f25ac149f60 2023-11-24T06:09:54.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-24T06:09:54.933 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-24T06:09:55.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.028+0000 7f25a1ffb700 1 -- 172.21.15.97:0/2080911545 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f25a4057da0 con 0x7f25ac149f60 2023-11-24T06:09:55.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.030+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f259006af00 msgr2=0x7f259006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.030+0000 7f25b386a700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f259006af00 0x7f259006d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f259c016040 tx=0x7f259c01e040 comp rx=0 tx=0).stop 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.030+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ac149f60 msgr2=0x7f25ac14a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.030+0000 7f25b386a700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ac149f60 0x7f25ac14a7f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0c70c0 tx=0x7f25a400d430 comp rx=0 tx=0).stop 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.031+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 shutdown_connections 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.031+0000 7f25b386a700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25ac149f60 0x7f25ac14a7f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.031+0000 7f25b386a700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f259006af00 0x7f259006d3c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.031+0000 7f25b386a700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2598003a00 0x7f25ac0b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.031+0000 7f25b386a700 1 --2- 172.21.15.97:0/2080911545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25ac15a450 0x7f25ac0b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.031+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 >> 172.21.15.97:0/2080911545 conn(0x7f25ac0b3780 msgr2=0x7f25ac0cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.031+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 shutdown_connections 2023-11-24T06:09:55.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.031+0000 7f25b386a700 1 -- 172.21.15.97:0/2080911545 wait complete. 2023-11-24T06:09:55.035 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-11-24T06:09:55.058 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:09:55.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.218+0000 7fb61663f700 1 Processor -- start 2023-11-24T06:09:55.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.218+0000 7fb61663f700 1 -- start start 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.219+0000 7fb61663f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb6100b5da0 0x7fb6100afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.219+0000 7fb61663f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6100b3c60 con 0x7fb6100746a0 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.219+0000 7fb61663f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6100b58c0 con 0x7fb61015e160 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.219+0000 7fb61663f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100b4500 con 0x7fb6100b5da0 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.219+0000 7fb60f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb6100746a0 0x7fb61006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39354/0 (socket says 172.21.15.97:39354) 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.219+0000 7fb60f7fe700 1 -- 172.21.15.97:0/2336733143 learned_addr learned my addr 172.21.15.97:0/2336733143 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb614bdc700 1 --2- 172.21.15.97:0/2336733143 >> v2:172.21.15.183:3300/0 conn(0x7fb6100b5da0 0x7fb6100afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326014609 0 0) 0x7fb6100b3c60 con 0x7fb6100746a0 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5f80035b0 con 0x7fb6100746a0 2023-11-24T06:09:55.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3968000095 0 0) 0x7fb5f80035b0 con 0x7fb6100746a0 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 >> v2:172.21.15.183:3300/0 conn(0x7fb6100b5da0 msgr2=0x7fb6100afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 --2- 172.21.15.97:0/2336733143 >> v2:172.21.15.183:3300/0 conn(0x7fb6100b5da0 0x7fb6100afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 >> v1:172.21.15.183:6789/0 conn(0x7fb61015e160 legacy=0x7fb610073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61006bdc0 con 0x7fb6100746a0 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fb600001670 con 0x7fb6100746a0 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 >> v1:172.21.15.97:6789/0 conn(0x7fb6100746a0 legacy=0x7fb61006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8003a00 0x7fb5f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5f8004850 0x7fb5f8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8005210 0x7fb5f800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100b4500 con 0x7fb5f8003a00 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f800c0b0 con 0x7fb5f8005210 2023-11-24T06:09:55.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f800c270 con 0x7fb5f8004850 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60ffff700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8003a00 0x7fb5f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb614bdc700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8005210 0x7fb5f800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60f7fe700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5f8004850 0x7fb5f8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60ffff700 1 -- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5f8004850 msgr2=0x7fb5f8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60ffff700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb5f8004850 0x7fb5f8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60ffff700 1 -- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8005210 msgr2=0x7fb5f800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60ffff700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8005210 0x7fb5f800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb60ffff700 1 -- 172.21.15.97:0/2336733143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80035b0 con 0x7fb5f8003a00 2023-11-24T06:09:55.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb614bdc700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8005210 0x7fb5f800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.220+0000 7fb614bdc700 1 -- 172.21.15.97:0/2336733143 reap_dead start 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb60ffff700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8003a00 0x7fb5f8003e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb6040017c0 tx=0x7fb5f8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2896585eab7f10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6040052d0 con 0x7fb5f8003a00 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604005430 con 0x7fb5f8003a00 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb60effd700 1 -- 172.21.15.97:0/2336733143 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6040038c0 con 0x7fb5f8003a00 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb61663f700 1 -- 172.21.15.97:0/2336733143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8003a00 msgr2=0x7fb5f8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb61663f700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8003a00 0x7fb5f8003e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb6040017c0 tx=0x7fb5f8005880 comp rx=0 tx=0).stop 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb61663f700 1 -- 172.21.15.97:0/2336733143 shutdown_connections 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb61663f700 1 --2- 172.21.15.97:0/2336733143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8003a00 0x7fb5f8003e20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb61663f700 1 -- 172.21.15.97:0/2336733143 >> 172.21.15.97:0/2336733143 conn(0x7fb6100d0bf0 msgr2=0x7fb6100d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb61663f700 1 -- 172.21.15.97:0/2336733143 shutdown_connections 2023-11-24T06:09:55.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.221+0000 7fb61663f700 1 -- 172.21.15.97:0/2336733143 wait complete. 2023-11-24T06:09:55.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.223+0000 7fb61663f700 1 Processor -- start 2023-11-24T06:09:55.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.223+0000 7fb61663f700 1 -- start start 2023-11-24T06:09:55.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb61663f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8003a00 0x7fb6101f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:55.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb61663f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6101446b0 0x7fb6101f3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:55.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb61663f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6101f3ff0 0x7fb6100b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb61663f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101f70e0 con 0x7fb6101446b0 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb61663f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb610144b30 con 0x7fb5f8003a00 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb61663f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101f4600 con 0x7fb6101f3ff0 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6101446b0 0x7fb6101f3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6101446b0 0x7fb6101f3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46914/0 (socket says 172.21.15.97:46914) 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60f7fe700 1 -- 172.21.15.97:0/1764916183 learned_addr learned my addr 172.21.15.97:0/1764916183 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60ffff700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8003a00 0x7fb6101f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb614bdc700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6101f3ff0 0x7fb6100b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60f7fe700 1 -- 172.21.15.97:0/1764916183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6101f3ff0 msgr2=0x7fb6100b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60f7fe700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6101f3ff0 0x7fb6100b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60f7fe700 1 -- 172.21.15.97:0/1764916183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8003a00 msgr2=0x7fb6101f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:55.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60f7fe700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8003a00 0x7fb6101f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.224+0000 7fb60f7fe700 1 -- 172.21.15.97:0/1764916183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6101f48e0 con 0x7fb6101446b0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.225+0000 7fb60f7fe700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6101446b0 0x7fb6101f3ab0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb60000e800 tx=0x7fb60000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.225+0000 7fb60cff9700 1 -- 172.21.15.97:0/1764916183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600012590 con 0x7fb6101446b0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.225+0000 7fb60cff9700 1 -- 172.21.15.97:0/1764916183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb600012710 con 0x7fb6101446b0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.225+0000 7fb60cff9700 1 -- 172.21.15.97:0/1764916183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600012ac0 con 0x7fb6101446b0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.225+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6101f7380 con 0x7fb6101446b0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.225+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6101613c0 con 0x7fb6101446b0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.226+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6101f43d0 con 0x7fb6101446b0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.226+0000 7fb60cff9700 1 -- 172.21.15.97:0/1764916183 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb600012c20 con 0x7fb6101446b0 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.227+0000 7fb60cff9700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5fc06aed0 0x7fb5fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:55.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.227+0000 7fb60cff9700 1 -- 172.21.15.97:0/1764916183 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb600089150 con 0x7fb6101446b0 2023-11-24T06:09:55.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.227+0000 7fb60ffff700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5fc06aed0 0x7fb5fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:55.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.227+0000 7fb60ffff700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5fc06aed0 0x7fb5fc06d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb604000c00 tx=0x7fb604009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:55.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.231+0000 7fb60cff9700 1 -- 172.21.15.97:0/1764916183 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb600057d80 con 0x7fb6101446b0 2023-11-24T06:09:55.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:55.466+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb6101f4a70 con 0x7fb6101446b0 2023-11-24T06:09:56.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.029+0000 7fb60cff9700 1 -- 172.21.15.97:0/1764916183 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb600012ed0 con 0x7fb6101446b0 2023-11-24T06:09:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5fc06aed0 msgr2=0x7fb5fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5fc06aed0 0x7fb5fc06d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb604000c00 tx=0x7fb604009030 comp rx=0 tx=0).stop 2023-11-24T06:09:56.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6101446b0 msgr2=0x7fb6101f3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6101446b0 0x7fb6101f3ab0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb60000e800 tx=0x7fb60000eb10 comp rx=0 tx=0).stop 2023-11-24T06:09:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 shutdown_connections 2023-11-24T06:09:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb5fc06aed0 0x7fb5fc06d390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6101f3ff0 0x7fb6100b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8003a00 0x7fb6101f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 --2- 172.21.15.97:0/1764916183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6101446b0 0x7fb6101f3ab0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 >> 172.21.15.97:0/1764916183 conn(0x7fb6100d0bf0 msgr2=0x7fb610162510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:56.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 shutdown_connections 2023-11-24T06:09:56.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.034+0000 7fb61663f700 1 -- 172.21.15.97:0/1764916183 wait complete. 2023-11-24T06:09:56.049 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T06:09:56.065 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:56.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.183+0000 7faa6cd67700 1 Processor -- start 2023-11-24T06:09:56.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.183+0000 7faa6cd67700 1 -- start start 2023-11-24T06:09:56.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.184+0000 7faa6cd67700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa681446a0 0x7faa680b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.184+0000 7faa6cd67700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa6806c710 0x7faa680b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.184+0000 7faa6cd67700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa680af950 con 0x7faa680adf00 2023-11-24T06:09:56.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.184+0000 7faa6cd67700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa68144a80 con 0x7faa6806c710 2023-11-24T06:09:56.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.184+0000 7faa6cd67700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa680afc30 con 0x7faa681446a0 2023-11-24T06:09:56.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa66d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa6806c710 0x7faa680b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa65d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faa680adf00 0x7faa6806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39386/0 (socket says 172.21.15.97:39386) 2023-11-24T06:09:56.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa65d9b700 1 -- 172.21.15.97:0/3764898977 learned_addr learned my addr 172.21.15.97:0/3764898977 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:56.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa6659c700 1 --2- 172.21.15.97:0/3764898977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa681446a0 0x7faa680b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa66d9d700 1 -- 172.21.15.97:0/3764898977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa681446a0 msgr2=0x7faa680b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:56.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa66d9d700 1 --2- 172.21.15.97:0/3764898977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa681446a0 0x7faa680b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa66d9d700 1 -- 172.21.15.97:0/3764898977 >> v1:172.21.15.97:6789/0 conn(0x7faa680adf00 legacy=0x7faa6806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa6559a700 1 -- 172.21.15.97:0/3764898977 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954181254 0 0) 0x7faa680af950 con 0x7faa680adf00 2023-11-24T06:09:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa66d9d700 1 -- 172.21.15.97:0/3764898977 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680aff10 con 0x7faa6806c710 2023-11-24T06:09:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa66d9d700 1 --2- 172.21.15.97:0/3764898977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6806c710 0x7faa680b3d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa60009d90 tx=0x7faa6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af420fb97728816 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa6559a700 1 -- 172.21.15.97:0/3764898977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6000e070 con 0x7faa6806c710 2023-11-24T06:09:56.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa6559a700 1 -- 172.21.15.97:0/3764898977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6000bb20 con 0x7faa6806c710 2023-11-24T06:09:56.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.185+0000 7faa6559a700 1 -- 172.21.15.97:0/3764898977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60012430 con 0x7faa6806c710 2023-11-24T06:09:56.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.186+0000 7faa6cd67700 1 -- 172.21.15.97:0/3764898977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6806c710 msgr2=0x7faa680b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.186+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3764898977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6806c710 0x7faa680b3d40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faa60009d90 tx=0x7faa6000b330 comp rx=0 tx=0).stop 2023-11-24T06:09:56.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.186+0000 7faa6cd67700 1 -- 172.21.15.97:0/3764898977 shutdown_connections 2023-11-24T06:09:56.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.186+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3764898977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa6806c710 0x7faa680b3d40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.186+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3764898977 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa681446a0 0x7faa680b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.186+0000 7faa6cd67700 1 -- 172.21.15.97:0/3764898977 >> 172.21.15.97:0/3764898977 conn(0x7faa68071b90 msgr2=0x7faa68071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:56.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.186+0000 7faa6cd67700 1 -- 172.21.15.97:0/3764898977 shutdown_connections 2023-11-24T06:09:56.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.186+0000 7faa6cd67700 1 -- 172.21.15.97:0/3764898977 wait complete. 2023-11-24T06:09:56.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.187+0000 7faa6cd67700 1 Processor -- start 2023-11-24T06:09:56.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.187+0000 7faa6cd67700 1 -- start start 2023-11-24T06:09:56.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa6cd67700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa6806c710 0x7faa6815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa6cd67700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa680adf00 0x7faa68141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa6cd67700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa681446a0 0x7faa681421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa6cd67700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6815ac50 con 0x7faa681446a0 2023-11-24T06:09:56.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa6cd67700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6815ca60 con 0x7faa680adf00 2023-11-24T06:09:56.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa6cd67700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6815cd40 con 0x7faa6806c710 2023-11-24T06:09:56.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa681446a0 0x7faa681421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa681446a0 0x7faa681421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46942/0 (socket says 172.21.15.97:46942) 2023-11-24T06:09:56.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 -- 172.21.15.97:0/3227653780 learned_addr learned my addr 172.21.15.97:0/3227653780 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:56.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa65d9b700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa680adf00 0x7faa68141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa6659c700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa6806c710 0x7faa6815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 -- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa6806c710 msgr2=0x7faa6815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa6806c710 0x7faa6815db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 -- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa680adf00 msgr2=0x7faa68141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa680adf00 0x7faa68141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 -- 172.21.15.97:0/3227653780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa60009a70 con 0x7faa681446a0 2023-11-24T06:09:56.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa65d9b700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa680adf00 0x7faa68141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:56.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.188+0000 7faa66d9d700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa681446a0 0x7faa681421d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faa680729d0 tx=0x7faa60004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.189+0000 7faa56ffd700 1 -- 172.21.15.97:0/3227653780 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60012430 con 0x7faa681446a0 2023-11-24T06:09:56.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.189+0000 7faa56ffd700 1 -- 172.21.15.97:0/3227653780 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa60020070 con 0x7faa681446a0 2023-11-24T06:09:56.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.189+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6815d020 con 0x7faa681446a0 2023-11-24T06:09:56.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.189+0000 7faa56ffd700 1 -- 172.21.15.97:0/3227653780 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa60004280 con 0x7faa681446a0 2023-11-24T06:09:56.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.189+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6815d5e0 con 0x7faa681446a0 2023-11-24T06:09:56.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.190+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6815d7a0 con 0x7faa681446a0 2023-11-24T06:09:56.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.190+0000 7faa56ffd700 1 -- 172.21.15.97:0/3227653780 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa600043e0 con 0x7faa681446a0 2023-11-24T06:09:56.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.190+0000 7faa56ffd700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faa4806aed0 0x7faa4806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.191+0000 7faa56ffd700 1 -- 172.21.15.97:0/3227653780 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa60022070 con 0x7faa681446a0 2023-11-24T06:09:56.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.191+0000 7faa6659c700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faa4806aed0 0x7faa4806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.191+0000 7faa6659c700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faa4806aed0 0x7faa4806d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009a10 tx=0x7faa5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.195+0000 7faa56ffd700 1 -- 172.21.15.97:0/3227653780 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa60056620 con 0x7faa681446a0 2023-11-24T06:09:56.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.386+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa680753f0 con 0x7faa681446a0 2023-11-24T06:09:56.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.387+0000 7faa56ffd700 1 -- 172.21.15.97:0/3227653780 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16594 (secure 0 0 0) 0x7faa60060070 con 0x7faa681446a0 2023-11-24T06:09:56.390 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:56.390 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":32,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:55.467660+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:09:53.319367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:09:54.404860+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faa4806aed0 msgr2=0x7faa4806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faa4806aed0 0x7faa4806d390 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009a10 tx=0x7faa5c008040 comp rx=0 tx=0).stop 2023-11-24T06:09:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa681446a0 msgr2=0x7faa681421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa681446a0 0x7faa681421d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faa680729d0 tx=0x7faa60004ad0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 shutdown_connections 2023-11-24T06:09:56.393 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa681446a0 0x7faa681421d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faa4806aed0 0x7faa4806d390 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faa6806c710 0x7faa6815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 --2- 172.21.15.97:0/3227653780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa680adf00 0x7faa68141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.389+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 >> 172.21.15.97:0/3227653780 conn(0x7faa68071b90 msgr2=0x7faa680b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.390+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 shutdown_connections 2023-11-24T06:09:56.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.390+0000 7faa6cd67700 1 -- 172.21.15.97:0/3227653780 wait complete. 2023-11-24T06:09:56.405 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:56.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.521+0000 7f102112d700 1 Processor -- start 2023-11-24T06:09:56.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.521+0000 7f102112d700 1 -- start start 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f102112d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c15bdd0 0x7f101c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f102112d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f101c0b3370 0x7f101c0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f102112d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c0b2030 con 0x7f101c0d1c70 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f102112d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101c0b2f50 con 0x7f101c0b3370 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f102112d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0b22d0 con 0x7f101c15bdd0 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f101b59e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f101c0d1c70 0x7f101c0ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39404/0 (socket says 172.21.15.97:39404) 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f101b59e700 1 -- 172.21.15.97:0/429403461 learned_addr learned my addr 172.21.15.97:0/429403461 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f101ad9d700 1 --2- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c15bdd0 0x7f101c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.522+0000 7f101a59c700 1 --2- 172.21.15.97:0/429403461 >> v2:172.21.15.183:3300/0 conn(0x7f101c0b3370 0x7f101c0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f1019d9b700 1 -- 172.21.15.97:0/429403461 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756718450 0 0) 0x7f101c0b2030 con 0x7f101c0d1c70 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f1019d9b700 1 -- 172.21.15.97:0/429403461 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10080035b0 con 0x7f101c0d1c70 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f101a59c700 1 -- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c15bdd0 msgr2=0x7f101c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f101a59c700 1 --2- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c15bdd0 0x7f101c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f101a59c700 1 -- 172.21.15.97:0/429403461 >> v1:172.21.15.97:6789/0 conn(0x7f101c0d1c70 legacy=0x7f101c0ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f101a59c700 1 -- 172.21.15.97:0/429403461 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c0b25b0 con 0x7f101c0b3370 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f101ad9d700 1 --2- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c15bdd0 0x7f101c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f101a59c700 1 --2- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c0b3370 0x7f101c0b1af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f100c009840 tx=0x7f100c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d0201f689976cd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f1019d9b700 1 -- 172.21.15.97:0/429403461 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c011830 con 0x7f101c0b3370 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f1019d9b700 1 -- 172.21.15.97:0/429403461 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c011990 con 0x7f101c0b3370 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.523+0000 7f1019d9b700 1 -- 172.21.15.97:0/429403461 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c011d00 con 0x7f101c0b3370 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.524+0000 7f102112d700 1 -- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c0b3370 msgr2=0x7f101c0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.524+0000 7f102112d700 1 --2- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c0b3370 0x7f101c0b1af0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f100c009840 tx=0x7f100c009b50 comp rx=0 tx=0).stop 2023-11-24T06:09:56.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.524+0000 7f102112d700 1 -- 172.21.15.97:0/429403461 shutdown_connections 2023-11-24T06:09:56.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.524+0000 7f102112d700 1 --2- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c15bdd0 0x7f101c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.524+0000 7f102112d700 1 --2- 172.21.15.97:0/429403461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c0b3370 0x7f101c0b1af0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.524+0000 7f102112d700 1 -- 172.21.15.97:0/429403461 >> 172.21.15.97:0/429403461 conn(0x7f101c0731a0 msgr2=0x7f101c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:56.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.524+0000 7f102112d700 1 -- 172.21.15.97:0/429403461 shutdown_connections 2023-11-24T06:09:56.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.524+0000 7f102112d700 1 -- 172.21.15.97:0/429403461 wait complete. 2023-11-24T06:09:56.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.525+0000 7f102112d700 1 Processor -- start 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.525+0000 7f102112d700 1 -- start start 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f102112d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c0b3370 0x7f101c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f102112d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0d1c70 0x7f101c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f102112d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c15bdd0 0x7f101c0b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f102112d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0b6930 con 0x7f101c0d1c70 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f102112d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0b54e0 con 0x7f101c15bdd0 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f102112d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c156e20 con 0x7f101c0b3370 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0d1c70 0x7f101c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0d1c70 0x7f101c0b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46956/0 (socket says 172.21.15.97:46956) 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 -- 172.21.15.97:0/1455345800 learned_addr learned my addr 172.21.15.97:0/1455345800 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101ad9d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c0b3370 0x7f101c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101b59e700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c15bdd0 0x7f101c0b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 -- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c0b3370 msgr2=0x7f101c0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c0b3370 0x7f101c0b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 -- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c15bdd0 msgr2=0x7f101c0b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c15bdd0 0x7f101c0b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 -- 172.21.15.97:0/1455345800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10040094a0 con 0x7f101c0d1c70 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101b59e700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c15bdd0 0x7f101c0b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101ad9d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c0b3370 0x7f101c0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.526+0000 7f101a59c700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0d1c70 0x7f101c0b5eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f100c000c00 tx=0x7f100c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.527+0000 7f10137fe700 1 -- 172.21.15.97:0/1455345800 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c0045d0 con 0x7f101c0d1c70 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.527+0000 7f10137fe700 1 -- 172.21.15.97:0/1455345800 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c01f070 con 0x7f101c0d1c70 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.527+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c0094a0 con 0x7f101c0d1c70 2023-11-24T06:09:56.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.527+0000 7f10137fe700 1 -- 172.21.15.97:0/1455345800 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c0038a0 con 0x7f101c0d1c70 2023-11-24T06:09:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.527+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c0b4bb0 con 0x7f101c0d1c70 2023-11-24T06:09:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.528+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c0b57c0 con 0x7f101c0d1c70 2023-11-24T06:09:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.528+0000 7f10137fe700 1 -- 172.21.15.97:0/1455345800 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f100c003a00 con 0x7f101c0d1c70 2023-11-24T06:09:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.529+0000 7f10137fe700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ffc06aed0 0x7f0ffc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.529+0000 7f10137fe700 1 -- 172.21.15.97:0/1455345800 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f100c021070 con 0x7f101c0d1c70 2023-11-24T06:09:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.529+0000 7f101ad9d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ffc06aed0 0x7f0ffc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.534 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.529+0000 7f101ad9d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ffc06aed0 0x7f0ffc06d390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f101c1f7350 tx=0x7f1004008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.533+0000 7f10137fe700 1 -- 172.21.15.97:0/1455345800 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f100c055f90 con 0x7f101c0d1c70 2023-11-24T06:09:56.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.724+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f101c0b5620 con 0x7f101c0d1c70 2023-11-24T06:09:56.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.725+0000 7f10137fe700 1 -- 172.21.15.97:0/1455345800 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16594 (secure 0 0 0) 0x7f101c0b5620 con 0x7f101c0d1c70 2023-11-24T06:09:56.728 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:56.729 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":32,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:55.467660+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:09:53.319367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:09:54.404860+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:56.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ffc06aed0 msgr2=0x7f0ffc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ffc06aed0 0x7f0ffc06d390 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f101c1f7350 tx=0x7f1004008040 comp rx=0 tx=0).stop 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0d1c70 msgr2=0x7f101c0b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0d1c70 0x7f101c0b5eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f100c000c00 tx=0x7f100c004750 comp rx=0 tx=0).stop 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 shutdown_connections 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f101c15bdd0 0x7f101c0b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0ffc06aed0 0x7f0ffc06d390 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f101c0b3370 0x7f101c0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 --2- 172.21.15.97:0/1455345800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f101c0d1c70 0x7f101c0b5eb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 >> 172.21.15.97:0/1455345800 conn(0x7f101c0731a0 msgr2=0x7f101c159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.727+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 shutdown_connections 2023-11-24T06:09:56.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.728+0000 7f102112d700 1 -- 172.21.15.97:0/1455345800 wait complete. 2023-11-24T06:09:56.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:09:56.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.862+0000 7ff3058d3700 1 Processor -- start 2023-11-24T06:09:56.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.862+0000 7ff3058d3700 1 -- start start 2023-11-24T06:09:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff3058d3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff30015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff3058d3700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3000ae5e0 con 0x7ff3000b3c60 2023-11-24T06:09:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff3058d3700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30015be30 con 0x7ff3001587f0 2023-11-24T06:09:56.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff3058d3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30015c110 con 0x7ff30015b370 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2ff7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff3000b3c60 0x7ff3000af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39418/0 (socket says 172.21.15.97:39418) 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2ff7fe700 1 -- 172.21.15.97:0/4016631156 learned_addr learned my addr 172.21.15.97:0/4016631156 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2fe7fc700 1 --2- 172.21.15.97:0/4016631156 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff30015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2fdffb700 1 -- 172.21.15.97:0/4016631156 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840971704 0 0) 0x7ff3000ae5e0 con 0x7ff3000b3c60 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2fdffb700 1 -- 172.21.15.97:0/4016631156 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e80057c0 con 0x7ff3000b3c60 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2fdffb700 1 -- 172.21.15.97:0/4016631156 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736388539 0 0) 0x7ff30015be30 con 0x7ff3001587f0 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2fdffb700 1 -- 172.21.15.97:0/4016631156 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3000ae5e0 con 0x7ff3001587f0 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2fe7fc700 1 -- 172.21.15.97:0/4016631156 >> v1:172.21.15.183:6789/0 conn(0x7ff3001587f0 legacy=0x7ff30006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.863+0000 7ff2fe7fc700 1 -- 172.21.15.97:0/4016631156 >> v1:172.21.15.97:6789/0 conn(0x7ff3000b3c60 legacy=0x7ff3000af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff2fe7fc700 1 -- 172.21.15.97:0/4016631156 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3000ae860 con 0x7ff30015b370 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff2fe7fc700 1 --2- 172.21.15.97:0/4016631156 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff30015b7d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009870 tx=0x7ff2f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a95acbd06a96506 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff2fdffb700 1 -- 172.21.15.97:0/4016631156 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0011830 con 0x7ff30015b370 2023-11-24T06:09:56.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff2fdffb700 1 -- 172.21.15.97:0/4016631156 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f0011990 con 0x7ff30015b370 2023-11-24T06:09:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff2fdffb700 1 -- 172.21.15.97:0/4016631156 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0011d00 con 0x7ff30015b370 2023-11-24T06:09:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff3058d3700 1 -- 172.21.15.97:0/4016631156 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 msgr2=0x7ff30015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff3058d3700 1 --2- 172.21.15.97:0/4016631156 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff30015b7d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009870 tx=0x7ff2f0009b80 comp rx=0 tx=0).stop 2023-11-24T06:09:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff3058d3700 1 -- 172.21.15.97:0/4016631156 shutdown_connections 2023-11-24T06:09:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff3058d3700 1 --2- 172.21.15.97:0/4016631156 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff30015b7d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff3058d3700 1 -- 172.21.15.97:0/4016631156 >> 172.21.15.97:0/4016631156 conn(0x7ff3000d0bf0 msgr2=0x7ff3000d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff3058d3700 1 -- 172.21.15.97:0/4016631156 shutdown_connections 2023-11-24T06:09:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.864+0000 7ff3058d3700 1 -- 172.21.15.97:0/4016631156 wait complete. 2023-11-24T06:09:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.866+0000 7ff3058d3700 1 Processor -- start 2023-11-24T06:09:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.866+0000 7ff3058d3700 1 -- start start 2023-11-24T06:09:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.866+0000 7ff3058d3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3000b3c60 0x7ff30006ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.866+0000 7ff3058d3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3001587f0 0x7ff30014ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.866+0000 7ff3058d3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff300149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.866+0000 7ff3058d3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30014a9f0 con 0x7ff3001587f0 2023-11-24T06:09:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.866+0000 7ff3058d3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30014c0c0 con 0x7ff3000b3c60 2023-11-24T06:09:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.866+0000 7ff3058d3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff300149d70 con 0x7ff30015b370 2023-11-24T06:09:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3001587f0 0x7ff30014ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3001587f0 0x7ff30014ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46970/0 (socket says 172.21.15.97:46970) 2023-11-24T06:09:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 -- 172.21.15.97:0/1601093981 learned_addr learned my addr 172.21.15.97:0/1601093981 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2feffd700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3000b3c60 0x7ff30006ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 -- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 msgr2=0x7ff300149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2ff7fe700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff300149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff300149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 -- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3000b3c60 msgr2=0x7ff30006ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3000b3c60 0x7ff30006ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 -- 172.21.15.97:0/1601093981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f00094a0 con 0x7ff3001587f0 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2feffd700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3000b3c60 0x7ff30006ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2ff7fe700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff300149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2fe7fc700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3001587f0 0x7ff30014ae10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009bc0 tx=0x7ff2f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2e7fff700 1 -- 172.21.15.97:0/1601093981 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f00045c0 con 0x7ff3001587f0 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2e7fff700 1 -- 172.21.15.97:0/1601093981 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f001e070 con 0x7ff3001587f0 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff2e7fff700 1 -- 172.21.15.97:0/1601093981 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003900 con 0x7ff3001587f0 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30014a050 con 0x7ff3001587f0 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.867+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff30014a610 con 0x7ff3001587f0 2023-11-24T06:09:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.868+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff300068fb0 con 0x7ff3001587f0 2023-11-24T06:09:56.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.869+0000 7ff2e7fff700 1 -- 172.21.15.97:0/1601093981 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2f0003a60 con 0x7ff3001587f0 2023-11-24T06:09:56.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.870+0000 7ff2e7fff700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff2ec06aed0 0x7ff2ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:56.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.870+0000 7ff2e7fff700 1 -- 172.21.15.97:0/1601093981 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2f008aa10 con 0x7ff3001587f0 2023-11-24T06:09:56.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.870+0000 7ff2feffd700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff2ec06aed0 0x7ff2ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:56.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.870+0000 7ff2feffd700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff2ec06aed0 0x7ff2ec06d390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40011e0 tx=0x7ff2f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:56.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:56.873+0000 7ff2e7fff700 1 -- 172.21.15.97:0/1601093981 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2f0055b30 con 0x7ff3001587f0 2023-11-24T06:09:57.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:57.116+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff30014ab30 con 0x7ff3001587f0 2023-11-24T06:09:58.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.057+0000 7ff2e7fff700 1 -- 172.21.15.97:0/1601093981 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff30014ab30 con 0x7ff3001587f0 2023-11-24T06:09:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff2ec06aed0 msgr2=0x7ff2ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff2ec06aed0 0x7ff2ec06d390 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40011e0 tx=0x7ff2f4009040 comp rx=0 tx=0).stop 2023-11-24T06:09:58.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3001587f0 msgr2=0x7ff30014ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3001587f0 0x7ff30014ae10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009bc0 tx=0x7ff2f0004750 comp rx=0 tx=0).stop 2023-11-24T06:09:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 shutdown_connections 2023-11-24T06:09:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff2ec06aed0 0x7ff2ec06d390 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff30015b370 0x7ff300149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3000b3c60 0x7ff30006ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 --2- 172.21.15.97:0/1601093981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3001587f0 0x7ff30014ae10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 >> 172.21.15.97:0/1601093981 conn(0x7ff3000d0bf0 msgr2=0x7ff30006e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 shutdown_connections 2023-11-24T06:09:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.061+0000 7ff3058d3700 1 -- 172.21.15.97:0/1601093981 wait complete. 2023-11-24T06:09:58.079 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:09:58.079 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:09:58.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.206+0000 7f16758de700 1 Processor -- start 2023-11-24T06:09:58.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.206+0000 7f16758de700 1 -- start start 2023-11-24T06:09:58.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f16758de700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f16700b2270 0x7f167006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:58.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f16758de700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f16700b28b0 0x7f167006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:58.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f16758de700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16700b2650 con 0x7f167006ba70 2023-11-24T06:09:58.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f16758de700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16700d1240 con 0x7f16700b2270 2023-11-24T06:09:58.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f16758de700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16700b43d0 con 0x7f16700b28b0 2023-11-24T06:09:58.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f16700b2270 0x7f167006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:58.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f16700b2270 0x7f167006aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46976/0 (socket says 172.21.15.97:46976) 2023-11-24T06:09:58.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166effd700 1 -- 172.21.15.97:0/756986536 learned_addr learned my addr 172.21.15.97:0/756986536 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:58.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166effd700 1 -- 172.21.15.97:0/756986536 >> v2:172.21.15.183:3300/0 conn(0x7f16700b28b0 msgr2=0x7f167006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:58.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166effd700 1 --2- 172.21.15.97:0/756986536 >> v2:172.21.15.183:3300/0 conn(0x7f16700b28b0 0x7f167006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166effd700 1 -- 172.21.15.97:0/756986536 >> v1:172.21.15.183:6789/0 conn(0x7f167006ba70 legacy=0x7f1670161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166effd700 1 -- 172.21.15.97:0/756986536 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700b46b0 con 0x7f16700b2270 2023-11-24T06:09:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166effd700 1 --2- 172.21.15.97:0/756986536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b2270 0x7f167006aff0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1670071d00 tx=0x7f1664009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844326ef985822f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166dffb700 1 -- 172.21.15.97:0/756986536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664004230 con 0x7f16700b2270 2023-11-24T06:09:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166dffb700 1 -- 172.21.15.97:0/756986536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1664004390 con 0x7f16700b2270 2023-11-24T06:09:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.207+0000 7f166dffb700 1 -- 172.21.15.97:0/756986536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640036a0 con 0x7f16700b2270 2023-11-24T06:09:58.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.208+0000 7f16758de700 1 -- 172.21.15.97:0/756986536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b2270 msgr2=0x7f167006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.208+0000 7f16758de700 1 --2- 172.21.15.97:0/756986536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b2270 0x7f167006aff0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1670071d00 tx=0x7f1664009bd0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.208+0000 7f16758de700 1 -- 172.21.15.97:0/756986536 shutdown_connections 2023-11-24T06:09:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.208+0000 7f16758de700 1 --2- 172.21.15.97:0/756986536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b2270 0x7f167006aff0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.208+0000 7f16758de700 1 --2- 172.21.15.97:0/756986536 >> v2:172.21.15.183:3300/0 conn(0x7f16700b28b0 0x7f167006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.208+0000 7f16758de700 1 -- 172.21.15.97:0/756986536 >> 172.21.15.97:0/756986536 conn(0x7f16700b3810 msgr2=0x7f16700b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.208+0000 7f16758de700 1 -- 172.21.15.97:0/756986536 shutdown_connections 2023-11-24T06:09:58.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.208+0000 7f16758de700 1 -- 172.21.15.97:0/756986536 wait complete. 2023-11-24T06:09:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.209+0000 7f16758de700 1 Processor -- start 2023-11-24T06:09:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.209+0000 7f16758de700 1 -- start start 2023-11-24T06:09:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f16758de700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f167006ba70 0x7f1670152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:58.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f16758de700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f16700b2270 0x7f16700c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f16758de700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b28b0 0x7f16700ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f16758de700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16700cb9e0 con 0x7f16700b28b0 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f16758de700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16700c8e70 con 0x7f167006ba70 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f16758de700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16700c9150 con 0x7f16700b2270 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b28b0 0x7f16700ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b28b0 0x7f16700ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46986/0 (socket says 172.21.15.97:46986) 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166f7fe700 1 -- 172.21.15.97:0/3425681223 learned_addr learned my addr 172.21.15.97:0/3425681223 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166effd700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f167006ba70 0x7f1670152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166e7fc700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f16700b2270 0x7f16700c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166f7fe700 1 -- 172.21.15.97:0/3425681223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f16700b2270 msgr2=0x7f16700c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166f7fe700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f16700b2270 0x7f16700c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166f7fe700 1 -- 172.21.15.97:0/3425681223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f167006ba70 msgr2=0x7f1670152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166f7fe700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f167006ba70 0x7f1670152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.210+0000 7f166f7fe700 1 -- 172.21.15.97:0/3425681223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1664009550 con 0x7f16700b28b0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.211+0000 7f166f7fe700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b28b0 0x7f16700ceb60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f166800cc10 tx=0x7f166800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.211+0000 7f1657fff700 1 -- 172.21.15.97:0/3425681223 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166800aa60 con 0x7f16700b28b0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.211+0000 7f1657fff700 1 -- 172.21.15.97:0/3425681223 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166800abc0 con 0x7f16700b28b0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.211+0000 7f1657fff700 1 -- 172.21.15.97:0/3425681223 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668012420 con 0x7f16700b28b0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.211+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16700c9430 con 0x7f16700b28b0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.211+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16700c99f0 con 0x7f16700b28b0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.212+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1670068fb0 con 0x7f16700b28b0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.212+0000 7f1657fff700 1 -- 172.21.15.97:0/3425681223 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1668012580 con 0x7f16700b28b0 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.213+0000 7f1657fff700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f165c06aed0 0x7f165c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:58.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.213+0000 7f166effd700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f165c06aed0 0x7f165c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:58.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.213+0000 7f1657fff700 1 -- 172.21.15.97:0/3425681223 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16680898f0 con 0x7f16700b28b0 2023-11-24T06:09:58.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.213+0000 7f166effd700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f165c06aed0 0x7f165c06d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1664009130 tx=0x7f1664008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:58.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.217+0000 7f1657fff700 1 -- 172.21.15.97:0/3425681223 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1668054910 con 0x7f16700b28b0 2023-11-24T06:09:58.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:58.450+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f16700b7c80 con 0x7f16700b28b0 2023-11-24T06:09:59.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.061+0000 7f1657fff700 1 -- 172.21.15.97:0/3425681223 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1668057f30 con 0x7f16700b28b0 2023-11-24T06:09:59.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f165c06aed0 msgr2=0x7f165c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f165c06aed0 0x7f165c06d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1664009130 tx=0x7f1664008030 comp rx=0 tx=0).stop 2023-11-24T06:09:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b28b0 msgr2=0x7f16700ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b28b0 0x7f16700ceb60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f166800cc10 tx=0x7f166800cf20 comp rx=0 tx=0).stop 2023-11-24T06:09:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 shutdown_connections 2023-11-24T06:09:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16700b28b0 0x7f16700ceb60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f167006ba70 0x7f1670152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f165c06aed0 0x7f165c06d390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 --2- 172.21.15.97:0/3425681223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f16700b2270 0x7f16700c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 >> 172.21.15.97:0/3425681223 conn(0x7f16700b3810 msgr2=0x7f16700d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 shutdown_connections 2023-11-24T06:09:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.065+0000 7f16758de700 1 -- 172.21.15.97:0/3425681223 wait complete. 2023-11-24T06:09:59.081 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:09:59.082 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:09:59.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.200+0000 7fd8ed2c5700 1 Processor -- start 2023-11-24T06:09:59.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.200+0000 7fd8ed2c5700 1 -- start start 2023-11-24T06:09:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8ed2c5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e815e1e0 0x7fd8e80d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8ed2c5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd8e80d3a00 0x7fd8e80d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8ed2c5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e80d20b0 con 0x7fd8e80d0db0 2023-11-24T06:09:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8ed2c5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e80d3150 con 0x7fd8e80d3a00 2023-11-24T06:09:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8ed2c5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80d1240 con 0x7fd8e815e1e0 2023-11-24T06:09:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e77fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd8e80d0db0 0x7fd8e806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39446/0 (socket says 172.21.15.97:39446) 2023-11-24T06:09:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e77fe700 1 -- 172.21.15.97:0/12822049 learned_addr learned my addr 172.21.15.97:0/12822049 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:09:59.207 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e67fc700 1 --2- 172.21.15.97:0/12822049 >> v2:172.21.15.183:3300/0 conn(0x7fd8e80d3a00 0x7fd8e80d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e6ffd700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e815e1e0 0x7fd8e80d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884828543 0 0) 0x7fd8e80d20b0 con 0x7fd8e80d0db0 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8d00035b0 con 0x7fd8e80d0db0 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1558191124 0 0) 0x7fd8d00035b0 con 0x7fd8e80d0db0 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e815e1e0 msgr2=0x7fd8e80d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e815e1e0 0x7fd8e80d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 >> v2:172.21.15.183:3300/0 conn(0x7fd8e80d3a00 msgr2=0x7fd8e80d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 --2- 172.21.15.97:0/12822049 >> v2:172.21.15.183:3300/0 conn(0x7fd8e80d3a00 0x7fd8e80d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80d2390 con 0x7fd8e80d0db0 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.201+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fd8e00019f0 con 0x7fd8e80d0db0 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 >> v1:172.21.15.97:6789/0 conn(0x7fd8e80d0db0 legacy=0x7fd8e806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0003a00 0x7fd8d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e815e1e0 0x7fd8d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8d0004d90 0x7fd8d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80d3150 con 0x7fd8d0004d90 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80d1240 con 0x7fd8d0003a00 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d000ba10 con 0x7fd8e815e1e0 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e77fe700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8d0004d90 0x7fd8d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e6ffd700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0003a00 0x7fd8d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e67fc700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e815e1e0 0x7fd8d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e77fe700 1 -- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e815e1e0 msgr2=0x7fd8d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e77fe700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e815e1e0 0x7fd8d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e77fe700 1 -- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0003a00 msgr2=0x7fd8d0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:59.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e77fe700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8d0003a00 0x7fd8d0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e77fe700 1 -- 172.21.15.97:0/12822049 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00035b0 con 0x7fd8d0004d90 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e77fe700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8d0004d90 0x7fd8d0005210 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000a8d0 tx=0x7fd8d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c7febf4f2e6775 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000a340 con 0x7fd8d0004d90 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e0012600 con 0x7fd8d0004d90 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.202+0000 7fd8e5ffb700 1 -- 172.21.15.97:0/12822049 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00129b0 con 0x7fd8d0004d90 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.203+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/12822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8d0004d90 msgr2=0x7fd8d0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.203+0000 7fd8ed2c5700 1 --2- 172.21.15.97:0/12822049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8d0004d90 0x7fd8d0005210 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd8e000a8d0 tx=0x7fd8d0005880 comp rx=0 tx=0).stop 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.203+0000 7fd8e77fe700 1 -- 172.21.15.97:0/12822049 reap_dead start 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.203+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/12822049 shutdown_connections 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.203+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/12822049 >> 172.21.15.97:0/12822049 conn(0x7fd8e80b5b50 msgr2=0x7fd8e80b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:59.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.203+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/12822049 shutdown_connections 2023-11-24T06:09:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.203+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/12822049 wait complete. 2023-11-24T06:09:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.204+0000 7fd8ed2c5700 1 Processor -- start 2023-11-24T06:09:59.211 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.204+0000 7fd8ed2c5700 1 -- start start 2023-11-24T06:09:59.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8ed2c5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8e8152180 0x7fd8e815f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8ed2c5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e8145320 0x7fd8e81457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8ed2c5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e81557f0 0x7fd8e8155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8ed2c5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80cdfc0 con 0x7fd8e81557f0 2023-11-24T06:09:59.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8ed2c5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e815fcd0 con 0x7fd8e8152180 2023-11-24T06:09:59.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8ed2c5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e8145ce0 con 0x7fd8e8145320 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e81557f0 0x7fd8e8155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e81557f0 0x7fd8e8155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47006/0 (socket says 172.21.15.97:47006) 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 -- 172.21.15.97:0/2001913407 learned_addr learned my addr 172.21.15.97:0/2001913407 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e67fc700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e8145320 0x7fd8e81457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e6ffd700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8e8152180 0x7fd8e815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 -- 172.21.15.97:0/2001913407 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e8145320 msgr2=0x7fd8e81457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e8145320 0x7fd8e81457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 -- 172.21.15.97:0/2001913407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8e8152180 msgr2=0x7fd8e815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8e8152180 0x7fd8e815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 -- 172.21.15.97:0/2001913407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e815ff00 con 0x7fd8e81557f0 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.205+0000 7fd8e77fe700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e81557f0 0x7fd8e8155c70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0004770 tx=0x7fd8e0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.206+0000 7fd8cf7fe700 1 -- 172.21.15.97:0/2001913407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000a340 con 0x7fd8e81557f0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.206+0000 7fd8cf7fe700 1 -- 172.21.15.97:0/2001913407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e00019c0 con 0x7fd8e81557f0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.206+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e81601e0 con 0x7fd8e81557f0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.206+0000 7fd8cf7fe700 1 -- 172.21.15.97:0/2001913407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0027410 con 0x7fd8e81557f0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.206+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e81607a0 con 0x7fd8e81557f0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.207+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e8160df0 con 0x7fd8e81557f0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.207+0000 7fd8cf7fe700 1 -- 172.21.15.97:0/2001913407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8e0019070 con 0x7fd8e81557f0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.207+0000 7fd8cf7fe700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd8d406aed0 0x7fd8d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.207+0000 7fd8cf7fe700 1 -- 172.21.15.97:0/2001913407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8e008a930 con 0x7fd8e81557f0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.207+0000 7fd8e6ffd700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd8d406aed0 0x7fd8d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.208+0000 7fd8e6ffd700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd8d406aed0 0x7fd8d406d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8e81431e0 tx=0x7fd8dc009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.212+0000 7fd8cf7fe700 1 -- 172.21.15.97:0/2001913407 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8e0055b30 con 0x7fd8e81557f0 2023-11-24T06:09:59.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.446+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8e80cf700 con 0x7fd8e81557f0 2023-11-24T06:09:59.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.446+0000 7fd8cf7fe700 1 -- 172.21.15.97:0/2001913407 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fd8e0022090 con 0x7fd8e81557f0 2023-11-24T06:09:59.449 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:59.449 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1778230784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1778230784},{"type":"v1","addr":"172.21.15.97:6836","nonce":1778230784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2183416217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2183416217},{"type":"v1","addr":"172.21.15.183:6834","nonce":2183416217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3650123299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3650123299},{"type":"v1","addr":"172.21.15.183:6835","nonce":3650123299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:09:55.467969+0000","modified":"2023-11-24T06:09:58.451611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5070},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5070":{"gid":5070,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3767182596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3767182596},{"type":"v1","addr":"172.21.15.97:6837","nonce":3767182596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:09:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd8d406aed0 msgr2=0x7fd8d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd8d406aed0 0x7fd8d406d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd8e81431e0 tx=0x7fd8dc009210 comp rx=0 tx=0).stop 2023-11-24T06:09:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e81557f0 msgr2=0x7fd8e8155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e81557f0 0x7fd8e8155c70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0004770 tx=0x7fd8e0001360 comp rx=0 tx=0).stop 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 shutdown_connections 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8e81557f0 0x7fd8e8155c70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8e8152180 0x7fd8e815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd8d406aed0 0x7fd8d406d390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 --2- 172.21.15.97:0/2001913407 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd8e8145320 0x7fd8e81457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 >> 172.21.15.97:0/2001913407 conn(0x7fd8e80b5b50 msgr2=0x7fd8e80b6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 shutdown_connections 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.448+0000 7fd8ed2c5700 1 -- 172.21.15.97:0/2001913407 wait complete. 2023-11-24T06:09:59.452 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2023-11-24T06:09:59.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:09:59.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.580+0000 7f7babd23700 1 Processor -- start 2023-11-24T06:09:59.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.581+0000 7f7babd23700 1 -- start start 2023-11-24T06:09:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.581+0000 7f7babd23700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7ba415e080 0x7f7ba40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.581+0000 7f7babd23700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ba40af4d0 0x7f7ba40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.581+0000 7f7babd23700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba40ae430 con 0x7f7ba406c000 2023-11-24T06:09:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.581+0000 7f7babd23700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba40af0b0 con 0x7f7ba40af4d0 2023-11-24T06:09:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.581+0000 7f7babd23700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba40b05a0 con 0x7f7ba415e080 2023-11-24T06:09:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba92be700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ba40af4d0 0x7f7ba40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba92be700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ba40af4d0 0x7f7ba40afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47010/0 (socket says 172.21.15.97:47010) 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba92be700 1 -- 172.21.15.97:0/2012915147 learned_addr learned my addr 172.21.15.97:0/2012915147 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba9abf700 1 --2- 172.21.15.97:0/2012915147 >> v2:172.21.15.183:3300/0 conn(0x7f7ba415e080 0x7f7ba40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba92be700 1 -- 172.21.15.97:0/2012915147 >> v2:172.21.15.183:3300/0 conn(0x7f7ba415e080 msgr2=0x7f7ba40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba92be700 1 --2- 172.21.15.97:0/2012915147 >> v2:172.21.15.183:3300/0 conn(0x7f7ba415e080 0x7f7ba40b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba92be700 1 -- 172.21.15.97:0/2012915147 >> v1:172.21.15.183:6789/0 conn(0x7f7ba406c000 legacy=0x7f7ba4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba92be700 1 -- 172.21.15.97:0/2012915147 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40ae710 con 0x7f7ba40af4d0 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba92be700 1 --2- 172.21.15.97:0/2012915147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba40af4d0 0x7f7ba40afb90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4072a90 tx=0x7f7ba0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5307da0b088a3a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba8abd700 1 -- 172.21.15.97:0/2012915147 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0004230 con 0x7f7ba40af4d0 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba8abd700 1 -- 172.21.15.97:0/2012915147 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba0004390 con 0x7f7ba40af4d0 2023-11-24T06:09:59.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.582+0000 7f7ba8abd700 1 -- 172.21.15.97:0/2012915147 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00036a0 con 0x7f7ba40af4d0 2023-11-24T06:09:59.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.583+0000 7f7babd23700 1 -- 172.21.15.97:0/2012915147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba40af4d0 msgr2=0x7f7ba40afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.583+0000 7f7babd23700 1 --2- 172.21.15.97:0/2012915147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba40af4d0 0x7f7ba40afb90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4072a90 tx=0x7f7ba0009c00 comp rx=0 tx=0).stop 2023-11-24T06:09:59.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.583+0000 7f7babd23700 1 -- 172.21.15.97:0/2012915147 shutdown_connections 2023-11-24T06:09:59.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.583+0000 7f7babd23700 1 --2- 172.21.15.97:0/2012915147 >> v2:172.21.15.183:3300/0 conn(0x7f7ba415e080 0x7f7ba40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.583+0000 7f7babd23700 1 --2- 172.21.15.97:0/2012915147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba40af4d0 0x7f7ba40afb90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.583+0000 7f7babd23700 1 -- 172.21.15.97:0/2012915147 >> 172.21.15.97:0/2012915147 conn(0x7f7ba4071b90 msgr2=0x7f7ba4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:59.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.583+0000 7f7babd23700 1 -- 172.21.15.97:0/2012915147 shutdown_connections 2023-11-24T06:09:59.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.583+0000 7f7babd23700 1 -- 172.21.15.97:0/2012915147 wait complete. 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.584+0000 7f7babd23700 1 Processor -- start 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.584+0000 7f7babd23700 1 -- start start 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7babd23700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406c000 0x7f7ba414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7babd23700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ba40af4d0 0x7f7ba41473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7babd23700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba415e080 0x7f7ba41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7babd23700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba414a7c0 con 0x7f7ba415e080 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7babd23700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4154c70 con 0x7f7ba406c000 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7babd23700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba4154f50 con 0x7f7ba40af4d0 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba415e080 0x7f7ba41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba415e080 0x7f7ba41561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47020/0 (socket says 172.21.15.97:47020) 2023-11-24T06:09:59.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 -- 172.21.15.97:0/1908123256 learned_addr learned my addr 172.21.15.97:0/1908123256 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7ba9abf700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406c000 0x7f7ba414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7ba92be700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ba40af4d0 0x7f7ba41473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 -- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ba40af4d0 msgr2=0x7f7ba41473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ba40af4d0 0x7f7ba41473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 -- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406c000 msgr2=0x7f7ba414a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406c000 0x7f7ba414a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 -- 172.21.15.97:0/1908123256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba0009550 con 0x7f7ba415e080 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7ba92be700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ba40af4d0 0x7f7ba41473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7baa2c0700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba415e080 0x7f7ba41561b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00cc10 tx=0x7f7b9c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7b9a7fc700 1 -- 172.21.15.97:0/1908123256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00ab10 con 0x7f7ba415e080 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7b9a7fc700 1 -- 172.21.15.97:0/1908123256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c00ac70 con 0x7f7ba415e080 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba4155230 con 0x7f7ba415e080 2023-11-24T06:09:59.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.585+0000 7f7b9a7fc700 1 -- 172.21.15.97:0/1908123256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c012420 con 0x7f7ba415e080 2023-11-24T06:09:59.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.586+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba41557f0 con 0x7f7ba415e080 2023-11-24T06:09:59.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.586+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba41559b0 con 0x7f7ba415e080 2023-11-24T06:09:59.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.587+0000 7f7b9a7fc700 1 -- 172.21.15.97:0/1908123256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b9c012580 con 0x7f7ba415e080 2023-11-24T06:09:59.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.587+0000 7f7b9a7fc700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b8806aed0 0x7f7b8806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.587+0000 7f7b9a7fc700 1 -- 172.21.15.97:0/1908123256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b9c089920 con 0x7f7ba415e080 2023-11-24T06:09:59.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.587+0000 7f7ba9abf700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b8806aed0 0x7f7b8806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.588+0000 7f7ba9abf700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b8806aed0 0x7f7b8806d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b94006fd0 tx=0x7f7b94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.592+0000 7f7b9a7fc700 1 -- 172.21.15.97:0/1908123256 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b9c054a60 con 0x7f7ba415e080 2023-11-24T06:09:59.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.783+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ba40753f0 con 0x7f7ba415e080 2023-11-24T06:09:59.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.784+0000 7f7b9a7fc700 1 -- 172.21.15.97:0/1908123256 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f7b9c058080 con 0x7f7ba415e080 2023-11-24T06:09:59.787 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:09:59.787 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:57.042131+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:09:53.319367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:09:54.404860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:09:59.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b8806aed0 msgr2=0x7f7b8806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b8806aed0 0x7f7b8806d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b94006fd0 tx=0x7f7b94008040 comp rx=0 tx=0).stop 2023-11-24T06:09:59.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba415e080 msgr2=0x7f7ba41561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba415e080 0x7f7ba41561b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00cc10 tx=0x7f7b9c00cf20 comp rx=0 tx=0).stop 2023-11-24T06:09:59.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 shutdown_connections 2023-11-24T06:09:59.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba415e080 0x7f7ba41561b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ba406c000 0x7f7ba414a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b8806aed0 0x7f7b8806d390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 --2- 172.21.15.97:0/1908123256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7ba40af4d0 0x7f7ba41473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 >> 172.21.15.97:0/1908123256 conn(0x7f7ba4071b90 msgr2=0x7f7ba4160470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:59.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 shutdown_connections 2023-11-24T06:09:59.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.786+0000 7f7babd23700 1 -- 172.21.15.97:0/1908123256 wait complete. 2023-11-24T06:09:59.801 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:09:59.801 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:09:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.917+0000 7f3b59ab4700 1 Processor -- start 2023-11-24T06:09:59.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.918+0000 7f3b59ab4700 1 -- start start 2023-11-24T06:09:59.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.918+0000 7f3b59ab4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b5415e080 0x7f3b540b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.918+0000 7f3b59ab4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5406c710 0x7f3b540afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.918+0000 7f3b59ab4700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b540ae430 con 0x7f3b540af4d0 2023-11-24T06:09:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.918+0000 7f3b59ab4700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b540af0b0 con 0x7f3b5415e080 2023-11-24T06:09:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.918+0000 7f3b59ab4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540b05a0 con 0x7f3b5406c710 2023-11-24T06:09:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.918+0000 7f3b537fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b5415e080 0x7f3b540b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.918+0000 7f3b537fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b5415e080 0x7f3b540b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47024/0 (socket says 172.21.15.97:47024) 2023-11-24T06:09:59.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b537fe700 1 -- 172.21.15.97:0/3742419153 learned_addr learned my addr 172.21.15.97:0/3742419153 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:59.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b53fff700 1 --2- 172.21.15.97:0/3742419153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5406c710 0x7f3b540afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b537fe700 1 -- 172.21.15.97:0/3742419153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5406c710 msgr2=0x7f3b540afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:09:59.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b537fe700 1 --2- 172.21.15.97:0/3742419153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5406c710 0x7f3b540afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b537fe700 1 -- 172.21.15.97:0/3742419153 >> v1:172.21.15.183:6789/0 conn(0x7f3b540af4d0 legacy=0x7f3b5406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b537fe700 1 -- 172.21.15.97:0/3742419153 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b540ae710 con 0x7f3b5415e080 2023-11-24T06:09:59.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b537fe700 1 --2- 172.21.15.97:0/3742419153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5415e080 0x7f3b540b5050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009920 tx=0x7f3b44009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcb47eab6906d32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b52ffd700 1 -- 172.21.15.97:0/3742419153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400ba20 con 0x7f3b5415e080 2023-11-24T06:09:59.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b52ffd700 1 -- 172.21.15.97:0/3742419153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4400bb80 con 0x7f3b5415e080 2023-11-24T06:09:59.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.919+0000 7f3b52ffd700 1 -- 172.21.15.97:0/3742419153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44009e70 con 0x7f3b5415e080 2023-11-24T06:09:59.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.920+0000 7f3b59ab4700 1 -- 172.21.15.97:0/3742419153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5415e080 msgr2=0x7f3b540b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.920+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/3742419153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5415e080 0x7f3b540b5050 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009920 tx=0x7f3b44009c30 comp rx=0 tx=0).stop 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.920+0000 7f3b59ab4700 1 -- 172.21.15.97:0/3742419153 shutdown_connections 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.920+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/3742419153 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5406c710 0x7f3b540afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.920+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/3742419153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5415e080 0x7f3b540b5050 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.920+0000 7f3b59ab4700 1 -- 172.21.15.97:0/3742419153 >> 172.21.15.97:0/3742419153 conn(0x7f3b54071b90 msgr2=0x7f3b54071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.920+0000 7f3b59ab4700 1 -- 172.21.15.97:0/3742419153 shutdown_connections 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.920+0000 7f3b59ab4700 1 -- 172.21.15.97:0/3742419153 wait complete. 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.921+0000 7f3b59ab4700 1 Processor -- start 2023-11-24T06:09:59.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.921+0000 7f3b59ab4700 1 -- start start 2023-11-24T06:09:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b59ab4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5406c710 0x7f3b54151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b59ab4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b540af4d0 0x7f3b5415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b59ab4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5415e080 0x7f3b5406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b59ab4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5415fa30 con 0x7f3b540af4d0 2023-11-24T06:09:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b59ab4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b541584b0 con 0x7f3b5406c710 2023-11-24T06:09:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b59ab4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b54151a40 con 0x7f3b5415e080 2023-11-24T06:09:59.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b540af4d0 0x7f3b5415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b540af4d0 0x7f3b5415e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47030/0 (socket says 172.21.15.97:47030) 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 -- 172.21.15.97:0/1818029110 learned_addr learned my addr 172.21.15.97:0/1818029110 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b537fe700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5406c710 0x7f3b54151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b53fff700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5415e080 0x7f3b5406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 -- 172.21.15.97:0/1818029110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5415e080 msgr2=0x7f3b5406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5415e080 0x7f3b5406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 -- 172.21.15.97:0/1818029110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5406c710 msgr2=0x7f3b54151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5406c710 0x7f3b54151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 -- 172.21.15.97:0/1818029110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b44009550 con 0x7f3b540af4d0 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b4affd700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b540af4d0 0x7f3b5415e460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00c500 tx=0x7f3b3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b50ff9700 1 -- 172.21.15.97:0/1818029110 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0052d0 con 0x7f3b540af4d0 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b50ff9700 1 -- 172.21.15.97:0/1818029110 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3c005430 con 0x7f3b540af4d0 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b50ff9700 1 -- 172.21.15.97:0/1818029110 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c006aa0 con 0x7f3b540af4d0 2023-11-24T06:09:59.931 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5415eeb0 con 0x7f3b540af4d0 2023-11-24T06:09:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.922+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5415f470 con 0x7f3b540af4d0 2023-11-24T06:09:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.923+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b540753f0 con 0x7f3b540af4d0 2023-11-24T06:09:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.924+0000 7f3b50ff9700 1 -- 172.21.15.97:0/1818029110 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b3c003680 con 0x7f3b540af4d0 2023-11-24T06:09:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.924+0000 7f3b50ff9700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3b3406aed0 0x7f3b3406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:09:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.924+0000 7f3b50ff9700 1 -- 172.21.15.97:0/1818029110 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b3c088890 con 0x7f3b540af4d0 2023-11-24T06:09:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.924+0000 7f3b537fe700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3b3406aed0 0x7f3b3406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:09:59.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.925+0000 7f3b537fe700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3b3406aed0 0x7f3b3406d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b44000c00 tx=0x7f3b44008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:09:59.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:09:59.928+0000 7f3b50ff9700 1 -- 172.21.15.97:0/1818029110 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b3c053940 con 0x7f3b540af4d0 2023-11-24T06:10:00.010 INFO:tasks.ceph.mds.a.smithi097.stderr:2023-11-24T06:10:00.008+0000 7f7c6b681700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:10:00.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.118+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b540aecd0 con 0x7f3b3406aed0 2023-11-24T06:10:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.125+0000 7f3b50ff9700 1 -- 172.21.15.97:0/1818029110 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277110 (secure 0 0 0) 0x7f3b4400b820 con 0x7f3b3406aed0 2023-11-24T06:10:00.131 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3b3406aed0 msgr2=0x7f3b3406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3b3406aed0 0x7f3b3406d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b44000c00 tx=0x7f3b44008030 comp rx=0 tx=0).stop 2023-11-24T06:10:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b540af4d0 msgr2=0x7f3b5415e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b540af4d0 0x7f3b5415e460 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00c500 tx=0x7f3b3c00c8c0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 shutdown_connections 2023-11-24T06:10:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3b3406aed0 0x7f3b3406d390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b5415e080 0x7f3b5406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5406c710 0x7f3b54151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 --2- 172.21.15.97:0/1818029110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b540af4d0 0x7f3b5415e460 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 >> 172.21.15.97:0/1818029110 conn(0x7f3b54071b90 msgr2=0x7f3b540b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:00.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 shutdown_connections 2023-11-24T06:10:00.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.129+0000 7f3b59ab4700 1 -- 172.21.15.97:0/1818029110 wait complete. 2023-11-24T06:10:00.137 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:10:00.143 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-24T06:09:58.337328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219928,"kb_used_data":3736,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754792,"statfs":{"total":773094113280,"available":772868907008,"internally_reserved":0,"allocated":3825664,"data_stored":2175468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.314128"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.030043+0000","last_change":"2023-11-24T06:09:54.030043+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.030043+0000","last_undegraded":"2023-11-24T06:09:54.030043+0000","last_fullsized":"2023-11-24T06:09:54.030043+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.027181+0000","last_change":"2023-11-24T06:09:54.027181+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.027181+0000","last_undegraded":"2023-11-24T06:09:54.027181+0000","last_fullsized":"2023-11-24T06:09:54.027181+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.030323+0000","last_change":"2023-11-24T06:09:54.030323+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.030323+0000","last_undegraded":"2023-11-24T06:09:54.030323+0000","last_fullsized":"2023-11-24T06:09:54.030323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.030129+0000","last_change":"2023-11-24T06:09:54.030129+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.030129+0000","last_undegraded":"2023-11-24T06:09:54.030129+0000","last_fullsized":"2023-11-24T06:09:54.030129+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.028576+0000","last_change":"2023-11-24T06:09:54.028576+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.028576+0000","last_undegraded":"2023-11-24T06:09:54.028576+0000","last_fullsized":"2023-11-24T06:09:54.028576+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.028797+0000","last_change":"2023-11-24T06:09:54.028797+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.028797+0000","last_undegraded":"2023-11-24T06:09:54.028797+0000","last_fullsized":"2023-11-24T06:09:54.028797+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.031362+0000","last_change":"2023-11-24T06:09:54.031362+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.031362+0000","last_undegraded":"2023-11-24T06:09:54.031362+0000","last_fullsized":"2023-11-24T06:09:54.031362+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.028408+0000","last_change":"2023-11-24T06:09:54.028408+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.028408+0000","last_undegraded":"2023-11-24T06:09:54.028408+0000","last_fullsized":"2023-11-24T06:09:54.028408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.030389+0000","last_change":"2023-11-24T06:09:54.030389+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.030389+0000","last_undegraded":"2023-11-24T06:09:54.030389+0000","last_fullsized":"2023-11-24T06:09:54.030389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.026703+0000","last_change":"2023-11-24T06:09:54.026703+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.026703+0000","last_undegraded":"2023-11-24T06:09:54.026703+0000","last_fullsized":"2023-11-24T06:09:54.026703+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.027202+0000","last_change":"2023-11-24T06:09:54.027202+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.027202+0000","last_undegraded":"2023-11-24T06:09:54.027202+0000","last_fullsized":"2023-11-24T06:09:54.027202+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.030895+0000","last_change":"2023-11-24T06:09:54.030895+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.030895+0000","last_undegraded":"2023-11-24T06:09:54.030895+0000","last_fullsized":"2023-11-24T06:09:54.030895+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.027376+0000","last_change":"2023-11-24T06:09:54.027376+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.027376+0000","last_undegraded":"2023-11-24T06:09:54.027376+0000","last_fullsized":"2023-11-24T06:09:54.027376+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.028800+0000","last_change":"2023-11-24T06:09:54.028800+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.028800+0000","last_undegraded":"2023-11-24T06:09:54.028800+0000","last_fullsized":"2023-11-24T06:09:54.028800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.030623+0000","last_change":"2023-11-24T06:09:54.030623+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.030623+0000","last_undegraded":"2023-11-24T06:09:54.030623+0000","last_fullsized":"2023-11-24T06:09:54.030623+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.031350+0000","last_change":"2023-11-24T06:09:54.031350+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.031350+0000","last_undegraded":"2023-11-24T06:09:54.031350+0000","last_fullsized":"2023-11-24T06:09:54.031350+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.029528+0000","last_change":"2023-11-24T06:09:54.029528+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.029528+0000","last_undegraded":"2023-11-24T06:09:54.029528+0000","last_fullsized":"2023-11-24T06:09:54.029528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.027936+0000","last_change":"2023-11-24T06:09:54.027936+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.027936+0000","last_undegraded":"2023-11-24T06:09:54.027936+0000","last_fullsized":"2023-11-24T06:09:54.027936+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.027177+0000","last_change":"2023-11-24T06:09:54.027177+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.027177+0000","last_undegraded":"2023-11-24T06:09:54.027177+0000","last_fullsized":"2023-11-24T06:09:54.027177+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.027128+0000","last_change":"2023-11-24T06:09:54.027128+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.027128+0000","last_undegraded":"2023-11-24T06:09:54.027128+0000","last_fullsized":"2023-11-24T06:09:54.027128+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.026636+0000","last_change":"2023-11-24T06:09:54.026636+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.026636+0000","last_undegraded":"2023-11-24T06:09:54.026636+0000","last_fullsized":"2023-11-24T06:09:54.026636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.030056+0000","last_change":"2023-11-24T06:09:54.030056+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.030056+0000","last_undegraded":"2023-11-24T06:09:54.030056+0000","last_fullsized":"2023-11-24T06:09:54.030056+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.028709+0000","last_change":"2023-11-24T06:09:54.028709+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.028709+0000","last_undegraded":"2023-11-24T06:09:54.028709+0000","last_fullsized":"2023-11-24T06:09:54.028709+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.027206+0000","last_change":"2023-11-24T06:09:54.027206+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.027206+0000","last_undegraded":"2023-11-24T06:09:54.027206+0000","last_fullsized":"2023-11-24T06:09:54.027206+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.029000+0000","last_change":"2023-11-24T06:09:54.029000+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.029000+0000","last_undegraded":"2023-11-24T06:09:54.029000+0000","last_fullsized":"2023-11-24T06:09:54.029000+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.026105+0000","last_change":"2023-11-24T06:09:54.026105+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.026105+0000","last_undegraded":"2023-11-24T06:09:54.026105+0000","last_fullsized":"2023-11-24T06:09:54.026105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.029122+0000","last_change":"2023-11-24T06:09:54.029122+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.029122+0000","last_undegraded":"2023-11-24T06:09:54.029122+0000","last_fullsized":"2023-11-24T06:09:54.029122+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.026114+0000","last_change":"2023-11-24T06:09:54.026114+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.026114+0000","last_undegraded":"2023-11-24T06:09:54.026114+0000","last_fullsized":"2023-11-24T06:09:54.026114+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.029564+0000","last_change":"2023-11-24T06:09:54.029564+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.029564+0000","last_undegraded":"2023-11-24T06:09:54.029564+0000","last_fullsized":"2023-11-24T06:09:54.029564+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.027468+0000","last_change":"2023-11-24T06:09:54.027468+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.027468+0000","last_undegraded":"2023-11-24T06:09:54.027468+0000","last_fullsized":"2023-11-24T06:09:54.027468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.029693+0000","last_change":"2023-11-24T06:09:54.029693+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.029693+0000","last_undegraded":"2023-11-24T06:09:54.029693+0000","last_fullsized":"2023-11-24T06:09:54.029693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:09:55.022757+0000","last_change":"2023-11-24T06:09:55.022757+0000","last_active":"2023-11-24T06:09:55.022757+0000","last_peered":"2023-11-24T06:09:55.022757+0000","last_clean":"2023-11-24T06:09:55.022757+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:55.022757+0000","last_undegraded":"2023-11-24T06:09:55.022757+0000","last_fullsized":"2023-11-24T06:09:55.022757+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2023-11-24T06:09:54.030058+0000","last_change":"2023-11-24T06:09:54.030058+0000","last_active":"2023-11-24T06:09:54.014422+0000","last_peered":"2023-11-24T06:09:54.014422+0000","last_clean":"2023-11-24T06:09:54.014422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:09:54.030058+0000","last_undegraded":"2023-11-24T06:09:54.030058+0000","last_fullsized":"2023-11-24T06:09:54.030058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":172,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-11-24T06:09:49.187279+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:09:49.187279+0000","last_peered":"2023-11-24T06:09:49.187279+0000","last_clean":"2023-11-24T06:09:49.187279+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:09:49.187279+0000","last_undegraded":"2023-11-24T06:09:49.187279+0000","last_fullsized":"2023-11-24T06:09:49.187279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":717618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42199999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.624}]},{"osd":6,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":323584,"data_stored":121433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":8.4849999999999994,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.625},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.28299999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.311}]},{"osd":4,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":711801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":12.378,"5min":12.378,"15min":12.378},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.38}]},{"osd":2,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.5199999999999996,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.32100000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.371}]},{"osd":4,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":15.319000000000001,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.7590000000000003,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.45900000000000002},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":15.356,"5min":15.356,"15min":15.356},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.726,"5min":4.726,"15min":4.726},"last":0.501}]},{"osd":7,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":4.7869999999999999,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":15.388,"5min":15.388,"15min":15.388},"last":0.64600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":121433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.28599999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.372}]},{"osd":3,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41499999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.501}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.45600000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.43099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.315},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.38}]},{"osd":2,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.92400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.61599999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.497},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.64800000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":121433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.335,"5min":3.335,"15min":3.335},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.438}]},{"osd":1,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":4.4489999999999998,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.55000000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.5069999999999997,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:10:00.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:10:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.302+0000 7ffb2d7a0700 1 Processor -- start 2023-11-24T06:10:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.302+0000 7ffb2d7a0700 1 -- start start 2023-11-24T06:10:00.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb2d7a0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb280b2b60 0x7ffb280ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb2d7a0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280cf0d0 0x7ffb280d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb2d7a0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2806bc00 con 0x7ffb2815e0f0 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb2d7a0700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb280cf560 con 0x7ffb280b2b60 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb2d7a0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2815e4d0 con 0x7ffb280cf0d0 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb267fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb280b2b60 0x7ffb280ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb267fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffb280b2b60 0x7ffb280ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47036/0 (socket says 172.21.15.97:47036) 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb267fc700 1 -- 172.21.15.97:0/3860113145 learned_addr learned my addr 172.21.15.97:0/3860113145 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb277fe700 1 --2- 172.21.15.97:0/3860113145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280cf0d0 0x7ffb280d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb267fc700 1 -- 172.21.15.97:0/3860113145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280cf0d0 msgr2=0x7ffb280d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb267fc700 1 --2- 172.21.15.97:0/3860113145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280cf0d0 0x7ffb280d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb267fc700 1 -- 172.21.15.97:0/3860113145 >> v1:172.21.15.183:6789/0 conn(0x7ffb2815e0f0 legacy=0x7ffb280b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:00.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.303+0000 7ffb267fc700 1 -- 172.21.15.97:0/3860113145 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2806aff0 con 0x7ffb280b2b60 2023-11-24T06:10:00.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb267fc700 1 --2- 172.21.15.97:0/3860113145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280b2b60 0x7ffb280ceb90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb180098f0 tx=0x7ffb18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb25a4d05c3c2985 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:00.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb25ffb700 1 -- 172.21.15.97:0/3860113145 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18004230 con 0x7ffb280b2b60 2023-11-24T06:10:00.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb25ffb700 1 -- 172.21.15.97:0/3860113145 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb18004390 con 0x7ffb280b2b60 2023-11-24T06:10:00.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb25ffb700 1 -- 172.21.15.97:0/3860113145 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb180036a0 con 0x7ffb280b2b60 2023-11-24T06:10:00.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/3860113145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280b2b60 msgr2=0x7ffb280ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/3860113145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280b2b60 0x7ffb280ceb90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffb180098f0 tx=0x7ffb18009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/3860113145 shutdown_connections 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/3860113145 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280cf0d0 0x7ffb280d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/3860113145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280b2b60 0x7ffb280ceb90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/3860113145 >> 172.21.15.97:0/3860113145 conn(0x7ffb280b5ce0 msgr2=0x7ffb280b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/3860113145 shutdown_connections 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.304+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/3860113145 wait complete. 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.306+0000 7ffb2d7a0700 1 Processor -- start 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.306+0000 7ffb2d7a0700 1 -- start start 2023-11-24T06:10:00.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.306+0000 7ffb2d7a0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280b2b60 0x7ffb2814c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.306+0000 7ffb2d7a0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb280cf0d0 0x7ffb280cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.306+0000 7ffb2d7a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb2815e0f0 0x7ffb280cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.306+0000 7ffb2d7a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280c8c50 con 0x7ffb2815e0f0 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.306+0000 7ffb2d7a0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280cc440 con 0x7ffb280cf0d0 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.306+0000 7ffb2d7a0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280cd280 con 0x7ffb280b2b60 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb2815e0f0 0x7ffb280cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb2815e0f0 0x7ffb280cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47046/0 (socket says 172.21.15.97:47046) 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 -- 172.21.15.97:0/1223516242 learned_addr learned my addr 172.21.15.97:0/1223516242 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb267fc700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb280cf0d0 0x7ffb280cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb26ffd700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280b2b60 0x7ffb2814c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 -- 172.21.15.97:0/1223516242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280b2b60 msgr2=0x7ffb2814c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280b2b60 0x7ffb2814c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 -- 172.21.15.97:0/1223516242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb280cf0d0 msgr2=0x7ffb280cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb280cf0d0 0x7ffb280cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 -- 172.21.15.97:0/1223516242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18009550 con 0x7ffb2815e0f0 2023-11-24T06:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb277fe700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb2815e0f0 0x7ffb280cd710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb20007e40 tx=0x7ffb2000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb0f7fe700 1 -- 172.21.15.97:0/1223516242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2000eea0 con 0x7ffb2815e0f0 2023-11-24T06:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb0f7fe700 1 -- 172.21.15.97:0/1223516242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb20014420 con 0x7ffb2815e0f0 2023-11-24T06:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb280cc670 con 0x7ffb2815e0f0 2023-11-24T06:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb0f7fe700 1 -- 172.21.15.97:0/1223516242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb200147b0 con 0x7ffb2815e0f0 2023-11-24T06:10:00.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.307+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb280ccc30 con 0x7ffb2815e0f0 2023-11-24T06:10:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.308+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb280751e0 con 0x7ffb2815e0f0 2023-11-24T06:10:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.309+0000 7ffb0f7fe700 1 -- 172.21.15.97:0/1223516242 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb20014910 con 0x7ffb2815e0f0 2023-11-24T06:10:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.309+0000 7ffb0f7fe700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb1406aed0 0x7ffb1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.309+0000 7ffb0f7fe700 1 -- 172.21.15.97:0/1223516242 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb20089d10 con 0x7ffb2815e0f0 2023-11-24T06:10:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.309+0000 7ffb26ffd700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb1406aed0 0x7ffb1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.309+0000 7ffb26ffd700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb1406aed0 0x7ffb1406d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00a7d0 tx=0x7ffb1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:00.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.313+0000 7ffb0f7fe700 1 -- 172.21.15.97:0/1223516242 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb20054e50 con 0x7ffb2815e0f0 2023-11-24T06:10:00.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.500+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb2806b890 con 0x7ffb1406aed0 2023-11-24T06:10:00.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.508+0000 7ffb0f7fe700 1 -- 172.21.15.97:0/1223516242 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289181 (secure 0 0 0) 0x7ffb1c0113d0 con 0x7ffb1406aed0 2023-11-24T06:10:00.514 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:00.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb1406aed0 msgr2=0x7ffb1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb1406aed0 0x7ffb1406d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00a7d0 tx=0x7ffb1c008040 comp rx=0 tx=0).stop 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb2815e0f0 msgr2=0x7ffb280cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb2815e0f0 0x7ffb280cd710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffb20007e40 tx=0x7ffb2000e3c0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 shutdown_connections 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb2815e0f0 0x7ffb280cd710 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb1406aed0 0x7ffb1406d390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb280b2b60 0x7ffb2814c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 --2- 172.21.15.97:0/1223516242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb280cf0d0 0x7ffb280cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 >> 172.21.15.97:0/1223516242 conn(0x7ffb280b5ce0 msgr2=0x7ffb280b92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 shutdown_connections 2023-11-24T06:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.511+0000 7ffb2d7a0700 1 -- 172.21.15.97:0/1223516242 wait complete. 2023-11-24T06:10:00.518 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:10:00.527 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-11-24T06:10:00.337672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220336,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754384,"statfs":{"total":773094113280,"available":772868489216,"internally_reserved":0,"allocated":4235264,"data_stored":2358292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000944"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797864+0000","last_change":"2023-11-24T06:09:56.058235+0000","last_active":"2023-11-24T06:09:57.797864+0000","last_peered":"2023-11-24T06:09:57.797864+0000","last_clean":"2023-11-24T06:09:57.797864+0000","last_became_active":"2023-11-24T06:09:56.057943+0000","last_became_peered":"2023-11-24T06:09:56.057943+0000","last_unstale":"2023-11-24T06:09:57.797864+0000","last_undegraded":"2023-11-24T06:09:57.797864+0000","last_fullsized":"2023-11-24T06:09:57.797864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:25.766325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179912+0000","last_change":"2023-11-24T06:09:56.049370+0000","last_active":"2023-11-24T06:09:57.179912+0000","last_peered":"2023-11-24T06:09:57.179912+0000","last_clean":"2023-11-24T06:09:57.179912+0000","last_became_active":"2023-11-24T06:09:56.049108+0000","last_became_peered":"2023-11-24T06:09:56.049108+0000","last_unstale":"2023-11-24T06:09:57.179912+0000","last_undegraded":"2023-11-24T06:09:57.179912+0000","last_fullsized":"2023-11-24T06:09:57.179912+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:25.333454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.051002+0000","last_change":"2023-11-24T06:09:56.059807+0000","last_active":"2023-11-24T06:09:57.051002+0000","last_peered":"2023-11-24T06:09:57.051002+0000","last_clean":"2023-11-24T06:09:57.051002+0000","last_became_active":"2023-11-24T06:09:56.059477+0000","last_became_peered":"2023-11-24T06:09:56.059477+0000","last_unstale":"2023-11-24T06:09:57.051002+0000","last_undegraded":"2023-11-24T06:09:57.051002+0000","last_fullsized":"2023-11-24T06:09:57.051002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:13.452656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.674474+0000","last_change":"2023-11-24T06:09:56.054209+0000","last_active":"2023-11-24T06:09:57.674474+0000","last_peered":"2023-11-24T06:09:57.674474+0000","last_clean":"2023-11-24T06:09:57.674474+0000","last_became_active":"2023-11-24T06:09:56.050894+0000","last_became_peered":"2023-11-24T06:09:56.050894+0000","last_unstale":"2023-11-24T06:09:57.674474+0000","last_undegraded":"2023-11-24T06:09:57.674474+0000","last_fullsized":"2023-11-24T06:09:57.674474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:09.036629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.187695+0000","last_change":"2023-11-24T06:09:56.057348+0000","last_active":"2023-11-24T06:09:57.187695+0000","last_peered":"2023-11-24T06:09:57.187695+0000","last_clean":"2023-11-24T06:09:57.187695+0000","last_became_active":"2023-11-24T06:09:56.057099+0000","last_became_peered":"2023-11-24T06:09:56.057099+0000","last_unstale":"2023-11-24T06:09:57.187695+0000","last_undegraded":"2023-11-24T06:09:57.187695+0000","last_fullsized":"2023-11-24T06:09:57.187695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:00.753408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179538+0000","last_change":"2023-11-24T06:09:56.047556+0000","last_active":"2023-11-24T06:09:57.179538+0000","last_peered":"2023-11-24T06:09:57.179538+0000","last_clean":"2023-11-24T06:09:57.179538+0000","last_became_active":"2023-11-24T06:09:56.047245+0000","last_became_peered":"2023-11-24T06:09:56.047245+0000","last_unstale":"2023-11-24T06:09:57.179538+0000","last_undegraded":"2023-11-24T06:09:57.179538+0000","last_fullsized":"2023-11-24T06:09:57.179538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:09.461169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.798008+0000","last_change":"2023-11-24T06:09:56.059262+0000","last_active":"2023-11-24T06:09:57.798008+0000","last_peered":"2023-11-24T06:09:57.798008+0000","last_clean":"2023-11-24T06:09:57.798008+0000","last_became_active":"2023-11-24T06:09:56.058975+0000","last_became_peered":"2023-11-24T06:09:56.058975+0000","last_unstale":"2023-11-24T06:09:57.798008+0000","last_undegraded":"2023-11-24T06:09:57.798008+0000","last_fullsized":"2023-11-24T06:09:57.798008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:33:07.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797270+0000","last_change":"2023-11-24T06:09:56.053650+0000","last_active":"2023-11-24T06:09:57.797270+0000","last_peered":"2023-11-24T06:09:57.797270+0000","last_clean":"2023-11-24T06:09:57.797270+0000","last_became_active":"2023-11-24T06:09:56.053340+0000","last_became_peered":"2023-11-24T06:09:56.053340+0000","last_unstale":"2023-11-24T06:09:57.797270+0000","last_undegraded":"2023-11-24T06:09:57.797270+0000","last_fullsized":"2023-11-24T06:09:57.797270+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:36.494812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.053088+0000","last_change":"2023-11-24T06:09:56.056615+0000","last_active":"2023-11-24T06:09:57.053088+0000","last_peered":"2023-11-24T06:09:57.053088+0000","last_clean":"2023-11-24T06:09:57.053088+0000","last_became_active":"2023-11-24T06:09:56.056352+0000","last_became_peered":"2023-11-24T06:09:56.056352+0000","last_unstale":"2023-11-24T06:09:57.053088+0000","last_undegraded":"2023-11-24T06:09:57.053088+0000","last_fullsized":"2023-11-24T06:09:57.053088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:09.036629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.674629+0000","last_change":"2023-11-24T06:09:56.055681+0000","last_active":"2023-11-24T06:09:57.674629+0000","last_peered":"2023-11-24T06:09:57.674629+0000","last_clean":"2023-11-24T06:09:57.674629+0000","last_became_active":"2023-11-24T06:09:56.055351+0000","last_became_peered":"2023-11-24T06:09:56.055351+0000","last_unstale":"2023-11-24T06:09:57.674629+0000","last_undegraded":"2023-11-24T06:09:57.674629+0000","last_fullsized":"2023-11-24T06:09:57.674629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:59.321450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797927+0000","last_change":"2023-11-24T06:09:56.060377+0000","last_active":"2023-11-24T06:09:57.797927+0000","last_peered":"2023-11-24T06:09:57.797927+0000","last_clean":"2023-11-24T06:09:57.797927+0000","last_became_active":"2023-11-24T06:09:56.059808+0000","last_became_peered":"2023-11-24T06:09:56.059808+0000","last_unstale":"2023-11-24T06:09:57.797927+0000","last_undegraded":"2023-11-24T06:09:57.797927+0000","last_fullsized":"2023-11-24T06:09:57.797927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:37.946793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.051290+0000","last_change":"2023-11-24T06:09:56.051066+0000","last_active":"2023-11-24T06:09:57.051290+0000","last_peered":"2023-11-24T06:09:57.051290+0000","last_clean":"2023-11-24T06:09:57.051290+0000","last_became_active":"2023-11-24T06:09:56.050662+0000","last_became_peered":"2023-11-24T06:09:56.050662+0000","last_unstale":"2023-11-24T06:09:57.051290+0000","last_undegraded":"2023-11-24T06:09:57.051290+0000","last_fullsized":"2023-11-24T06:09:57.051290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:33:07.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.187835+0000","last_change":"2023-11-24T06:09:56.059431+0000","last_active":"2023-11-24T06:09:57.187835+0000","last_peered":"2023-11-24T06:09:57.187835+0000","last_clean":"2023-11-24T06:09:57.187835+0000","last_became_active":"2023-11-24T06:09:56.058620+0000","last_became_peered":"2023-11-24T06:09:56.058620+0000","last_unstale":"2023-11-24T06:09:57.187835+0000","last_undegraded":"2023-11-24T06:09:57.187835+0000","last_fullsized":"2023-11-24T06:09:57.187835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:18:41.220659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179577+0000","last_change":"2023-11-24T06:09:56.053335+0000","last_active":"2023-11-24T06:09:57.179577+0000","last_peered":"2023-11-24T06:09:57.179577+0000","last_clean":"2023-11-24T06:09:57.179577+0000","last_became_active":"2023-11-24T06:09:56.053049+0000","last_became_peered":"2023-11-24T06:09:56.053049+0000","last_unstale":"2023-11-24T06:09:57.179577+0000","last_undegraded":"2023-11-24T06:09:57.179577+0000","last_fullsized":"2023-11-24T06:09:57.179577+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:01.712292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.050905+0000","last_change":"2023-11-24T06:09:56.058231+0000","last_active":"2023-11-24T06:09:57.050905+0000","last_peered":"2023-11-24T06:09:57.050905+0000","last_clean":"2023-11-24T06:09:57.050905+0000","last_became_active":"2023-11-24T06:09:56.057923+0000","last_became_peered":"2023-11-24T06:09:56.057923+0000","last_unstale":"2023-11-24T06:09:57.050905+0000","last_undegraded":"2023-11-24T06:09:57.050905+0000","last_fullsized":"2023-11-24T06:09:57.050905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:59.321450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797571+0000","last_change":"2023-11-24T06:09:56.055802+0000","last_active":"2023-11-24T06:09:57.797571+0000","last_peered":"2023-11-24T06:09:57.797571+0000","last_clean":"2023-11-24T06:09:57.797571+0000","last_became_active":"2023-11-24T06:09:56.055481+0000","last_became_peered":"2023-11-24T06:09:56.055481+0000","last_unstale":"2023-11-24T06:09:57.797571+0000","last_undegraded":"2023-11-24T06:09:57.797571+0000","last_fullsized":"2023-11-24T06:09:57.797571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:57:29.925125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797690+0000","last_change":"2023-11-24T06:09:56.051729+0000","last_active":"2023-11-24T06:09:57.797690+0000","last_peered":"2023-11-24T06:09:57.797690+0000","last_clean":"2023-11-24T06:09:57.797690+0000","last_became_active":"2023-11-24T06:09:56.051428+0000","last_became_peered":"2023-11-24T06:09:56.051428+0000","last_unstale":"2023-11-24T06:09:57.797690+0000","last_undegraded":"2023-11-24T06:09:57.797690+0000","last_fullsized":"2023-11-24T06:09:57.797690+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:21.150064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179939+0000","last_change":"2023-11-24T06:09:56.054485+0000","last_active":"2023-11-24T06:09:57.179939+0000","last_peered":"2023-11-24T06:09:57.179939+0000","last_clean":"2023-11-24T06:09:57.179939+0000","last_became_active":"2023-11-24T06:09:56.054216+0000","last_became_peered":"2023-11-24T06:09:56.054216+0000","last_unstale":"2023-11-24T06:09:57.179939+0000","last_undegraded":"2023-11-24T06:09:57.179939+0000","last_fullsized":"2023-11-24T06:09:57.179939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:49.350500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179829+0000","last_change":"2023-11-24T06:09:56.059807+0000","last_active":"2023-11-24T06:09:57.179829+0000","last_peered":"2023-11-24T06:09:57.179829+0000","last_clean":"2023-11-24T06:09:57.179829+0000","last_became_active":"2023-11-24T06:09:56.059481+0000","last_became_peered":"2023-11-24T06:09:56.059481+0000","last_unstale":"2023-11-24T06:09:57.179829+0000","last_undegraded":"2023-11-24T06:09:57.179829+0000","last_fullsized":"2023-11-24T06:09:57.179829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:07.981397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.187834+0000","last_change":"2023-11-24T06:09:56.058807+0000","last_active":"2023-11-24T06:09:57.187834+0000","last_peered":"2023-11-24T06:09:57.187834+0000","last_clean":"2023-11-24T06:09:57.187834+0000","last_became_active":"2023-11-24T06:09:56.058264+0000","last_became_peered":"2023-11-24T06:09:56.058264+0000","last_unstale":"2023-11-24T06:09:57.187834+0000","last_undegraded":"2023-11-24T06:09:57.187834+0000","last_fullsized":"2023-11-24T06:09:57.187834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:11:36.391859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.052422+0000","last_change":"2023-11-24T06:09:56.049330+0000","last_active":"2023-11-24T06:09:57.052422+0000","last_peered":"2023-11-24T06:09:57.052422+0000","last_clean":"2023-11-24T06:09:57.052422+0000","last_became_active":"2023-11-24T06:09:56.049078+0000","last_became_peered":"2023-11-24T06:09:56.049078+0000","last_unstale":"2023-11-24T06:09:57.052422+0000","last_undegraded":"2023-11-24T06:09:57.052422+0000","last_fullsized":"2023-11-24T06:09:57.052422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:37.909661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.052465+0000","last_change":"2023-11-24T06:09:56.044323+0000","last_active":"2023-11-24T06:09:57.052465+0000","last_peered":"2023-11-24T06:09:57.052465+0000","last_clean":"2023-11-24T06:09:57.052465+0000","last_became_active":"2023-11-24T06:09:56.043953+0000","last_became_peered":"2023-11-24T06:09:56.043953+0000","last_unstale":"2023-11-24T06:09:57.052465+0000","last_undegraded":"2023-11-24T06:09:57.052465+0000","last_fullsized":"2023-11-24T06:09:57.052465+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:25.634889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.048039+0000","last_change":"2023-11-24T06:09:56.054623+0000","last_active":"2023-11-24T06:09:57.048039+0000","last_peered":"2023-11-24T06:09:57.048039+0000","last_clean":"2023-11-24T06:09:57.048039+0000","last_became_active":"2023-11-24T06:09:56.054364+0000","last_became_peered":"2023-11-24T06:09:56.054364+0000","last_unstale":"2023-11-24T06:09:57.048039+0000","last_undegraded":"2023-11-24T06:09:57.048039+0000","last_fullsized":"2023-11-24T06:09:57.048039+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:25.766325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.050643+0000","last_change":"2023-11-24T06:09:56.054064+0000","last_active":"2023-11-24T06:09:57.050643+0000","last_peered":"2023-11-24T06:09:57.050643+0000","last_clean":"2023-11-24T06:09:57.050643+0000","last_became_active":"2023-11-24T06:09:56.053781+0000","last_became_peered":"2023-11-24T06:09:56.053781+0000","last_unstale":"2023-11-24T06:09:57.050643+0000","last_undegraded":"2023-11-24T06:09:57.050643+0000","last_fullsized":"2023-11-24T06:09:57.050643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:57:29.925125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.674827+0000","last_change":"2023-11-24T06:09:56.053857+0000","last_active":"2023-11-24T06:09:57.674827+0000","last_peered":"2023-11-24T06:09:57.674827+0000","last_clean":"2023-11-24T06:09:57.674827+0000","last_became_active":"2023-11-24T06:09:56.052674+0000","last_became_peered":"2023-11-24T06:09:56.052674+0000","last_unstale":"2023-11-24T06:09:57.674827+0000","last_undegraded":"2023-11-24T06:09:57.674827+0000","last_fullsized":"2023-11-24T06:09:57.674827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:13.452656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179985+0000","last_change":"2023-11-24T06:09:56.051992+0000","last_active":"2023-11-24T06:09:57.179985+0000","last_peered":"2023-11-24T06:09:57.179985+0000","last_clean":"2023-11-24T06:09:57.179985+0000","last_became_active":"2023-11-24T06:09:56.051684+0000","last_became_peered":"2023-11-24T06:09:56.051684+0000","last_unstale":"2023-11-24T06:09:57.179985+0000","last_undegraded":"2023-11-24T06:09:57.179985+0000","last_fullsized":"2023-11-24T06:09:57.179985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:02.505363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.674864+0000","last_change":"2023-11-24T06:09:56.054539+0000","last_active":"2023-11-24T06:09:57.674864+0000","last_peered":"2023-11-24T06:09:57.674864+0000","last_clean":"2023-11-24T06:09:57.674864+0000","last_became_active":"2023-11-24T06:09:56.050515+0000","last_became_peered":"2023-11-24T06:09:56.050515+0000","last_unstale":"2023-11-24T06:09:57.674864+0000","last_undegraded":"2023-11-24T06:09:57.674864+0000","last_fullsized":"2023-11-24T06:09:57.674864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:13:25.443710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.051297+0000","last_change":"2023-11-24T06:09:56.049350+0000","last_active":"2023-11-24T06:09:57.051297+0000","last_peered":"2023-11-24T06:09:57.051297+0000","last_clean":"2023-11-24T06:09:57.051297+0000","last_became_active":"2023-11-24T06:09:56.049047+0000","last_became_peered":"2023-11-24T06:09:56.049047+0000","last_unstale":"2023-11-24T06:09:57.051297+0000","last_undegraded":"2023-11-24T06:09:57.051297+0000","last_fullsized":"2023-11-24T06:09:57.051297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:09.461169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.052560+0000","last_change":"2023-11-24T06:09:56.045099+0000","last_active":"2023-11-24T06:09:57.052560+0000","last_peered":"2023-11-24T06:09:57.052560+0000","last_clean":"2023-11-24T06:09:57.052560+0000","last_became_active":"2023-11-24T06:09:56.044127+0000","last_became_peered":"2023-11-24T06:09:56.044127+0000","last_unstale":"2023-11-24T06:09:57.052560+0000","last_undegraded":"2023-11-24T06:09:57.052560+0000","last_fullsized":"2023-11-24T06:09:57.052560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:36.494812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797390+0000","last_change":"2023-11-24T06:09:56.053390+0000","last_active":"2023-11-24T06:09:57.797390+0000","last_peered":"2023-11-24T06:09:57.797390+0000","last_clean":"2023-11-24T06:09:57.797390+0000","last_became_active":"2023-11-24T06:09:56.052068+0000","last_became_peered":"2023-11-24T06:09:56.052068+0000","last_unstale":"2023-11-24T06:09:57.797390+0000","last_undegraded":"2023-11-24T06:09:57.797390+0000","last_fullsized":"2023-11-24T06:09:57.797390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.598642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797945+0000","last_change":"2023-11-24T06:09:56.053398+0000","last_active":"2023-11-24T06:09:57.797945+0000","last_peered":"2023-11-24T06:09:57.797945+0000","last_clean":"2023-11-24T06:09:57.797945+0000","last_became_active":"2023-11-24T06:09:56.053136+0000","last_became_peered":"2023-11-24T06:09:56.053136+0000","last_unstale":"2023-11-24T06:09:57.797945+0000","last_undegraded":"2023-11-24T06:09:57.797945+0000","last_fullsized":"2023-11-24T06:09:57.797945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:51.088028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797731+0000","last_change":"2023-11-24T06:09:56.052814+0000","last_active":"2023-11-24T06:09:57.797731+0000","last_peered":"2023-11-24T06:09:57.797731+0000","last_clean":"2023-11-24T06:09:57.797731+0000","last_became_active":"2023-11-24T06:09:56.052539+0000","last_became_peered":"2023-11-24T06:09:56.052539+0000","last_unstale":"2023-11-24T06:09:57.797731+0000","last_undegraded":"2023-11-24T06:09:57.797731+0000","last_fullsized":"2023-11-24T06:09:57.797731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:13.452656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.048223+0000","last_change":"2023-11-24T06:09:56.055152+0000","last_active":"2023-11-24T06:09:57.048223+0000","last_peered":"2023-11-24T06:09:57.048223+0000","last_clean":"2023-11-24T06:09:57.048223+0000","last_became_active":"2023-11-24T06:09:56.054918+0000","last_became_peered":"2023-11-24T06:09:56.054918+0000","last_unstale":"2023-11-24T06:09:57.048223+0000","last_undegraded":"2023-11-24T06:09:57.048223+0000","last_fullsized":"2023-11-24T06:09:57.048223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:15:27.152797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.059814+0000","last_change":"2023-11-24T06:09:55.047852+0000","last_active":"2023-11-24T06:09:56.059814+0000","last_peered":"2023-11-24T06:09:56.059814+0000","last_clean":"2023-11-24T06:09:56.059814+0000","last_became_active":"2023-11-24T06:09:55.047545+0000","last_became_peered":"2023-11-24T06:09:55.047545+0000","last_unstale":"2023-11-24T06:09:56.059814+0000","last_undegraded":"2023-11-24T06:09:56.059814+0000","last_fullsized":"2023-11-24T06:09:56.059814+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:42.441971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797882+0000","last_change":"2023-11-24T06:09:56.049796+0000","last_active":"2023-11-24T06:09:57.797882+0000","last_peered":"2023-11-24T06:09:57.797882+0000","last_clean":"2023-11-24T06:09:57.797882+0000","last_became_active":"2023-11-24T06:09:56.049539+0000","last_became_peered":"2023-11-24T06:09:56.049539+0000","last_unstale":"2023-11-24T06:09:57.797882+0000","last_undegraded":"2023-11-24T06:09:57.797882+0000","last_fullsized":"2023-11-24T06:09:57.797882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:49.632078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.054750+0000","last_change":"2023-11-24T06:09:55.051344+0000","last_active":"2023-11-24T06:09:56.054750+0000","last_peered":"2023-11-24T06:09:56.054750+0000","last_clean":"2023-11-24T06:09:56.054750+0000","last_became_active":"2023-11-24T06:09:55.051074+0000","last_became_peered":"2023-11-24T06:09:55.051074+0000","last_unstale":"2023-11-24T06:09:56.054750+0000","last_undegraded":"2023-11-24T06:09:56.054750+0000","last_fullsized":"2023-11-24T06:09:56.054750+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:18:40.212324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.052578+0000","last_change":"2023-11-24T06:09:56.051861+0000","last_active":"2023-11-24T06:09:57.052578+0000","last_peered":"2023-11-24T06:09:57.052578+0000","last_clean":"2023-11-24T06:09:57.052578+0000","last_became_active":"2023-11-24T06:09:56.051584+0000","last_became_peered":"2023-11-24T06:09:56.051584+0000","last_unstale":"2023-11-24T06:09:57.052578+0000","last_undegraded":"2023-11-24T06:09:57.052578+0000","last_fullsized":"2023-11-24T06:09:57.052578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.598642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.060125+0000","last_change":"2023-11-24T06:09:55.048049+0000","last_active":"2023-11-24T06:09:56.060125+0000","last_peered":"2023-11-24T06:09:56.060125+0000","last_clean":"2023-11-24T06:09:56.060125+0000","last_became_active":"2023-11-24T06:09:55.047780+0000","last_became_peered":"2023-11-24T06:09:55.047780+0000","last_unstale":"2023-11-24T06:09:56.060125+0000","last_undegraded":"2023-11-24T06:09:56.060125+0000","last_fullsized":"2023-11-24T06:09:56.060125+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:24.626555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.674880+0000","last_change":"2023-11-24T06:09:56.050311+0000","last_active":"2023-11-24T06:09:57.674880+0000","last_peered":"2023-11-24T06:09:57.674880+0000","last_clean":"2023-11-24T06:09:57.674880+0000","last_became_active":"2023-11-24T06:09:56.049706+0000","last_became_peered":"2023-11-24T06:09:56.049706+0000","last_unstale":"2023-11-24T06:09:57.674880+0000","last_undegraded":"2023-11-24T06:09:57.674880+0000","last_fullsized":"2023-11-24T06:09:57.674880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:32.236252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.058954+0000","last_change":"2023-11-24T06:09:55.044380+0000","last_active":"2023-11-24T06:09:56.058954+0000","last_peered":"2023-11-24T06:09:56.058954+0000","last_clean":"2023-11-24T06:09:56.058954+0000","last_became_active":"2023-11-24T06:09:55.044078+0000","last_became_peered":"2023-11-24T06:09:55.044078+0000","last_unstale":"2023-11-24T06:09:56.058954+0000","last_undegraded":"2023-11-24T06:09:56.058954+0000","last_fullsized":"2023-11-24T06:09:56.058954+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:29:35.678333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797871+0000","last_change":"2023-11-24T06:09:56.052405+0000","last_active":"2023-11-24T06:09:57.797871+0000","last_peered":"2023-11-24T06:09:57.797871+0000","last_clean":"2023-11-24T06:09:57.797871+0000","last_became_active":"2023-11-24T06:09:56.051777+0000","last_became_peered":"2023-11-24T06:09:56.051777+0000","last_unstale":"2023-11-24T06:09:57.797871+0000","last_undegraded":"2023-11-24T06:09:57.797871+0000","last_fullsized":"2023-11-24T06:09:57.797871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:32.236252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.062855+0000","last_change":"2023-11-24T06:09:55.046488+0000","last_active":"2023-11-24T06:09:56.062855+0000","last_peered":"2023-11-24T06:09:56.062855+0000","last_clean":"2023-11-24T06:09:56.062855+0000","last_became_active":"2023-11-24T06:09:55.046168+0000","last_became_peered":"2023-11-24T06:09:55.046168+0000","last_unstale":"2023-11-24T06:09:56.062855+0000","last_undegraded":"2023-11-24T06:09:56.062855+0000","last_fullsized":"2023-11-24T06:09:56.062855+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:36:36.511404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.050874+0000","last_change":"2023-11-24T06:09:56.049811+0000","last_active":"2023-11-24T06:09:57.050874+0000","last_peered":"2023-11-24T06:09:57.050874+0000","last_clean":"2023-11-24T06:09:57.050874+0000","last_became_active":"2023-11-24T06:09:56.049551+0000","last_became_peered":"2023-11-24T06:09:56.049551+0000","last_unstale":"2023-11-24T06:09:57.050874+0000","last_undegraded":"2023-11-24T06:09:57.050874+0000","last_fullsized":"2023-11-24T06:09:57.050874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:09.036629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.062678+0000","last_change":"2023-11-24T06:09:55.046398+0000","last_active":"2023-11-24T06:09:56.062678+0000","last_peered":"2023-11-24T06:09:56.062678+0000","last_clean":"2023-11-24T06:09:56.062678+0000","last_became_active":"2023-11-24T06:09:55.046045+0000","last_became_peered":"2023-11-24T06:09:55.046045+0000","last_unstale":"2023-11-24T06:09:56.062678+0000","last_undegraded":"2023-11-24T06:09:56.062678+0000","last_fullsized":"2023-11-24T06:09:56.062678+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:56.459027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.052229+0000","last_change":"2023-11-24T06:09:56.055266+0000","last_active":"2023-11-24T06:09:57.052229+0000","last_peered":"2023-11-24T06:09:57.052229+0000","last_clean":"2023-11-24T06:09:57.052229+0000","last_became_active":"2023-11-24T06:09:56.052692+0000","last_became_peered":"2023-11-24T06:09:56.052692+0000","last_unstale":"2023-11-24T06:09:57.052229+0000","last_undegraded":"2023-11-24T06:09:57.052229+0000","last_fullsized":"2023-11-24T06:09:57.052229+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:20.772641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.035079+0000","last_change":"2023-11-24T06:09:55.048557+0000","last_active":"2023-11-24T06:09:56.035079+0000","last_peered":"2023-11-24T06:09:56.035079+0000","last_clean":"2023-11-24T06:09:56.035079+0000","last_became_active":"2023-11-24T06:09:55.048306+0000","last_became_peered":"2023-11-24T06:09:55.048306+0000","last_unstale":"2023-11-24T06:09:56.035079+0000","last_undegraded":"2023-11-24T06:09:56.035079+0000","last_fullsized":"2023-11-24T06:09:56.035079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:13:24.435376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.674325+0000","last_change":"2023-11-24T06:09:56.054004+0000","last_active":"2023-11-24T06:09:57.674325+0000","last_peered":"2023-11-24T06:09:57.674325+0000","last_clean":"2023-11-24T06:09:57.674325+0000","last_became_active":"2023-11-24T06:09:56.050827+0000","last_became_peered":"2023-11-24T06:09:56.050827+0000","last_unstale":"2023-11-24T06:09:57.674325+0000","last_undegraded":"2023-11-24T06:09:57.674325+0000","last_fullsized":"2023-11-24T06:09:57.674325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.598642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.036910+0000","last_change":"2023-11-24T06:09:55.042354+0000","last_active":"2023-11-24T06:09:56.036910+0000","last_peered":"2023-11-24T06:09:56.036910+0000","last_clean":"2023-11-24T06:09:56.036910+0000","last_became_active":"2023-11-24T06:09:55.042069+0000","last_became_peered":"2023-11-24T06:09:55.042069+0000","last_unstale":"2023-11-24T06:09:56.036910+0000","last_undegraded":"2023-11-24T06:09:56.036910+0000","last_fullsized":"2023-11-24T06:09:56.036910+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:53:04.156056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.051132+0000","last_change":"2023-11-24T06:09:56.046268+0000","last_active":"2023-11-24T06:09:57.051132+0000","last_peered":"2023-11-24T06:09:57.051132+0000","last_clean":"2023-11-24T06:09:57.051132+0000","last_became_active":"2023-11-24T06:09:56.045974+0000","last_became_peered":"2023-11-24T06:09:56.045974+0000","last_unstale":"2023-11-24T06:09:57.051132+0000","last_undegraded":"2023-11-24T06:09:57.051132+0000","last_fullsized":"2023-11-24T06:09:57.051132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:32.236252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.034647+0000","last_change":"2023-11-24T06:09:55.051372+0000","last_active":"2023-11-24T06:09:56.034647+0000","last_peered":"2023-11-24T06:09:56.034647+0000","last_clean":"2023-11-24T06:09:56.034647+0000","last_became_active":"2023-11-24T06:09:55.051115+0000","last_became_peered":"2023-11-24T06:09:55.051115+0000","last_unstale":"2023-11-24T06:09:56.034647+0000","last_undegraded":"2023-11-24T06:09:56.034647+0000","last_fullsized":"2023-11-24T06:09:56.034647+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:08.028294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179900+0000","last_change":"2023-11-24T06:09:56.054550+0000","last_active":"2023-11-24T06:09:57.179900+0000","last_peered":"2023-11-24T06:09:57.179900+0000","last_clean":"2023-11-24T06:09:57.179900+0000","last_became_active":"2023-11-24T06:09:56.054269+0000","last_became_peered":"2023-11-24T06:09:56.054269+0000","last_unstale":"2023-11-24T06:09:57.179900+0000","last_undegraded":"2023-11-24T06:09:57.179900+0000","last_fullsized":"2023-11-24T06:09:57.179900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:16.307481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.044054+0000","last_change":"2023-11-24T06:09:55.048495+0000","last_active":"2023-11-24T06:09:56.044054+0000","last_peered":"2023-11-24T06:09:56.044054+0000","last_clean":"2023-11-24T06:09:56.044054+0000","last_became_active":"2023-11-24T06:09:55.048333+0000","last_became_peered":"2023-11-24T06:09:55.048333+0000","last_unstale":"2023-11-24T06:09:56.044054+0000","last_undegraded":"2023-11-24T06:09:56.044054+0000","last_fullsized":"2023-11-24T06:09:56.044054+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:37.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179751+0000","last_change":"2023-11-24T06:09:56.051976+0000","last_active":"2023-11-24T06:09:57.179751+0000","last_peered":"2023-11-24T06:09:57.179751+0000","last_clean":"2023-11-24T06:09:57.179751+0000","last_became_active":"2023-11-24T06:09:56.051688+0000","last_became_peered":"2023-11-24T06:09:56.051688+0000","last_unstale":"2023-11-24T06:09:57.179751+0000","last_undegraded":"2023-11-24T06:09:57.179751+0000","last_fullsized":"2023-11-24T06:09:57.179751+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:51:15.191800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.055261+0000","last_change":"2023-11-24T06:09:55.051381+0000","last_active":"2023-11-24T06:09:56.055261+0000","last_peered":"2023-11-24T06:09:56.055261+0000","last_clean":"2023-11-24T06:09:56.055261+0000","last_became_active":"2023-11-24T06:09:55.051094+0000","last_became_peered":"2023-11-24T06:09:55.051094+0000","last_unstale":"2023-11-24T06:09:56.055261+0000","last_undegraded":"2023-11-24T06:09:56.055261+0000","last_fullsized":"2023-11-24T06:09:56.055261+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:35.486477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.051140+0000","last_change":"2023-11-24T06:09:56.059202+0000","last_active":"2023-11-24T06:09:57.051140+0000","last_peered":"2023-11-24T06:09:57.051140+0000","last_clean":"2023-11-24T06:09:57.051140+0000","last_became_active":"2023-11-24T06:09:56.058789+0000","last_became_peered":"2023-11-24T06:09:56.058789+0000","last_unstale":"2023-11-24T06:09:57.051140+0000","last_undegraded":"2023-11-24T06:09:57.051140+0000","last_fullsized":"2023-11-24T06:09:57.051140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:37.946793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.064740+0000","last_change":"2023-11-24T06:09:55.048025+0000","last_active":"2023-11-24T06:09:56.064740+0000","last_peered":"2023-11-24T06:09:56.064740+0000","last_clean":"2023-11-24T06:09:56.064740+0000","last_became_active":"2023-11-24T06:09:55.047759+0000","last_became_peered":"2023-11-24T06:09:55.047759+0000","last_unstale":"2023-11-24T06:09:56.064740+0000","last_undegraded":"2023-11-24T06:09:56.064740+0000","last_fullsized":"2023-11-24T06:09:56.064740+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:19.764307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.674367+0000","last_change":"2023-11-24T06:09:56.047483+0000","last_active":"2023-11-24T06:09:57.674367+0000","last_peered":"2023-11-24T06:09:57.674367+0000","last_clean":"2023-11-24T06:09:57.674367+0000","last_became_active":"2023-11-24T06:09:56.047210+0000","last_became_peered":"2023-11-24T06:09:56.047210+0000","last_unstale":"2023-11-24T06:09:57.674367+0000","last_undegraded":"2023-11-24T06:09:57.674367+0000","last_fullsized":"2023-11-24T06:09:57.674367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:57:29.925125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.060774+0000","last_change":"2023-11-24T06:09:55.042425+0000","last_active":"2023-11-24T06:09:56.060774+0000","last_peered":"2023-11-24T06:09:56.060774+0000","last_clean":"2023-11-24T06:09:56.060774+0000","last_became_active":"2023-11-24T06:09:55.042111+0000","last_became_peered":"2023-11-24T06:09:55.042111+0000","last_unstale":"2023-11-24T06:09:56.060774+0000","last_undegraded":"2023-11-24T06:09:56.060774+0000","last_fullsized":"2023-11-24T06:09:56.060774+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:24.757991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179608+0000","last_change":"2023-11-24T06:09:56.049343+0000","last_active":"2023-11-24T06:09:57.179608+0000","last_peered":"2023-11-24T06:09:57.179608+0000","last_clean":"2023-11-24T06:09:57.179608+0000","last_became_active":"2023-11-24T06:09:56.049061+0000","last_became_peered":"2023-11-24T06:09:56.049061+0000","last_unstale":"2023-11-24T06:09:57.179608+0000","last_undegraded":"2023-11-24T06:09:57.179608+0000","last_fullsized":"2023-11-24T06:09:57.179608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:09.552205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.041194+0000","last_change":"2023-11-24T06:09:55.049885+0000","last_active":"2023-11-24T06:09:56.041194+0000","last_peered":"2023-11-24T06:09:56.041194+0000","last_clean":"2023-11-24T06:09:56.041194+0000","last_became_active":"2023-11-24T06:09:55.049557+0000","last_became_peered":"2023-11-24T06:09:55.049557+0000","last_unstale":"2023-11-24T06:09:56.041194+0000","last_undegraded":"2023-11-24T06:09:56.041194+0000","last_fullsized":"2023-11-24T06:09:56.041194+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:19.764307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797761+0000","last_change":"2023-11-24T06:09:56.053031+0000","last_active":"2023-11-24T06:09:57.797761+0000","last_peered":"2023-11-24T06:09:57.797761+0000","last_clean":"2023-11-24T06:09:57.797761+0000","last_became_active":"2023-11-24T06:09:56.052147+0000","last_became_peered":"2023-11-24T06:09:56.052147+0000","last_unstale":"2023-11-24T06:09:57.797761+0000","last_undegraded":"2023-11-24T06:09:57.797761+0000","last_fullsized":"2023-11-24T06:09:57.797761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:13:25.443710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.033663+0000","last_change":"2023-11-24T06:09:55.051880+0000","last_active":"2023-11-24T06:09:56.033663+0000","last_peered":"2023-11-24T06:09:56.033663+0000","last_clean":"2023-11-24T06:09:56.033663+0000","last_became_active":"2023-11-24T06:09:55.051633+0000","last_became_peered":"2023-11-24T06:09:55.051633+0000","last_unstale":"2023-11-24T06:09:56.033663+0000","last_undegraded":"2023-11-24T06:09:56.033663+0000","last_fullsized":"2023-11-24T06:09:56.033663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:12:24.757991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797466+0000","last_change":"2023-11-24T06:09:56.050433+0000","last_active":"2023-11-24T06:09:57.797466+0000","last_peered":"2023-11-24T06:09:57.797466+0000","last_clean":"2023-11-24T06:09:57.797466+0000","last_became_active":"2023-11-24T06:09:56.050134+0000","last_became_peered":"2023-11-24T06:09:56.050134+0000","last_unstale":"2023-11-24T06:09:57.797466+0000","last_undegraded":"2023-11-24T06:09:57.797466+0000","last_fullsized":"2023-11-24T06:09:57.797466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:36:37.519738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.060536+0000","last_change":"2023-11-24T06:09:55.047309+0000","last_active":"2023-11-24T06:09:56.060536+0000","last_peered":"2023-11-24T06:09:56.060536+0000","last_clean":"2023-11-24T06:09:56.060536+0000","last_became_active":"2023-11-24T06:09:55.047030+0000","last_became_peered":"2023-11-24T06:09:55.047030+0000","last_unstale":"2023-11-24T06:09:56.060536+0000","last_undegraded":"2023-11-24T06:09:56.060536+0000","last_fullsized":"2023-11-24T06:09:56.060536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:36:36.511404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.052872+0000","last_change":"2023-11-24T06:09:56.048787+0000","last_active":"2023-11-24T06:09:57.052872+0000","last_peered":"2023-11-24T06:09:57.052872+0000","last_clean":"2023-11-24T06:09:57.052872+0000","last_became_active":"2023-11-24T06:09:56.048447+0000","last_became_peered":"2023-11-24T06:09:56.048447+0000","last_unstale":"2023-11-24T06:09:57.052872+0000","last_undegraded":"2023-11-24T06:09:57.052872+0000","last_fullsized":"2023-11-24T06:09:57.052872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:57:29.925125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.062133+0000","last_change":"2023-11-24T06:09:55.048528+0000","last_active":"2023-11-24T06:09:56.062133+0000","last_peered":"2023-11-24T06:09:56.062133+0000","last_clean":"2023-11-24T06:09:56.062133+0000","last_became_active":"2023-11-24T06:09:55.048045+0000","last_became_peered":"2023-11-24T06:09:55.048045+0000","last_unstale":"2023-11-24T06:09:56.062133+0000","last_undegraded":"2023-11-24T06:09:56.062133+0000","last_fullsized":"2023-11-24T06:09:56.062133+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:24.626555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.051108+0000","last_change":"2023-11-24T06:09:56.046451+0000","last_active":"2023-11-24T06:09:57.051108+0000","last_peered":"2023-11-24T06:09:57.051108+0000","last_clean":"2023-11-24T06:09:57.051108+0000","last_became_active":"2023-11-24T06:09:56.046152+0000","last_became_peered":"2023-11-24T06:09:56.046152+0000","last_unstale":"2023-11-24T06:09:57.051108+0000","last_undegraded":"2023-11-24T06:09:57.051108+0000","last_fullsized":"2023-11-24T06:09:57.051108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:13:25.443710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.060138+0000","last_change":"2023-11-24T06:09:55.043025+0000","last_active":"2023-11-24T06:09:56.060138+0000","last_peered":"2023-11-24T06:09:56.060138+0000","last_clean":"2023-11-24T06:09:56.060138+0000","last_became_active":"2023-11-24T06:09:55.042756+0000","last_became_peered":"2023-11-24T06:09:55.042756+0000","last_unstale":"2023-11-24T06:09:56.060138+0000","last_undegraded":"2023-11-24T06:09:56.060138+0000","last_fullsized":"2023-11-24T06:09:56.060138+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:50.304687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.674630+0000","last_change":"2023-11-24T06:09:56.052297+0000","last_active":"2023-11-24T06:09:57.674630+0000","last_peered":"2023-11-24T06:09:57.674630+0000","last_clean":"2023-11-24T06:09:57.674630+0000","last_became_active":"2023-11-24T06:09:56.050589+0000","last_became_peered":"2023-11-24T06:09:56.050589+0000","last_unstale":"2023-11-24T06:09:57.674630+0000","last_undegraded":"2023-11-24T06:09:57.674630+0000","last_fullsized":"2023-11-24T06:09:57.674630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:21.150064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.063740+0000","last_change":"2023-11-24T06:09:55.040147+0000","last_active":"2023-11-24T06:09:56.063740+0000","last_peered":"2023-11-24T06:09:56.063740+0000","last_clean":"2023-11-24T06:09:56.063740+0000","last_became_active":"2023-11-24T06:09:55.039878+0000","last_became_peered":"2023-11-24T06:09:55.039878+0000","last_unstale":"2023-11-24T06:09:56.063740+0000","last_undegraded":"2023-11-24T06:09:56.063740+0000","last_fullsized":"2023-11-24T06:09:56.063740+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:17.779921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.187657+0000","last_change":"2023-11-24T06:09:56.054643+0000","last_active":"2023-11-24T06:09:57.187657+0000","last_peered":"2023-11-24T06:09:57.187657+0000","last_clean":"2023-11-24T06:09:57.187657+0000","last_became_active":"2023-11-24T06:09:56.054375+0000","last_became_peered":"2023-11-24T06:09:56.054375+0000","last_unstale":"2023-11-24T06:09:57.187657+0000","last_undegraded":"2023-11-24T06:09:57.187657+0000","last_fullsized":"2023-11-24T06:09:57.187657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:57.467361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.036751+0000","last_change":"2023-11-24T06:09:55.046566+0000","last_active":"2023-11-24T06:09:56.036751+0000","last_peered":"2023-11-24T06:09:56.036751+0000","last_clean":"2023-11-24T06:09:56.036751+0000","last_became_active":"2023-11-24T06:09:55.046138+0000","last_became_peered":"2023-11-24T06:09:55.046138+0000","last_unstale":"2023-11-24T06:09:56.036751+0000","last_undegraded":"2023-11-24T06:09:56.036751+0000","last_fullsized":"2023-11-24T06:09:56.036751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:36:00.283528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.187724+0000","last_change":"2023-11-24T06:09:56.057612+0000","last_active":"2023-11-24T06:09:57.187724+0000","last_peered":"2023-11-24T06:09:57.187724+0000","last_clean":"2023-11-24T06:09:57.187724+0000","last_became_active":"2023-11-24T06:09:56.057347+0000","last_became_peered":"2023-11-24T06:09:56.057347+0000","last_unstale":"2023-11-24T06:09:57.187724+0000","last_undegraded":"2023-11-24T06:09:57.187724+0000","last_fullsized":"2023-11-24T06:09:57.187724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:01.778052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.057727+0000","last_change":"2023-11-24T06:09:55.042544+0000","last_active":"2023-11-24T06:09:56.057727+0000","last_peered":"2023-11-24T06:09:56.057727+0000","last_clean":"2023-11-24T06:09:56.057727+0000","last_became_active":"2023-11-24T06:09:55.042189+0000","last_became_peered":"2023-11-24T06:09:55.042189+0000","last_unstale":"2023-11-24T06:09:56.057727+0000","last_undegraded":"2023-11-24T06:09:56.057727+0000","last_fullsized":"2023-11-24T06:09:56.057727+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:50.304687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.187718+0000","last_change":"2023-11-24T06:09:56.053812+0000","last_active":"2023-11-24T06:09:57.187718+0000","last_peered":"2023-11-24T06:09:57.187718+0000","last_clean":"2023-11-24T06:09:57.187718+0000","last_became_active":"2023-11-24T06:09:56.053475+0000","last_became_peered":"2023-11-24T06:09:56.053475+0000","last_unstale":"2023-11-24T06:09:57.187718+0000","last_undegraded":"2023-11-24T06:09:57.187718+0000","last_fullsized":"2023-11-24T06:09:57.187718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:53:05.164390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.047838+0000","last_change":"2023-11-24T06:09:55.048081+0000","last_active":"2023-11-24T06:09:56.047838+0000","last_peered":"2023-11-24T06:09:56.047838+0000","last_clean":"2023-11-24T06:09:56.047838+0000","last_became_active":"2023-11-24T06:09:55.047720+0000","last_became_peered":"2023-11-24T06:09:55.047720+0000","last_unstale":"2023-11-24T06:09:56.047838+0000","last_undegraded":"2023-11-24T06:09:56.047838+0000","last_fullsized":"2023-11-24T06:09:56.047838+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:18:40.212324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.187514+0000","last_change":"2023-11-24T06:09:56.053108+0000","last_active":"2023-11-24T06:09:57.187514+0000","last_peered":"2023-11-24T06:09:57.187514+0000","last_clean":"2023-11-24T06:09:57.187514+0000","last_became_active":"2023-11-24T06:09:56.052817+0000","last_became_peered":"2023-11-24T06:09:56.052817+0000","last_unstale":"2023-11-24T06:09:57.187514+0000","last_undegraded":"2023-11-24T06:09:57.187514+0000","last_fullsized":"2023-11-24T06:09:57.187514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:18.788256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.037069+0000","last_change":"2023-11-24T06:09:55.044987+0000","last_active":"2023-11-24T06:09:56.037069+0000","last_peered":"2023-11-24T06:09:56.037069+0000","last_clean":"2023-11-24T06:09:56.037069+0000","last_became_active":"2023-11-24T06:09:55.044712+0000","last_became_peered":"2023-11-24T06:09:55.044712+0000","last_unstale":"2023-11-24T06:09:56.037069+0000","last_undegraded":"2023-11-24T06:09:56.037069+0000","last_fullsized":"2023-11-24T06:09:56.037069+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:18:40.212324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797643+0000","last_change":"2023-11-24T06:09:56.059238+0000","last_active":"2023-11-24T06:09:57.797643+0000","last_peered":"2023-11-24T06:09:57.797643+0000","last_clean":"2023-11-24T06:09:57.797643+0000","last_became_active":"2023-11-24T06:09:56.058636+0000","last_became_peered":"2023-11-24T06:09:56.058636+0000","last_unstale":"2023-11-24T06:09:57.797643+0000","last_undegraded":"2023-11-24T06:09:57.797643+0000","last_fullsized":"2023-11-24T06:09:57.797643+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:59.321450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.038206+0000","last_change":"2023-11-24T06:09:55.051596+0000","last_active":"2023-11-24T06:09:56.038206+0000","last_peered":"2023-11-24T06:09:56.038206+0000","last_clean":"2023-11-24T06:09:56.038206+0000","last_became_active":"2023-11-24T06:09:55.051313+0000","last_became_peered":"2023-11-24T06:09:55.051313+0000","last_unstale":"2023-11-24T06:09:56.038206+0000","last_undegraded":"2023-11-24T06:09:56.038206+0000","last_fullsized":"2023-11-24T06:09:56.038206+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:19.764307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.187700+0000","last_change":"2023-11-24T06:09:56.052807+0000","last_active":"2023-11-24T06:09:57.187700+0000","last_peered":"2023-11-24T06:09:57.187700+0000","last_clean":"2023-11-24T06:09:57.187700+0000","last_became_active":"2023-11-24T06:09:56.052526+0000","last_became_peered":"2023-11-24T06:09:56.052526+0000","last_unstale":"2023-11-24T06:09:57.187700+0000","last_undegraded":"2023-11-24T06:09:57.187700+0000","last_fullsized":"2023-11-24T06:09:57.187700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:36:01.291862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.058565+0000","last_change":"2023-11-24T06:09:55.049080+0000","last_active":"2023-11-24T06:09:56.058565+0000","last_peered":"2023-11-24T06:09:56.058565+0000","last_clean":"2023-11-24T06:09:56.058565+0000","last_became_active":"2023-11-24T06:09:55.048838+0000","last_became_peered":"2023-11-24T06:09:55.048838+0000","last_unstale":"2023-11-24T06:09:56.058565+0000","last_undegraded":"2023-11-24T06:09:56.058565+0000","last_fullsized":"2023-11-24T06:09:56.058565+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:36.901327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.050939+0000","last_change":"2023-11-24T06:09:56.059094+0000","last_active":"2023-11-24T06:09:57.050939+0000","last_peered":"2023-11-24T06:09:57.050939+0000","last_clean":"2023-11-24T06:09:57.050939+0000","last_became_active":"2023-11-24T06:09:56.058801+0000","last_became_peered":"2023-11-24T06:09:56.058801+0000","last_unstale":"2023-11-24T06:09:57.050939+0000","last_undegraded":"2023-11-24T06:09:57.050939+0000","last_fullsized":"2023-11-24T06:09:57.050939+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:20:21.150064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.048339+0000","last_change":"2023-11-24T06:09:55.048076+0000","last_active":"2023-11-24T06:09:57.048339+0000","last_peered":"2023-11-24T06:09:57.048339+0000","last_clean":"2023-11-24T06:09:57.048339+0000","last_became_active":"2023-11-24T06:09:55.047818+0000","last_became_peered":"2023-11-24T06:09:55.047818+0000","last_unstale":"2023-11-24T06:09:57.048339+0000","last_undegraded":"2023-11-24T06:09:57.048339+0000","last_fullsized":"2023-11-24T06:09:57.048339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:50.079694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.048179+0000","last_change":"2023-11-24T06:09:56.058402+0000","last_active":"2023-11-24T06:09:57.048179+0000","last_peered":"2023-11-24T06:09:57.048179+0000","last_clean":"2023-11-24T06:09:57.048179+0000","last_became_active":"2023-11-24T06:09:56.058135+0000","last_became_peered":"2023-11-24T06:09:56.058135+0000","last_unstale":"2023-11-24T06:09:57.048179+0000","last_undegraded":"2023-11-24T06:09:57.048179+0000","last_fullsized":"2023-11-24T06:09:57.048179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:49.632078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.041152+0000","last_change":"2023-11-24T06:09:55.047569+0000","last_active":"2023-11-24T06:09:56.041152+0000","last_peered":"2023-11-24T06:09:56.041152+0000","last_clean":"2023-11-24T06:09:56.041152+0000","last_became_active":"2023-11-24T06:09:55.047269+0000","last_became_peered":"2023-11-24T06:09:55.047269+0000","last_unstale":"2023-11-24T06:09:56.041152+0000","last_undegraded":"2023-11-24T06:09:56.041152+0000","last_fullsized":"2023-11-24T06:09:56.041152+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:15:26.144462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797554+0000","last_change":"2023-11-24T06:09:56.050271+0000","last_active":"2023-11-24T06:09:57.797554+0000","last_peered":"2023-11-24T06:09:57.797554+0000","last_clean":"2023-11-24T06:09:57.797554+0000","last_became_active":"2023-11-24T06:09:56.049974+0000","last_became_peered":"2023-11-24T06:09:56.049974+0000","last_unstale":"2023-11-24T06:09:57.797554+0000","last_undegraded":"2023-11-24T06:09:57.797554+0000","last_fullsized":"2023-11-24T06:09:57.797554+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:51.313021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.045494+0000","last_change":"2023-11-24T06:09:55.042888+0000","last_active":"2023-11-24T06:09:56.045494+0000","last_peered":"2023-11-24T06:09:56.045494+0000","last_clean":"2023-11-24T06:09:56.045494+0000","last_became_active":"2023-11-24T06:09:55.042267+0000","last_became_peered":"2023-11-24T06:09:55.042267+0000","last_unstale":"2023-11-24T06:09:56.045494+0000","last_undegraded":"2023-11-24T06:09:56.045494+0000","last_fullsized":"2023-11-24T06:09:56.045494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:36:36.511404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.048447+0000","last_change":"2023-11-24T06:09:56.056129+0000","last_active":"2023-11-24T06:09:57.048447+0000","last_peered":"2023-11-24T06:09:57.048447+0000","last_clean":"2023-11-24T06:09:57.048447+0000","last_became_active":"2023-11-24T06:09:56.055842+0000","last_became_peered":"2023-11-24T06:09:56.055842+0000","last_unstale":"2023-11-24T06:09:57.048447+0000","last_undegraded":"2023-11-24T06:09:57.048447+0000","last_fullsized":"2023-11-24T06:09:57.048447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:31:20.772641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.041057+0000","last_change":"2023-11-24T06:09:55.048795+0000","last_active":"2023-11-24T06:09:56.041057+0000","last_peered":"2023-11-24T06:09:56.041057+0000","last_clean":"2023-11-24T06:09:56.041057+0000","last_became_active":"2023-11-24T06:09:55.048533+0000","last_became_peered":"2023-11-24T06:09:55.048533+0000","last_unstale":"2023-11-24T06:09:56.041057+0000","last_undegraded":"2023-11-24T06:09:56.041057+0000","last_fullsized":"2023-11-24T06:09:56.041057+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:48.623744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.179415+0000","last_change":"2023-11-24T06:09:56.050872+0000","last_active":"2023-11-24T06:09:57.179415+0000","last_peered":"2023-11-24T06:09:57.179415+0000","last_clean":"2023-11-24T06:09:57.179415+0000","last_became_active":"2023-11-24T06:09:56.050591+0000","last_became_peered":"2023-11-24T06:09:56.050591+0000","last_unstale":"2023-11-24T06:09:57.179415+0000","last_undegraded":"2023-11-24T06:09:57.179415+0000","last_fullsized":"2023-11-24T06:09:57.179415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:33:07.107621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.037224+0000","last_change":"2023-11-24T06:09:55.040289+0000","last_active":"2023-11-24T06:09:56.037224+0000","last_peered":"2023-11-24T06:09:56.037224+0000","last_clean":"2023-11-24T06:09:56.037224+0000","last_became_active":"2023-11-24T06:09:55.040006+0000","last_became_peered":"2023-11-24T06:09:55.040006+0000","last_unstale":"2023-11-24T06:09:56.037224+0000","last_undegraded":"2023-11-24T06:09:56.037224+0000","last_fullsized":"2023-11-24T06:09:56.037224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:42.441971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.052175+0000","last_change":"2023-11-24T06:09:56.052111+0000","last_active":"2023-11-24T06:09:57.052175+0000","last_peered":"2023-11-24T06:09:57.052175+0000","last_clean":"2023-11-24T06:09:57.052175+0000","last_became_active":"2023-11-24T06:09:56.051845+0000","last_became_peered":"2023-11-24T06:09:56.051845+0000","last_unstale":"2023-11-24T06:09:57.052175+0000","last_undegraded":"2023-11-24T06:09:57.052175+0000","last_fullsized":"2023-11-24T06:09:57.052175+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:15:27.152797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.033817+0000","last_change":"2023-11-24T06:09:55.051436+0000","last_active":"2023-11-24T06:09:56.033817+0000","last_peered":"2023-11-24T06:09:56.033817+0000","last_clean":"2023-11-24T06:09:56.033817+0000","last_became_active":"2023-11-24T06:09:55.051184+0000","last_became_peered":"2023-11-24T06:09:55.051184+0000","last_unstale":"2023-11-24T06:09:56.033817+0000","last_undegraded":"2023-11-24T06:09:56.033817+0000","last_fullsized":"2023-11-24T06:09:56.033817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:50.304687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T06:09:57.797582+0000","last_change":"2023-11-24T06:09:56.054557+0000","last_active":"2023-11-24T06:09:57.797582+0000","last_peered":"2023-11-24T06:09:57.797582+0000","last_clean":"2023-11-24T06:09:57.797582+0000","last_became_active":"2023-11-24T06:09:56.054287+0000","last_became_peered":"2023-11-24T06:09:56.054287+0000","last_unstale":"2023-11-24T06:09:57.797582+0000","last_undegraded":"2023-11-24T06:09:57.797582+0000","last_fullsized":"2023-11-24T06:09:57.797582+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:55.022757+0000","last_clean_scrub_stamp":"2023-11-24T06:09:55.022757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:33:09.036629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.049965+0000","last_change":"2023-11-24T06:09:55.043278+0000","last_active":"2023-11-24T06:09:56.049965+0000","last_peered":"2023-11-24T06:09:56.049965+0000","last_clean":"2023-11-24T06:09:56.049965+0000","last_became_active":"2023-11-24T06:09:55.042997+0000","last_became_peered":"2023-11-24T06:09:55.042997+0000","last_unstale":"2023-11-24T06:09:56.049965+0000","last_undegraded":"2023-11-24T06:09:56.049965+0000","last_fullsized":"2023-11-24T06:09:56.049965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:09:54.014422+0000","last_clean_scrub_stamp":"2023-11-24T06:09:54.014422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:37.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T06:09:56.041269+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:09:56.041269+0000","last_peered":"2023-11-24T06:09:56.041269+0000","last_clean":"2023-11-24T06:09:56.041269+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:09:56.041269+0000","last_undegraded":"2023-11-24T06:09:56.041269+0000","last_fullsized":"2023-11-24T06:09:56.041269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":737050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.623},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.498}]},{"osd":2,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":8.4849999999999994,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.754}]},{"osd":7,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.378},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.52600000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.442}]},{"osd":4,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.621}]},{"osd":6,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.751}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":736960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":12.378,"5min":12.378,"15min":12.378},"last":0.432},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.5199999999999996,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":15.319000000000001,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.79800000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.7590000000000003,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.877},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":15.356,"5min":15.356,"15min":15.356},"last":1.0289999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.726,"5min":4.726,"15min":4.726},"last":0.90400000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":4.7869999999999999,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":15.388,"5min":15.388,"15min":15.388},"last":0.92600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.308}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.27900000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.32500000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.48499999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.63}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.38300000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.878}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83299999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.628},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.499}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.335,"5min":3.335,"15min":3.335},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":4.4489999999999998,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.5069999999999997,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.436},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.38600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.624},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:10:00.535 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:10:00.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:10:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.692+0000 7f5390947700 1 Processor -- start 2023-11-24T06:10:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.692+0000 7f5390947700 1 -- start start 2023-11-24T06:10:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f5390947700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f538c0cfdf0 0x7f538c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f5390947700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f538c0744d0 0x7f538c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f5390947700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538c0b5ee0 con 0x7f538c15be70 2023-11-24T06:10:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f5390947700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538c06bfc0 con 0x7f538c0cfdf0 2023-11-24T06:10:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f5390947700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538c074940 con 0x7f538c0744d0 2023-11-24T06:10:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f5389d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f538c0cfdf0 0x7f538c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f5389d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f538c0cfdf0 0x7f538c073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47052/0 (socket says 172.21.15.97:47052) 2023-11-24T06:10:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f5389d9b700 1 -- 172.21.15.97:0/1299437243 learned_addr learned my addr 172.21.15.97:0/1299437243 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.693+0000 7f538ad9d700 1 --2- 172.21.15.97:0/1299437243 >> v2:172.21.15.183:3300/0 conn(0x7f538c0744d0 0x7f538c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5389d9b700 1 -- 172.21.15.97:0/1299437243 >> v2:172.21.15.183:3300/0 conn(0x7f538c0744d0 msgr2=0x7f538c0b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5389d9b700 1 --2- 172.21.15.97:0/1299437243 >> v2:172.21.15.183:3300/0 conn(0x7f538c0744d0 0x7f538c0b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5389d9b700 1 -- 172.21.15.97:0/1299437243 >> v1:172.21.15.183:6789/0 conn(0x7f538c15be70 legacy=0x7f538c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5389d9b700 1 -- 172.21.15.97:0/1299437243 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c06ba00 con 0x7f538c0cfdf0 2023-11-24T06:10:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5389d9b700 1 --2- 172.21.15.97:0/1299437243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0cfdf0 0x7f538c073f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f538c071a00 tx=0x7f537c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1069ac35d1abbc19 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f538959a700 1 -- 172.21.15.97:0/1299437243 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c00b860 con 0x7f538c0cfdf0 2023-11-24T06:10:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f538959a700 1 -- 172.21.15.97:0/1299437243 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c00b9c0 con 0x7f538c0cfdf0 2023-11-24T06:10:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f538959a700 1 -- 172.21.15.97:0/1299437243 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c00bd50 con 0x7f538c0cfdf0 2023-11-24T06:10:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5390947700 1 -- 172.21.15.97:0/1299437243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0cfdf0 msgr2=0x7f538c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5390947700 1 --2- 172.21.15.97:0/1299437243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0cfdf0 0x7f538c073f90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f538c071a00 tx=0x7f537c009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5390947700 1 -- 172.21.15.97:0/1299437243 shutdown_connections 2023-11-24T06:10:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5390947700 1 --2- 172.21.15.97:0/1299437243 >> v2:172.21.15.183:3300/0 conn(0x7f538c0744d0 0x7f538c0b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5390947700 1 --2- 172.21.15.97:0/1299437243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0cfdf0 0x7f538c073f90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.694+0000 7f5390947700 1 -- 172.21.15.97:0/1299437243 >> 172.21.15.97:0/1299437243 conn(0x7f538c0d0bf0 msgr2=0x7f538c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.695+0000 7f5390947700 1 -- 172.21.15.97:0/1299437243 shutdown_connections 2023-11-24T06:10:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.695+0000 7f5390947700 1 -- 172.21.15.97:0/1299437243 wait complete. 2023-11-24T06:10:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.696+0000 7f5390947700 1 Processor -- start 2023-11-24T06:10:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.696+0000 7f5390947700 1 -- start start 2023-11-24T06:10:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f5390947700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0744d0 0x7f538c146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f5390947700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f538c0cfdf0 0x7f538c147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f5390947700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f538c15be70 0x7f538c145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f5390947700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c146670 con 0x7f538c0744d0 2023-11-24T06:10:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f5390947700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c147470 con 0x7f538c15be70 2023-11-24T06:10:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f5390947700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c146860 con 0x7f538c0cfdf0 2023-11-24T06:10:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0744d0 0x7f538c146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0744d0 0x7f538c146130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47056/0 (socket says 172.21.15.97:47056) 2023-11-24T06:10:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 -- 172.21.15.97:0/291217591 learned_addr learned my addr 172.21.15.97:0/291217591 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 -- 172.21.15.97:0/291217591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f538c0cfdf0 msgr2=0x7f538c147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f538c0cfdf0 0x7f538c147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 -- 172.21.15.97:0/291217591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f538c15be70 msgr2=0x7f538c145470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f538c15be70 0x7f538c145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 -- 172.21.15.97:0/291217591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c009550 con 0x7f538c0744d0 2023-11-24T06:10:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f538a59c700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0744d0 0x7f538c146130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f538000c450 tx=0x7f538000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f537affd700 1 -- 172.21.15.97:0/291217591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800052d0 con 0x7f538c0744d0 2023-11-24T06:10:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f537affd700 1 -- 172.21.15.97:0/291217591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5380005430 con 0x7f538c0744d0 2023-11-24T06:10:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f537affd700 1 -- 172.21.15.97:0/291217591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800038c0 con 0x7f538c0744d0 2023-11-24T06:10:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c146b40 con 0x7f538c0744d0 2023-11-24T06:10:00.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.697+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c147100 con 0x7f538c0744d0 2023-11-24T06:10:00.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.698+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c068fb0 con 0x7f538c0744d0 2023-11-24T06:10:00.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.699+0000 7f537affd700 1 -- 172.21.15.97:0/291217591 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5380003a20 con 0x7f538c0744d0 2023-11-24T06:10:00.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.699+0000 7f537affd700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f536c06aed0 0x7f536c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:00.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.699+0000 7f537affd700 1 -- 172.21.15.97:0/291217591 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5380088930 con 0x7f538c0744d0 2023-11-24T06:10:00.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.699+0000 7f5389d9b700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f536c06aed0 0x7f536c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:00.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.700+0000 7f5389d9b700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f536c06aed0 0x7f536c06d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f538c0cfdc0 tx=0x7f537c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:00.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.703+0000 7f537affd700 1 -- 172.21.15.97:0/291217591 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53800539e0 con 0x7f538c0744d0 2023-11-24T06:10:00.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.971+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f538c0b2020 con 0x7f538c0744d0 2023-11-24T06:10:00.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.974+0000 7f537affd700 1 -- 172.21.15.97:0/291217591 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5380057000 con 0x7f538c0744d0 2023-11-24T06:10:00.977 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-11-24T06:10:00.977 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA== 2023-11-24T06:10:00.977 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-11-24T06:10:00.977 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-11-24T06:10:00.977 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:10:00.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f536c06aed0 msgr2=0x7f536c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f536c06aed0 0x7f536c06d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f538c0cfdc0 tx=0x7f537c010000 comp rx=0 tx=0).stop 2023-11-24T06:10:00.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0744d0 msgr2=0x7f538c146130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:00.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0744d0 0x7f538c146130 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f538000c450 tx=0x7f538000c810 comp rx=0 tx=0).stop 2023-11-24T06:10:00.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 shutdown_connections 2023-11-24T06:10:00.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f538c15be70 0x7f538c145470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f538c0744d0 0x7f538c146130 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f536c06aed0 0x7f536c06d390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 --2- 172.21.15.97:0/291217591 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f538c0cfdf0 0x7f538c147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:00.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 >> 172.21.15.97:0/291217591 conn(0x7f538c0d0bf0 msgr2=0x7f538c0af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:00.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 shutdown_connections 2023-11-24T06:10:00.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:00.976+0000 7f5390947700 1 -- 172.21.15.97:0/291217591 wait complete. 2023-11-24T06:10:00.982 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-11-24T06:10:00.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:01.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.110+0000 7f523fdf6700 1 Processor -- start 2023-11-24T06:10:01.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.110+0000 7f523fdf6700 1 -- start start 2023-11-24T06:10:01.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523fdf6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f52381446a0 0x7f52380b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523fdf6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f52380adf00 0x7f523806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523fdf6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f523806da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523fdf6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5238144a80 con 0x7f52380adf00 2023-11-24T06:10:01.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523fdf6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523806c9c0 con 0x7f52381446a0 2023-11-24T06:10:01.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523fdf6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52380af950 con 0x7f523806c540 2023-11-24T06:10:01.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f52380adf00 0x7f523806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f52380adf00 0x7f523806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47066/0 (socket says 172.21.15.97:47066) 2023-11-24T06:10:01.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 -- 172.21.15.97:0/954114284 learned_addr learned my addr 172.21.15.97:0/954114284 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:01.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523e393700 1 --2- 172.21.15.97:0/954114284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f523806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523db92700 1 --2- 172.21.15.97:0/954114284 >> v2:172.21.15.183:3300/0 conn(0x7f52381446a0 0x7f52380b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 -- 172.21.15.97:0/954114284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 msgr2=0x7f523806da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 --2- 172.21.15.97:0/954114284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f523806da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 -- 172.21.15.97:0/954114284 >> v2:172.21.15.183:3300/0 conn(0x7f52381446a0 msgr2=0x7f52380b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 --2- 172.21.15.97:0/954114284 >> v2:172.21.15.183:3300/0 conn(0x7f52381446a0 0x7f52380b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 -- 172.21.15.97:0/954114284 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380afc30 con 0x7f52380adf00 2023-11-24T06:10:01.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523e393700 1 --2- 172.21.15.97:0/954114284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f523806da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:01.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523db92700 1 --2- 172.21.15.97:0/954114284 >> v2:172.21.15.183:3300/0 conn(0x7f52381446a0 0x7f52380b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:01.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523d391700 1 --2- 172.21.15.97:0/954114284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523806c000 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f52380729a0 tx=0x7f5228009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920c3bdefa2eccd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.111+0000 7f523cb90700 1 -- 172.21.15.97:0/954114284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800b870 con 0x7f52380adf00 2023-11-24T06:10:01.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523cb90700 1 -- 172.21.15.97:0/954114284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522800b9d0 con 0x7f52380adf00 2023-11-24T06:10:01.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523cb90700 1 -- 172.21.15.97:0/954114284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800bd60 con 0x7f52380adf00 2023-11-24T06:10:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 -- 172.21.15.97:0/954114284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 msgr2=0x7f523806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 --2- 172.21.15.97:0/954114284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523806c000 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f52380729a0 tx=0x7f5228009bd0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 -- 172.21.15.97:0/954114284 shutdown_connections 2023-11-24T06:10:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 --2- 172.21.15.97:0/954114284 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f523806da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 --2- 172.21.15.97:0/954114284 >> v2:172.21.15.183:3300/0 conn(0x7f52381446a0 0x7f52380b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 --2- 172.21.15.97:0/954114284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523806c000 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 -- 172.21.15.97:0/954114284 >> 172.21.15.97:0/954114284 conn(0x7f5238071b90 msgr2=0x7f5238071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:01.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 -- 172.21.15.97:0/954114284 shutdown_connections 2023-11-24T06:10:01.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.112+0000 7f523fdf6700 1 -- 172.21.15.97:0/954114284 wait complete. 2023-11-24T06:10:01.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.113+0000 7f523fdf6700 1 Processor -- start 2023-11-24T06:10:01.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.113+0000 7f523fdf6700 1 -- start start 2023-11-24T06:10:01.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523fdf6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f52381566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523fdf6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523813f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523fdf6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52381446a0 0x7f523813fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523fdf6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523815b470 con 0x7f52380adf00 2023-11-24T06:10:01.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523fdf6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523815d9d0 con 0x7f52381446a0 2023-11-24T06:10:01.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523fdf6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523815dcb0 con 0x7f523806c540 2023-11-24T06:10:01.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523d391700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523813f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523d391700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523813f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47072/0 (socket says 172.21.15.97:47072) 2023-11-24T06:10:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523d391700 1 -- 172.21.15.97:0/369915953 learned_addr learned my addr 172.21.15.97:0/369915953 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523db92700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f52381566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523e393700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52381446a0 0x7f523813fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523d391700 1 -- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 msgr2=0x7f52381566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523d391700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f52381566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523d391700 1 -- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52381446a0 msgr2=0x7f523813fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523d391700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52381446a0 0x7f523813fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523d391700 1 -- 172.21.15.97:0/369915953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5228009550 con 0x7f52380adf00 2023-11-24T06:10:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.114+0000 7f523e393700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52381446a0 0x7f523813fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:01.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.115+0000 7f523d391700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523813f760 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5228013000 tx=0x7f5228004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.115+0000 7f522e7fc700 1 -- 172.21.15.97:0/369915953 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228004310 con 0x7f52380adf00 2023-11-24T06:10:01.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.115+0000 7f522e7fc700 1 -- 172.21.15.97:0/369915953 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522801f070 con 0x7f52380adf00 2023-11-24T06:10:01.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.115+0000 7f522e7fc700 1 -- 172.21.15.97:0/369915953 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52280036a0 con 0x7f52380adf00 2023-11-24T06:10:01.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.115+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523815df90 con 0x7f52380adf00 2023-11-24T06:10:01.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.115+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5238074d70 con 0x7f52380adf00 2023-11-24T06:10:01.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.116+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52380753f0 con 0x7f52380adf00 2023-11-24T06:10:01.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.116+0000 7f522e7fc700 1 -- 172.21.15.97:0/369915953 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5228021030 con 0x7f52380adf00 2023-11-24T06:10:01.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.116+0000 7f522e7fc700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f521c06aed0 0x7f521c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.117+0000 7f522e7fc700 1 -- 172.21.15.97:0/369915953 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f522808a950 con 0x7f52380adf00 2023-11-24T06:10:01.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.117+0000 7f523db92700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f521c06aed0 0x7f521c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.117+0000 7f523db92700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f521c06aed0 0x7f521c06d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f523806da90 tx=0x7f5234008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.121+0000 7f522e7fc700 1 -- 172.21.15.97:0/369915953 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5228055a90 con 0x7f52380adf00 2023-11-24T06:10:01.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.352+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5238067f60 con 0x7f52380adf00 2023-11-24T06:10:01.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.353+0000 7f522e7fc700 1 -- 172.21.15.97:0/369915953 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f5228015090 con 0x7f52380adf00 2023-11-24T06:10:01.356 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:01.356 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1778230784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1778230784},{"type":"v1","addr":"172.21.15.97:6836","nonce":1778230784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2183416217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2183416217},{"type":"v1","addr":"172.21.15.183:6834","nonce":2183416217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3650123299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3650123299},{"type":"v1","addr":"172.21.15.183:6835","nonce":3650123299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:09:55.467969+0000","modified":"2023-11-24T06:09:58.451611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5070},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5070":{"gid":5070,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3767182596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3767182596},{"type":"v1","addr":"172.21.15.97:6837","nonce":3767182596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:10:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f521c06aed0 msgr2=0x7f521c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f521c06aed0 0x7f521c06d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f523806da90 tx=0x7f5234008040 comp rx=0 tx=0).stop 2023-11-24T06:10:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 msgr2=0x7f523813f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523813f760 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5228013000 tx=0x7f5228004750 comp rx=0 tx=0).stop 2023-11-24T06:10:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 shutdown_connections 2023-11-24T06:10:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52381446a0 0x7f523813fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f521c06aed0 0x7f521c06d390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f523806c540 0x7f52381566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 --2- 172.21.15.97:0/369915953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52380adf00 0x7f523813f760 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 >> 172.21.15.97:0/369915953 conn(0x7f5238071b90 msgr2=0x7f52380b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:01.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 shutdown_connections 2023-11-24T06:10:01.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.355+0000 7f523fdf6700 1 -- 172.21.15.97:0/369915953 wait complete. 2023-11-24T06:10:01.360 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 17 2023-11-24T06:10:01.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:09:55.467969+0000', 'modified': '2023-11-24T06:09:58.451611+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5070}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5070': {'gid': 5070, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/3767182596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 3767182596}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3767182596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:10:01.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:10:01.773 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T06:10:01.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.896+0000 7ff53ac6b700 1 Processor -- start 2023-11-24T06:10:01.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.896+0000 7ff53ac6b700 1 -- start start 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff53ac6b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5340afe70 0x7ff53415e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff53ac6b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53406dd80 con 0x7ff53415bef0 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff53ac6b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53415f880 con 0x7ff5341624d0 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff53ac6b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53415ed90 con 0x7ff5340afe70 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff533fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5340afe70 0x7ff53415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff538a07700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff53415bef0 0x7ff5340b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39530/0 (socket says 172.21.15.97:39530) 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff533fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5340afe70 0x7ff53415e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47080/0 (socket says 172.21.15.97:47080) 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff538a07700 1 -- 172.21.15.97:0/1456551068 learned_addr learned my addr 172.21.15.97:0/1456551068 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:01.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff533fff700 1 -- 172.21.15.97:0/1456551068 >> v1:172.21.15.183:6789/0 conn(0x7ff5341624d0 legacy=0x7ff53406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff533fff700 1 -- 172.21.15.97:0/1456551068 >> v1:172.21.15.97:6789/0 conn(0x7ff53415bef0 legacy=0x7ff5340b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff533fff700 1 -- 172.21.15.97:0/1456551068 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53415c2d0 con 0x7ff5340afe70 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff5337fe700 1 -- 172.21.15.97:0/1456551068 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993326024 0 0) 0x7ff53406dd80 con 0x7ff53415bef0 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff533fff700 1 --2- 172.21.15.97:0/1456551068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5340afe70 0x7ff53415e850 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff520009840 tx=0x7ff520009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df373331c52b5b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff5337fe700 1 -- 172.21.15.97:0/1456551068 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520004230 con 0x7ff5340afe70 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff5337fe700 1 -- 172.21.15.97:0/1456551068 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff520004390 con 0x7ff5340afe70 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.897+0000 7ff5337fe700 1 -- 172.21.15.97:0/1456551068 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200036a0 con 0x7ff5340afe70 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.898+0000 7ff53ac6b700 1 -- 172.21.15.97:0/1456551068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5340afe70 msgr2=0x7ff53415e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.898+0000 7ff53ac6b700 1 --2- 172.21.15.97:0/1456551068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5340afe70 0x7ff53415e850 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff520009840 tx=0x7ff520009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.898+0000 7ff53ac6b700 1 -- 172.21.15.97:0/1456551068 shutdown_connections 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.898+0000 7ff53ac6b700 1 --2- 172.21.15.97:0/1456551068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5340afe70 0x7ff53415e850 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.898+0000 7ff53ac6b700 1 -- 172.21.15.97:0/1456551068 >> 172.21.15.97:0/1456551068 conn(0x7ff534071ac0 msgr2=0x7ff534071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:01.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.898+0000 7ff53ac6b700 1 -- 172.21.15.97:0/1456551068 shutdown_connections 2023-11-24T06:10:01.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.898+0000 7ff53ac6b700 1 -- 172.21.15.97:0/1456551068 wait complete. 2023-11-24T06:10:01.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.899+0000 7ff53ac6b700 1 Processor -- start 2023-11-24T06:10:01.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.899+0000 7ff53ac6b700 1 -- start start 2023-11-24T06:10:01.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff53ac6b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5340afe70 0x7ff534147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff53ac6b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff53415bef0 0x7ff5341e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff53ac6b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341624d0 0x7ff53415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff53ac6b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53415dd80 con 0x7ff5341624d0 2023-11-24T06:10:01.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff53ac6b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53415d100 con 0x7ff5340afe70 2023-11-24T06:10:01.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff53ac6b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53415d3e0 con 0x7ff53415bef0 2023-11-24T06:10:01.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff539208700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341624d0 0x7ff53415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff539208700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341624d0 0x7ff53415e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47088/0 (socket says 172.21.15.97:47088) 2023-11-24T06:10:01.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff539208700 1 -- 172.21.15.97:0/3785678211 learned_addr learned my addr 172.21.15.97:0/3785678211 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:01.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff538a07700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5340afe70 0x7ff534147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff533fff700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff53415bef0 0x7ff5341e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff539208700 1 -- 172.21.15.97:0/3785678211 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff53415bef0 msgr2=0x7ff5341e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff539208700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff53415bef0 0x7ff5341e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff539208700 1 -- 172.21.15.97:0/3785678211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5340afe70 msgr2=0x7ff534147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff539208700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5340afe70 0x7ff534147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.900+0000 7ff539208700 1 -- 172.21.15.97:0/3785678211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200094a0 con 0x7ff5341624d0 2023-11-24T06:10:01.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.901+0000 7ff539208700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341624d0 0x7ff53415e1a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c7a0 tx=0x7ff52c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.901+0000 7ff5317fa700 1 -- 172.21.15.97:0/3785678211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c003e10 con 0x7ff5341624d0 2023-11-24T06:10:01.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.901+0000 7ff5317fa700 1 -- 172.21.15.97:0/3785678211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c005070 con 0x7ff5341624d0 2023-11-24T06:10:01.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.901+0000 7ff53ac6b700 1 -- 172.21.15.97:0/3785678211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53415d6c0 con 0x7ff5341624d0 2023-11-24T06:10:01.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.901+0000 7ff5317fa700 1 -- 172.21.15.97:0/3785678211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c005420 con 0x7ff5341624d0 2023-11-24T06:10:01.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.901+0000 7ff53ac6b700 1 -- 172.21.15.97:0/3785678211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53415cd60 con 0x7ff5341624d0 2023-11-24T06:10:01.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.902+0000 7ff5317fa700 1 -- 172.21.15.97:0/3785678211 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff52c005580 con 0x7ff5341624d0 2023-11-24T06:10:01.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.902+0000 7ff5317fa700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff51806aed0 0x7ff51806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.903+0000 7ff5317fa700 1 -- 172.21.15.97:0/3785678211 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff52c088b10 con 0x7ff5341624d0 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.903+0000 7ff538a07700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff51806aed0 0x7ff51806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.903+0000 7ff538a07700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff51806aed0 0x7ff51806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff528007fd0 tx=0x7ff528007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.905+0000 7ff546e64b80 1 Processor -- start 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.905+0000 7ff546e64b80 1 -- start start 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.905+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e566840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e566d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 0x564b4e549e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff546e64b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4e493770 con 0x564b4e520080 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff546e64b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4e47a750 con 0x564b4e535ed0 2023-11-24T06:10:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff546e64b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b4e478070 con 0x564b4e51d490 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e566840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e566840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47092/0 (socket says 172.21.15.97:47092) 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 -- 172.21.15.97:0/4116088843 learned_addr learned my addr 172.21.15.97:0/4116088843 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 -- 172.21.15.97:0/4116088843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 msgr2=0x564b4e549e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 --2- 172.21.15.97:0/4116088843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 0x564b4e549e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 -- 172.21.15.97:0/4116088843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 msgr2=0x564b4e566d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 --2- 172.21.15.97:0/4116088843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e566d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 -- 172.21.15.97:0/4116088843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b4e2f05e0 con 0x564b4e520080 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff538a07700 1 --2- 172.21.15.97:0/4116088843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e566840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff528001770 tx=0x7ff528013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cf7f421c1b3fc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff51e7fc700 1 -- 172.21.15.97:0/4116088843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528001ce0 con 0x564b4e520080 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff51e7fc700 1 -- 172.21.15.97:0/4116088843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff528013600 con 0x564b4e520080 2023-11-24T06:10:01.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.906+0000 7ff51e7fc700 1 -- 172.21.15.97:0/4116088843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528013970 con 0x564b4e520080 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 -- 172.21.15.97:0/4116088843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 msgr2=0x564b4e566840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 --2- 172.21.15.97:0/4116088843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e566840 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff528001770 tx=0x7ff528013000 comp rx=0 tx=0).stop 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 -- 172.21.15.97:0/4116088843 shutdown_connections 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 --2- 172.21.15.97:0/4116088843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 0x564b4e549e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 --2- 172.21.15.97:0/4116088843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e566840 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 --2- 172.21.15.97:0/4116088843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e566d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 -- 172.21.15.97:0/4116088843 >> 172.21.15.97:0/4116088843 conn(0x564b4e44e310 msgr2=0x564b4e564e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 -- 172.21.15.97:0/4116088843 shutdown_connections 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 -- 172.21.15.97:0/4116088843 wait complete. 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.907+0000 7ff546e64b80 1 -- start start 2023-11-24T06:10:01.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.908+0000 7ff546e64b80 1 Processor -- start 2023-11-24T06:10:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 0x564b4e54de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e55fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e514730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff546e64b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4e473960 con 0x564b4e520080 2023-11-24T06:10:01.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff546e64b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4e42d510 con 0x564b4e535ed0 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff546e64b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b4e42d1b0 con 0x564b4e51d490 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff546e64b80 10 client.0 _ll_register_callbacks cb 0x564b4e358a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff533fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e55fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff533fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e55fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47098/0 (socket says 172.21.15.97:47098) 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff533fff700 1 -- 172.21.15.97:0/282567132 learned_addr learned my addr 172.21.15.97:0/282567132 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff538a07700 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 0x564b4e54de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff539208700 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e514730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff533fff700 1 -- 172.21.15.97:0/282567132 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 msgr2=0x564b4e54de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff533fff700 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 0x564b4e54de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff533fff700 1 -- 172.21.15.97:0/282567132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 msgr2=0x564b4e514730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff533fff700 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e514730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.909+0000 7ff533fff700 1 -- 172.21.15.97:0/282567132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff528001450 con 0x564b4e520080 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff533fff700 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e55fe20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff520004000 tx=0x7ff520004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a2db3f0349d26 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff5137fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff5137fe700 1 -- 172.21.15.97:0/282567132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520003c60 con 0x564b4e520080 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff5137fe700 1 -- 172.21.15.97:0/282567132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff520003dc0 con 0x564b4e520080 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 1 -- 172.21.15.97:0/282567132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b4e512a40 con 0x564b4e520080 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff5137fe700 1 -- 172.21.15.97:0/282567132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5200045d0 con 0x564b4e520080 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 10 client.5253 fetch_fsmap learned FSMap version 17 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 1 -- 172.21.15.97:0/282567132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x564b4e29cd90 con 0x564b4e520080 2023-11-24T06:10:01.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff5137fe700 1 -- 172.21.15.97:0/282567132 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b4e512a40 con 0x564b4e520080 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff5137fe700 1 -- 172.21.15.97:0/282567132 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff520018070 con 0x564b4e520080 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 10 client.5253 fetch_fsmap finished waiting for FSMap version 17 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 10 client.5253 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 20 client.5253 populate_metadata read hostname 'smithi097' 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] conn(0x564b4e577890 0x564b4e54c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 4 client.5253 mds_command: new command op to 5070 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff546e64b80 1 -- 172.21.15.97:0/282567132 --> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b4e4ee840 con 0x564b4e577890 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.910+0000 7ff538a07700 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] conn(0x564b4e577890 0x564b4e54c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff5137fe700 10 client.5253 ms_handle_connect on v2:172.21.15.97:6835/3767182596 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff538a07700 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] conn(0x564b4e577890 0x564b4e54c990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c619a190611a494 server_cookie=b72751859017aa26 in_seq=0 out_seq=0 2023-11-24T06:10:01.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff5137fe700 1 -- 172.21.15.97:0/282567132 <== mds.0 v2:172.21.15.97:6835/3767182596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b4e4ee840 con 0x564b4e577890 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff5137fe700 10 client.5253 handle_command_reply: tid=0 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 client.5253 shutdown 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 -- 172.21.15.97:0/282567132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 msgr2=0x564b4e55fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e55fe20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff520004000 tx=0x7ff520004750 comp rx=0 tx=0).stop 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 -- 172.21.15.97:0/282567132 shutdown_connections 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] conn(0x564b4e577890 0x564b4e54c990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e514730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e51d490 0x564b4e54de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff5137fe700 0 client.5253 ms_handle_reset on v2:172.21.15.97:6835/3767182596 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 --2- 172.21.15.97:0/282567132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e520080 0x564b4e55fe20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.911+0000 7ff546e64b80 1 -- 172.21.15.97:0/282567132 >> 172.21.15.97:0/282567132 conn(0x564b4e44e310 msgr2=0x564b4e568c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.912+0000 7ff546e64b80 1 -- 172.21.15.97:0/282567132 shutdown_connections 2023-11-24T06:10:01.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.912+0000 7ff546e64b80 1 -- 172.21.15.97:0/282567132 wait complete. 2023-11-24T06:10:01.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.912+0000 7ff546e64b80 20 client.5253 trim_cache size 0 max 16384 2023-11-24T06:10:01.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.930+0000 7ff546e64b80 1 Processor -- start 2023-11-24T06:10:01.932 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.930+0000 7ff546e64b80 1 -- start start 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.930+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e51d490 0x564b4e55e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 0x564b4e5676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e567be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff546e64b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4e2f28d0 con 0x564b4e51d490 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff546e64b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4e51ad70 con 0x564b4e535ed0 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff546e64b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b4e587620 con 0x564b4e520080 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e51d490 0x564b4e55e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e51d490 0x564b4e55e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47106/0 (socket says 172.21.15.97:47106) 2023-11-24T06:10:01.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 -- 172.21.15.97:0/559490928 learned_addr learned my addr 172.21.15.97:0/559490928 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 -- 172.21.15.97:0/559490928 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 msgr2=0x564b4e5676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 --2- 172.21.15.97:0/559490928 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 0x564b4e5676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 -- 172.21.15.97:0/559490928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 msgr2=0x564b4e567be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 --2- 172.21.15.97:0/559490928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e567be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 -- 172.21.15.97:0/559490928 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5341489f0 con 0x564b4e51d490 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff538a07700 1 --2- 172.21.15.97:0/559490928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e51d490 0x564b4e55e080 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff5280078b0 tx=0x7ff528014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c91e5918ad154e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff51effd700 1 -- 172.21.15.97:0/559490928 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528013a70 con 0x564b4e51d490 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff51effd700 1 -- 172.21.15.97:0/559490928 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52800b040 con 0x564b4e51d490 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.931+0000 7ff51effd700 1 -- 172.21.15.97:0/559490928 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528013e10 con 0x564b4e51d490 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 -- 172.21.15.97:0/559490928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e51d490 msgr2=0x564b4e55e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 --2- 172.21.15.97:0/559490928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e51d490 0x564b4e55e080 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff5280078b0 tx=0x7ff528014000 comp rx=0 tx=0).stop 2023-11-24T06:10:01.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 -- 172.21.15.97:0/559490928 shutdown_connections 2023-11-24T06:10:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 --2- 172.21.15.97:0/559490928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e535ed0 0x564b4e567be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 --2- 172.21.15.97:0/559490928 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e51d490 0x564b4e55e080 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 --2- 172.21.15.97:0/559490928 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 0x564b4e5676a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 -- 172.21.15.97:0/559490928 >> 172.21.15.97:0/559490928 conn(0x564b4e44e310 msgr2=0x564b4e586d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 -- 172.21.15.97:0/559490928 shutdown_connections 2023-11-24T06:10:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 -- 172.21.15.97:0/559490928 wait complete. 2023-11-24T06:10:01.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.932+0000 7ff546e64b80 1 -- start start 2023-11-24T06:10:01.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff546e64b80 1 Processor -- start 2023-11-24T06:10:01.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e51d490 0x564b4e566fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 0x564b4e567aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff546e64b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e535ed0 0x564b4e5d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff546e64b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4e5cec20 con 0x564b4e535ed0 2023-11-24T06:10:01.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff546e64b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b4e55bc20 con 0x564b4e51d490 2023-11-24T06:10:01.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff546e64b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b4e5d1aa0 con 0x564b4e520080 2023-11-24T06:10:01.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff546e64b80 10 client.0 _ll_register_callbacks cb 0x564b4e358a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff539208700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e535ed0 0x564b4e5d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff539208700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e535ed0 0x564b4e5d1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47116/0 (socket says 172.21.15.97:47116) 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff539208700 1 -- 172.21.15.97:0/249516636 learned_addr learned my addr 172.21.15.97:0/249516636 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.933+0000 7ff533fff700 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 0x564b4e567aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff538a07700 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e51d490 0x564b4e566fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff539208700 1 -- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 msgr2=0x564b4e567aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff539208700 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 0x564b4e567aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff539208700 1 -- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e51d490 msgr2=0x564b4e566fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff539208700 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e51d490 0x564b4e566fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff539208700 1 -- 172.21.15.97:0/249516636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b4e2f05e0 con 0x564b4e535ed0 2023-11-24T06:10:01.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff538a07700 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e51d490 0x564b4e566fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff539208700 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e535ed0 0x564b4e5d1f10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0578a0 tx=0x7ff52c057e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705bd7e57a089456 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff5137fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff5137fe700 1 -- 172.21.15.97:0/249516636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c05f980 con 0x564b4e535ed0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff5137fe700 1 -- 172.21.15.97:0/249516636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff52c06f210 con 0x564b4e535ed0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 1 -- 172.21.15.97:0/249516636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff528013600 con 0x564b4e535ed0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff5137fe700 1 -- 172.21.15.97:0/249516636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff52c05ba60 con 0x564b4e535ed0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 10 client.5259 fetch_fsmap learned FSMap version 17 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 1 -- 172.21.15.97:0/249516636 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x564b4e285c10 con 0x564b4e535ed0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff5137fe700 1 -- 172.21.15.97:0/249516636 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c073080 con 0x564b4e535ed0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff5137fe700 1 -- 172.21.15.97:0/249516636 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff52c006ba0 con 0x564b4e535ed0 2023-11-24T06:10:01.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 10 client.5259 fetch_fsmap finished waiting for FSMap version 17 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 10 client.5259 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 20 client.5259 populate_metadata read hostname 'smithi097' 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] conn(0x564b4e5c81d0 0x564b4e5c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 4 client.5259 mds_command: new command op to 5070 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff546e64b80 1 -- 172.21.15.97:0/249516636 --> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564b4e439f80 con 0x564b4e5c81d0 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.934+0000 7ff538a07700 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] conn(0x564b4e5c81d0 0x564b4e5c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff5137fe700 10 client.5259 ms_handle_connect on v2:172.21.15.97:6835/3767182596 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff538a07700 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] conn(0x564b4e5c81d0 0x564b4e5c85b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8739c8c1a8a4292 server_cookie=cecb4e38cfcf2c16 in_seq=0 out_seq=0 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff5137fe700 1 -- 172.21.15.97:0/249516636 <== mds.0 v2:172.21.15.97:6835/3767182596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x564b4e439f80 con 0x564b4e5c81d0 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff5137fe700 10 client.5259 handle_command_reply: tid=0 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 client.5259 shutdown 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 -- 172.21.15.97:0/249516636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e535ed0 msgr2=0x564b4e5d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e535ed0 0x564b4e5d1f10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0578a0 tx=0x7ff52c057e50 comp rx=0 tx=0).stop 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 -- 172.21.15.97:0/249516636 shutdown_connections 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.97:6835/3767182596,v1:172.21.15.97:6837/3767182596] conn(0x564b4e5c81d0 0x564b4e5c85b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b4e535ed0 0x564b4e5d1f10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b4e51d490 0x564b4e566fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 --2- 172.21.15.97:0/249516636 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x564b4e520080 0x564b4e567aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 -- 172.21.15.97:0/249516636 >> 172.21.15.97:0/249516636 conn(0x564b4e44e310 msgr2=0x564b4e55f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff5137fe700 0 client.5259 ms_handle_reset on v2:172.21.15.97:6835/3767182596 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 -- 172.21.15.97:0/249516636 shutdown_connections 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 1 -- 172.21.15.97:0/249516636 wait complete. 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.935+0000 7ff546e64b80 20 client.5259 trim_cache size 0 max 16384 2023-11-24T06:10:01.945 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9", 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5070, 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.904828813, 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 17, 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 33, 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 33, 2023-11-24T06:10:01.946 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 13.788665088 2023-11-24T06:10:01.947 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:10:01.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 -- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff51806aed0 msgr2=0x7ff51806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff51806aed0 0x7ff51806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff528007fd0 tx=0x7ff528007de0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 -- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341624d0 msgr2=0x7ff53415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341624d0 0x7ff53415e1a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c7a0 tx=0x7ff52c00cab0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 -- 172.21.15.97:0/3785678211 shutdown_connections 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5341624d0 0x7ff53415e1a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5340afe70 0x7ff534147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff51806aed0 0x7ff51806d390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 --2- 172.21.15.97:0/3785678211 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff53415bef0 0x7ff5341e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 -- 172.21.15.97:0/3785678211 >> 172.21.15.97:0/3785678211 conn(0x7ff534071ac0 msgr2=0x7ff53406de60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 -- 172.21.15.97:0/3785678211 shutdown_connections 2023-11-24T06:10:01.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:01.936+0000 7ff53ac6b700 1 -- 172.21.15.97:0/3785678211 wait complete. 2023-11-24T06:10:01.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:10:02.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.069+0000 7f026115f700 1 Processor -- start 2023-11-24T06:10:02.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.069+0000 7f026115f700 1 -- start start 2023-11-24T06:10:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.069+0000 7f026115f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c162240 0x7f025c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.069+0000 7f026115f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f025c0ae4e0 0x7f025c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.069+0000 7f026115f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c0b46e0 con 0x7f025c06f360 2023-11-24T06:10:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.069+0000 7f026115f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c166f50 con 0x7f025c0ae4e0 2023-11-24T06:10:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.069+0000 7f026115f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0b49c0 con 0x7f025c162240 2023-11-24T06:10:02.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025b59e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f025c06f360 0x7f025c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39576/0 (socket says 172.21.15.97:39576) 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025b59e700 1 -- 172.21.15.97:0/2170286689 learned_addr learned my addr 172.21.15.97:0/2170286689 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025a59c700 1 --2- 172.21.15.97:0/2170286689 >> v2:172.21.15.183:3300/0 conn(0x7f025c0ae4e0 0x7f025c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025ad9d700 1 --2- 172.21.15.97:0/2170286689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c162240 0x7f025c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025ad9d700 1 -- 172.21.15.97:0/2170286689 >> v2:172.21.15.183:3300/0 conn(0x7f025c0ae4e0 msgr2=0x7f025c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025ad9d700 1 --2- 172.21.15.97:0/2170286689 >> v2:172.21.15.183:3300/0 conn(0x7f025c0ae4e0 0x7f025c06c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025ad9d700 1 -- 172.21.15.97:0/2170286689 >> v1:172.21.15.97:6789/0 conn(0x7f025c06f360 legacy=0x7f025c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025ad9d700 1 -- 172.21.15.97:0/2170286689 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0b4ca0 con 0x7f025c162240 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025a59c700 1 --2- 172.21.15.97:0/2170286689 >> v2:172.21.15.183:3300/0 conn(0x7f025c0ae4e0 0x7f025c06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:02.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.070+0000 7f025ad9d700 1 --2- 172.21.15.97:0/2170286689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c162240 0x7f025c0afed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0250009840 tx=0x7f0250009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a658632dd04fab78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:02.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.072+0000 7f0259d9b700 1 -- 172.21.15.97:0/2170286689 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250011830 con 0x7f025c162240 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.072+0000 7f0259d9b700 1 -- 172.21.15.97:0/2170286689 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0250011990 con 0x7f025c162240 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.072+0000 7f0259d9b700 1 -- 172.21.15.97:0/2170286689 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250011d20 con 0x7f025c162240 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.072+0000 7f026115f700 1 -- 172.21.15.97:0/2170286689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c162240 msgr2=0x7f025c0afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.072+0000 7f026115f700 1 --2- 172.21.15.97:0/2170286689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c162240 0x7f025c0afed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0250009840 tx=0x7f0250009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.073+0000 7f026115f700 1 -- 172.21.15.97:0/2170286689 shutdown_connections 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.073+0000 7f026115f700 1 --2- 172.21.15.97:0/2170286689 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c162240 0x7f025c0afed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.073+0000 7f026115f700 1 --2- 172.21.15.97:0/2170286689 >> v2:172.21.15.183:3300/0 conn(0x7f025c0ae4e0 0x7f025c06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.073+0000 7f026115f700 1 -- 172.21.15.97:0/2170286689 >> 172.21.15.97:0/2170286689 conn(0x7f025c071b90 msgr2=0x7f025c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.073+0000 7f026115f700 1 -- 172.21.15.97:0/2170286689 shutdown_connections 2023-11-24T06:10:02.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.073+0000 7f026115f700 1 -- 172.21.15.97:0/2170286689 wait complete. 2023-11-24T06:10:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.074+0000 7f026115f700 1 Processor -- start 2023-11-24T06:10:02.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.074+0000 7f026115f700 1 -- start start 2023-11-24T06:10:02.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f026115f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c06f360 0x7f025c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:02.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f026115f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025c0ae4e0 0x7f025c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:02.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f026115f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c162240 0x7f025c165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:02.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f026115f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c1f8d50 con 0x7f025c162240 2023-11-24T06:10:02.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f026115f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c1f7ee0 con 0x7f025c0ae4e0 2023-11-24T06:10:02.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f026115f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c1f8140 con 0x7f025c06f360 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c162240 0x7f025c165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c162240 0x7f025c165c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47130/0 (socket says 172.21.15.97:47130) 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 -- 172.21.15.97:0/781265174 learned_addr learned my addr 172.21.15.97:0/781265174 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025ad9d700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c06f360 0x7f025c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025a59c700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025c0ae4e0 0x7f025c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 -- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c06f360 msgr2=0x7f025c15c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c06f360 0x7f025c15c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 -- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025c0ae4e0 msgr2=0x7f025c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025c0ae4e0 0x7f025c09aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 -- 172.21.15.97:0/781265174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0094a0 con 0x7f025c162240 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025a59c700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025c0ae4e0 0x7f025c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:02.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025ad9d700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c06f360 0x7f025c15c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:02.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.075+0000 7f025b59e700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c162240 0x7f025c165c20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f025400cbc0 tx=0x7f025400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:02.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.076+0000 7f02437fe700 1 -- 172.21.15.97:0/781265174 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025400ab50 con 0x7f025c162240 2023-11-24T06:10:02.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.076+0000 7f02437fe700 1 -- 172.21.15.97:0/781265174 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025400acb0 con 0x7f025c162240 2023-11-24T06:10:02.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.076+0000 7f02437fe700 1 -- 172.21.15.97:0/781265174 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0254012600 con 0x7f025c162240 2023-11-24T06:10:02.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.076+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02500094a0 con 0x7f025c162240 2023-11-24T06:10:02.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.076+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c1f8700 con 0x7f025c162240 2023-11-24T06:10:02.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.077+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c0753f0 con 0x7f025c162240 2023-11-24T06:10:02.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.077+0000 7f02437fe700 1 -- 172.21.15.97:0/781265174 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f025401b3b0 con 0x7f025c162240 2023-11-24T06:10:02.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.077+0000 7f02437fe700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f024806aed0 0x7f024806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:02.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.078+0000 7f02437fe700 1 -- 172.21.15.97:0/781265174 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0254089240 con 0x7f025c162240 2023-11-24T06:10:02.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.078+0000 7f025ad9d700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f024806aed0 0x7f024806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:02.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.078+0000 7f025ad9d700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f024806aed0 0x7f024806d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f025001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:02.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.082+0000 7f02437fe700 1 -- 172.21.15.97:0/781265174 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02540543a0 con 0x7f025c162240 2023-11-24T06:10:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.315+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f025c067f60 con 0x7f025c162240 2023-11-24T06:10:02.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.316+0000 7f02437fe700 1 -- 172.21.15.97:0/781265174 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f02540579c0 con 0x7f025c162240 2023-11-24T06:10:02.319 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:10:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f024806aed0 msgr2=0x7f024806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f024806aed0 0x7f024806d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0250000c00 tx=0x7f025001c040 comp rx=0 tx=0).stop 2023-11-24T06:10:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c162240 msgr2=0x7f025c165c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c162240 0x7f025c165c20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f025400cbc0 tx=0x7f025400cf80 comp rx=0 tx=0).stop 2023-11-24T06:10:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 shutdown_connections 2023-11-24T06:10:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f025c162240 0x7f025c165c20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f024806aed0 0x7f024806d390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f025c06f360 0x7f025c15c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 --2- 172.21.15.97:0/781265174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f025c0ae4e0 0x7f025c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:02.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 >> 172.21.15.97:0/781265174 conn(0x7f025c071b90 msgr2=0x7f025c15d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:02.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 shutdown_connections 2023-11-24T06:10:02.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:02.318+0000 7f026115f700 1 -- 172.21.15.97:0/781265174 wait complete. 2023-11-24T06:10:02.332 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:10:02.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:10:02.332 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-24T06:10:02.333 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:10:02.333 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:02.333 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:10:02.333 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:10:02.333 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:10:02.333 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:10:02.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.333 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-24T06:10:02.351 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:02.351 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:02.351 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:02.352 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:02.352 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:02.352 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:02.352 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:02.352 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:02.353 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:02.353 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:02.353 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:02.353 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:02.353 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:10:02.353 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:02.354 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:10:02.354 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42134sec preferred_lft 42134sec 2023-11-24T06:10:02.354 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c9d4:801c:12b3:b866/64 scope link noprefixroute 2023-11-24T06:10:02.354 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:02.354 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:02.354 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:02.355 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:10:02.355 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:02.355 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::7c25:b1ff:fe16:36f1/64 scope link 2023-11-24T06:10:02.355 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:02.355 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:02.355 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether a2:28:0d:2d:82:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:02.356 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a028:dff:fe2d:822b/64 scope link 2023-11-24T06:10:02.356 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:02.356 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:02.356 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:10:02.356 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::1875:dff:fec0:4af7/64 scope link 2023-11-24T06:10:02.357 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:02.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.357 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:10:02.417 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-24T06:10:02.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.418 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-24T06:10:02.477 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-24T06:10:02.477 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:10:02.477 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:10:02.478 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:10:02.478 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:10:02.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.478 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:10:02.478 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:10:02.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:10:02.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:10:02.479 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:10:02.479 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:10:02.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.582 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:10:02.598 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:10:02.598 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:10:02.599 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:10:02.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-24T06:10:02.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:02.665 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:02.722 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:10:02.722 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:02.722 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:02.778 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:10:02.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:02.865 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:02.865 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:02.891 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:02.891 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:02.892 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:10:02.892 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:10:02.892 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:10:02.892 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:10:02.892 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:10:02.892 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:10:02.969 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:02.969 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:02.969 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:10:02.969 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:10:02.969 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:10:02.969 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:10:02.970 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:10:02.970 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:10:02.970 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:10:02.970 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:10:02.970 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:10:02.971 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:10:02.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:02.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:02.999 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:10:03.081 INFO:teuthology.orchestra.run.smithi097.stdout:client4814 2023-11-24T06:10:03.081 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:03.081 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:03.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:10:03.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.294+0000 7f170eb96700 1 Processor -- start 2023-11-24T06:10:03.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.294+0000 7f170eb96700 1 -- start start 2023-11-24T06:10:03.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170eb96700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f17080b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170eb96700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f170806c710 0x7f17080afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170eb96700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17080ae430 con 0x7f17080af4d0 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170eb96700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17080af0b0 con 0x7f170806c710 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170eb96700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17080b05a0 con 0x7f170815e080 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170c932700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f17080b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170d133700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f170806c710 0x7f17080afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170d133700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f170806c710 0x7f17080afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:41302/0 (socket says 172.21.15.97:41302) 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170c932700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f17080b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:34174/0 (socket says 172.21.15.97:34174) 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f170d133700 1 -- 172.21.15.97:0/910538122 learned_addr learned my addr 172.21.15.97:0/910538122 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:03.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.295+0000 7f1707fff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f17080af4d0 0x7f170806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:48912/0 (socket says 172.21.15.97:48912) 2023-11-24T06:10:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f170d133700 1 -- 172.21.15.97:0/910538122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 msgr2=0x7f17080b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f170d133700 1 --2- 172.21.15.97:0/910538122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f17080b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f170d133700 1 -- 172.21.15.97:0/910538122 >> v1:172.21.15.183:6789/0 conn(0x7f17080af4d0 legacy=0x7f170806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f170d133700 1 -- 172.21.15.97:0/910538122 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080ae710 con 0x7f170806c710 2023-11-24T06:10:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f170c932700 1 --2- 172.21.15.97:0/910538122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f17080b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f170d133700 1 --2- 172.21.15.97:0/910538122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f170806c710 0x7f17080afb90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1700009e80 tx=0x7f170000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=271dbbb785a7d4f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f17077fe700 1 -- 172.21.15.97:0/910538122 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170000e070 con 0x7f170806c710 2023-11-24T06:10:03.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f17077fe700 1 -- 172.21.15.97:0/910538122 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170000b8d0 con 0x7f170806c710 2023-11-24T06:10:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f17077fe700 1 -- 172.21.15.97:0/910538122 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170000bc80 con 0x7f170806c710 2023-11-24T06:10:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f170eb96700 1 -- 172.21.15.97:0/910538122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f170806c710 msgr2=0x7f17080afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.296+0000 7f170eb96700 1 --2- 172.21.15.97:0/910538122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f170806c710 0x7f17080afb90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1700009e80 tx=0x7f170000b330 comp rx=0 tx=0).stop 2023-11-24T06:10:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.297+0000 7f170eb96700 1 -- 172.21.15.97:0/910538122 shutdown_connections 2023-11-24T06:10:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.297+0000 7f170eb96700 1 --2- 172.21.15.97:0/910538122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f170806c710 0x7f17080afb90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.297+0000 7f170eb96700 1 --2- 172.21.15.97:0/910538122 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f17080b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.297+0000 7f170eb96700 1 -- 172.21.15.97:0/910538122 >> 172.21.15.97:0/910538122 conn(0x7f1708071b90 msgr2=0x7f1708071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.297+0000 7f170eb96700 1 -- 172.21.15.97:0/910538122 shutdown_connections 2023-11-24T06:10:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.297+0000 7f170eb96700 1 -- 172.21.15.97:0/910538122 wait complete. 2023-11-24T06:10:03.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.298+0000 7f170eb96700 1 Processor -- start 2023-11-24T06:10:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.298+0000 7f170eb96700 1 -- start start 2023-11-24T06:10:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170eb96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f170806c710 0x7f17081551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170eb96700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17080af4d0 0x7f17081566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170eb96700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f1708154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170eb96700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1708156c20 con 0x7f170806c710 2023-11-24T06:10:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170eb96700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17081576a0 con 0x7f17080af4d0 2023-11-24T06:10:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170eb96700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1708156e10 con 0x7f170815e080 2023-11-24T06:10:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f170806c710 0x7f17081551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f170806c710 0x7f17081551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47144/0 (socket says 172.21.15.97:47144) 2023-11-24T06:10:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 -- 172.21.15.97:0/3244930565 learned_addr learned my addr 172.21.15.97:0/3244930565 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170d133700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f1708154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f1707fff700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17080af4d0 0x7f17081566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 -- 172.21.15.97:0/3244930565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 msgr2=0x7f1708154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:03.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f1708154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 -- 172.21.15.97:0/3244930565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17080af4d0 msgr2=0x7f17081566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17080af4d0 0x7f17081566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 -- 172.21.15.97:0/3244930565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc0094a0 con 0x7f170806c710 2023-11-24T06:10:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170c932700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f170806c710 0x7f17081551e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00ebe0 tx=0x7f16fc009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f17057fa700 1 -- 172.21.15.97:0/3244930565 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00c8b0 con 0x7f170806c710 2023-11-24T06:10:03.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f17057fa700 1 -- 172.21.15.97:0/3244930565 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc00ca10 con 0x7f170806c710 2023-11-24T06:10:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1700009b30 con 0x7f170806c710 2023-11-24T06:10:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f17057fa700 1 -- 172.21.15.97:0/3244930565 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00cda0 con 0x7f170806c710 2023-11-24T06:10:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.299+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17081578f0 con 0x7f170806c710 2023-11-24T06:10:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.300+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17080753f0 con 0x7f170806c710 2023-11-24T06:10:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.301+0000 7f17057fa700 1 -- 172.21.15.97:0/3244930565 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f16fc013470 con 0x7f170806c710 2023-11-24T06:10:03.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.301+0000 7f17057fa700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f16ec06aed0 0x7f16ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.301+0000 7f17057fa700 1 -- 172.21.15.97:0/3244930565 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16fc089660 con 0x7f170806c710 2023-11-24T06:10:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.301+0000 7f1707fff700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f16ec06aed0 0x7f16ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.302+0000 7f1707fff700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f16ec06aed0 0x7f16ec06d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17080b4270 tx=0x7f16f4007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:03.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.306+0000 7f17057fa700 1 -- 172.21.15.97:0/3244930565 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16fc0547a0 con 0x7f170806c710 2023-11-24T06:10:03.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.541+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1708067f60 con 0x7f170806c710 2023-11-24T06:10:03.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.541+0000 7f17057fa700 1 -- 172.21.15.97:0/3244930565 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f16fc057dc0 con 0x7f170806c710 2023-11-24T06:10:03.544 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:10:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.543+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f16ec06aed0 msgr2=0x7f16ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:03.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.543+0000 7f170eb96700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f16ec06aed0 0x7f16ec06d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f17080b4270 tx=0x7f16f4007d30 comp rx=0 tx=0).stop 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.543+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f170806c710 msgr2=0x7f17081551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.543+0000 7f170eb96700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f170806c710 0x7f17081551e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00ebe0 tx=0x7f16fc009e20 comp rx=0 tx=0).stop 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.544+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 shutdown_connections 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.544+0000 7f170eb96700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f16ec06aed0 0x7f16ec06d390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.544+0000 7f170eb96700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f170815e080 0x7f1708154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.544+0000 7f170eb96700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f170806c710 0x7f17081551e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.544+0000 7f170eb96700 1 --2- 172.21.15.97:0/3244930565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17080af4d0 0x7f17081566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.544+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 >> 172.21.15.97:0/3244930565 conn(0x7f1708071b90 msgr2=0x7f17080b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.544+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 shutdown_connections 2023-11-24T06:10:03.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:03.544+0000 7f170eb96700 1 -- 172.21.15.97:0/3244930565 wait complete. 2023-11-24T06:10:03.558 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:10:03.558 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:10:03.558 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-11-24T06:10:03.558 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:10:03.559 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:03.559 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:10:03.559 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:10:03.559 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:10:03.559 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:10:03.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:03.559 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-11-24T06:10:03.581 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:03.581 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:03.581 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:03.582 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42125sec preferred_lft 42125sec 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::5382:2bb9:1085:5950/64 scope link noprefixroute 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::b8cd:8aff:fe67:b24d/64 scope link 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether fe:4f:a7:e6:1c:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:03.583 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::fc4f:a7ff:fee6:1c7f/64 scope link 2023-11-24T06:10:03.584 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:03.584 INFO:teuthology.orchestra.run.smithi183.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:03.584 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:10:03.584 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::4034:29ff:fe35:b2a5/64 scope link 2023-11-24T06:10:03.584 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:03.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:03.584 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:10:03.654 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-11-24T06:10:03.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:03.655 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-11-24T06:10:03.718 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-11-24T06:10:03.718 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:10:03.718 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:10:03.718 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:10:03.718 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:10:03.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:03.719 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:10:03.719 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:10:03.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:10:03.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:10:03.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:10:03.719 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:10:03.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:03.827 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:10:03.844 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:10:03.844 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:10:03.845 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:10:03.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:03.845 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2023-11-24T06:10:03.912 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:03.913 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:03.969 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:10:03.969 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:03.969 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:04.026 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:10:04.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:04.026 DEBUG:teuthology.orchestra.run.smithi183:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:04.113 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:04.113 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:04.137 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:04.137 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:04.137 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:10:04.138 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:10:04.138 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:10:04.138 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:10:04.138 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:10:04.138 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:10:04.211 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:04.211 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:04.211 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:10:04.211 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:10:04.211 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:10:04.212 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:10:04.212 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:10:04.212 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:10:04.212 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:10:04.212 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:10:04.212 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:10:04.213 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:10:04.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:04.214 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:04.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:04.239 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:10:04.312 INFO:teuthology.orchestra.run.smithi183.stdout:client5268 2023-11-24T06:10:04.312 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:04.312 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:04.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:10:04.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.497+0000 7fa56dbc5700 1 Processor -- start 2023-11-24T06:10:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.497+0000 7fa56dbc5700 1 -- start start 2023-11-24T06:10:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.498+0000 7fa56dbc5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa56806e9c0 0x7fa5680b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.498+0000 7fa56dbc5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56806ee50 con 0x7fa5680b01e0 2023-11-24T06:10:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.498+0000 7fa56dbc5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5680b05c0 con 0x7fa5680746a0 2023-11-24T06:10:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.498+0000 7fa56dbc5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5681470a0 con 0x7fa56806e9c0 2023-11-24T06:10:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.498+0000 7fa5677fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa5680b01e0 0x7fa568073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39604/0 (socket says 172.21.15.97:39604) 2023-11-24T06:10:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.498+0000 7fa5677fe700 1 -- 172.21.15.97:0/3350617389 learned_addr learned my addr 172.21.15.97:0/3350617389 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.498+0000 7fa567fff700 1 --2- 172.21.15.97:0/3350617389 >> v2:172.21.15.183:3300/0 conn(0x7fa56806e9c0 0x7fa5680b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56892861 0 0) 0x7fa56806ee50 con 0x7fa5680b01e0 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5500035b0 con 0x7fa5680b01e0 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174746588 0 0) 0x7fa5500035b0 con 0x7fa5680b01e0 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 >> v2:172.21.15.183:3300/0 conn(0x7fa56806e9c0 msgr2=0x7fa5680b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 --2- 172.21.15.97:0/3350617389 >> v2:172.21.15.183:3300/0 conn(0x7fa56806e9c0 0x7fa5680b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 >> v1:172.21.15.183:6789/0 conn(0x7fa5680746a0 legacy=0x7fa56806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680b4250 con 0x7fa5680b01e0 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fa55c001670 con 0x7fa5680b01e0 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 >> v1:172.21.15.97:6789/0 conn(0x7fa5680b01e0 legacy=0x7fa568073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa550003a00 0x7fa550003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa550004850 0x7fa550004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa55000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5681470a0 con 0x7fa550005210 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55000c1b0 con 0x7fa550003a00 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55000c370 con 0x7fa550004850 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa567fff700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa55000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa5677fe700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa550003a00 0x7fa550003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa566ffd700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa550004850 0x7fa550004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa567fff700 1 -- 172.21.15.97:0/3350617389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa550004850 msgr2=0x7fa550004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa567fff700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa550004850 0x7fa550004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa567fff700 1 -- 172.21.15.97:0/3350617389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa550003a00 msgr2=0x7fa550003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:04.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa567fff700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa550003a00 0x7fa550003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa567fff700 1 -- 172.21.15.97:0/3350617389 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500035b0 con 0x7fa550005210 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa567fff700 1 -- 172.21.15.97:0/3350617389 reap_dead start 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.499+0000 7fa567fff700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa55000ba10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa560009fc0 tx=0x7fa550005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ecf8538d94d99b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560003a20 con 0x7fa550005210 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56000f040 con 0x7fa550005210 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa5667fc700 1 -- 172.21.15.97:0/3350617389 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560003de0 con 0x7fa550005210 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3350617389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 msgr2=0x7fa55000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa56dbc5700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa55000ba10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa560009fc0 tx=0x7fa550005880 comp rx=0 tx=0).stop 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3350617389 shutdown_connections 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa56dbc5700 1 --2- 172.21.15.97:0/3350617389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa55000ba10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3350617389 >> 172.21.15.97:0/3350617389 conn(0x7fa5680d0b40 msgr2=0x7fa5680d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3350617389 shutdown_connections 2023-11-24T06:10:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.500+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3350617389 wait complete. 2023-11-24T06:10:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.501+0000 7fa56dbc5700 1 Processor -- start 2023-11-24T06:10:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa56dbc5700 1 -- start start 2023-11-24T06:10:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa56dbc5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa56815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa56dbc5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa56815b970 0x7fa568134530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa56dbc5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa568134a70 0x7fa5681334b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa56dbc5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa568134ef0 con 0x7fa550005210 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa56dbc5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5681324a0 con 0x7fa568134a70 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa56dbc5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa568133020 con 0x7fa56815b970 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa5677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa56815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa5677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa56815e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47164/0 (socket says 172.21.15.97:47164) 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa5677fe700 1 -- 172.21.15.97:0/3747798543 learned_addr learned my addr 172.21.15.97:0/3747798543 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa567fff700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa568134a70 0x7fa5681334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.502+0000 7fa566ffd700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa56815b970 0x7fa568134530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa5677fe700 1 -- 172.21.15.97:0/3747798543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa56815b970 msgr2=0x7fa568134530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:04.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa5677fe700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa56815b970 0x7fa568134530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa5677fe700 1 -- 172.21.15.97:0/3747798543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa568134a70 msgr2=0x7fa5681334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa5677fe700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa568134a70 0x7fa5681334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa5677fe700 1 -- 172.21.15.97:0/3747798543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56815eb40 con 0x7fa550005210 2023-11-24T06:10:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa5677fe700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa56815e600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0072f0 tx=0x7fa55c007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:04.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa54ffff700 1 -- 172.21.15.97:0/3747798543 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c011580 con 0x7fa550005210 2023-11-24T06:10:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa54ffff700 1 -- 172.21.15.97:0/3747798543 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c0116e0 con 0x7fa550005210 2023-11-24T06:10:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa54ffff700 1 -- 172.21.15.97:0/3747798543 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c011a70 con 0x7fa550005210 2023-11-24T06:10:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5681326f0 con 0x7fa550005210 2023-11-24T06:10:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.503+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa568132cb0 con 0x7fa550005210 2023-11-24T06:10:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.504+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa568067f60 con 0x7fa550005210 2023-11-24T06:10:04.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.504+0000 7fa54ffff700 1 -- 172.21.15.97:0/3747798543 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa55c011bd0 con 0x7fa550005210 2023-11-24T06:10:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.505+0000 7fa54ffff700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa55406aed0 0x7fa55406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.505+0000 7fa54ffff700 1 -- 172.21.15.97:0/3747798543 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa55c088b80 con 0x7fa550005210 2023-11-24T06:10:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.505+0000 7fa566ffd700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa55406aed0 0x7fa55406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:04.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.505+0000 7fa566ffd700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa55406aed0 0x7fa55406d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa568142d70 tx=0x7fa558009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:04.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.509+0000 7fa54ffff700 1 -- 172.21.15.97:0/3747798543 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa55c053cc0 con 0x7fa550005210 2023-11-24T06:10:04.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:04.743+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa56806c980 con 0x7fa550005210 2023-11-24T06:10:05.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.080+0000 7fa54ffff700 1 -- 172.21.15.97:0/3747798543 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7fa55c01a0c0 con 0x7fa550005210 2023-11-24T06:10:05.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa55406aed0 msgr2=0x7fa55406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa55406aed0 0x7fa55406d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa568142d70 tx=0x7fa558009040 comp rx=0 tx=0).stop 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 msgr2=0x7fa56815e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa56815e600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0072f0 tx=0x7fa55c007600 comp rx=0 tx=0).stop 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 shutdown_connections 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa568134a70 0x7fa5681334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa550005210 0x7fa56815e600 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa55406aed0 0x7fa55406d390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 --2- 172.21.15.97:0/3747798543 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa56815b970 0x7fa568134530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 >> 172.21.15.97:0/3747798543 conn(0x7fa5680d0b40 msgr2=0x7fa5680b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 shutdown_connections 2023-11-24T06:10:05.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.083+0000 7fa56dbc5700 1 -- 172.21.15.97:0/3747798543 wait complete. 2023-11-24T06:10:05.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:10:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.217+0000 7f1bbbea1700 1 Processor -- start 2023-11-24T06:10:05.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.218+0000 7f1bbbea1700 1 -- start start 2023-11-24T06:10:05.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.218+0000 7f1bbbea1700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1bb40d2380 0x7f1bb41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.218+0000 7f1bbbea1700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb40b22d0 con 0x7f1bb40b3370 2023-11-24T06:10:05.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.218+0000 7f1bbbea1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb40b25b0 con 0x7f1bb415bdd0 2023-11-24T06:10:05.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.218+0000 7f1bbbea1700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb40b2f50 con 0x7f1bb40d2380 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.218+0000 7f1bb943c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1bb40b3370 0x7f1bb40d1c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39626/0 (socket says 172.21.15.97:39626) 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.218+0000 7f1bb943c700 1 -- 172.21.15.97:0/3723308897 learned_addr learned my addr 172.21.15.97:0/3723308897 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bba43e700 1 --2- 172.21.15.97:0/3723308897 >> v2:172.21.15.183:3300/0 conn(0x7f1bb40d2380 0x7f1bb41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202510223 0 0) 0x7f1bb40b22d0 con 0x7f1bb40b3370 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba00035b0 con 0x7f1bb40b3370 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2184467819 0 0) 0x7f1ba00035b0 con 0x7f1bb40b3370 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 >> v2:172.21.15.183:3300/0 conn(0x7f1bb40d2380 msgr2=0x7f1bb41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 --2- 172.21.15.97:0/3723308897 >> v2:172.21.15.183:3300/0 conn(0x7f1bb40d2380 0x7f1bb41604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 >> v1:172.21.15.183:6789/0 conn(0x7f1bb415bdd0 legacy=0x7f1bb40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb40b2890 con 0x7f1bb40b3370 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f1bb0001670 con 0x7f1bb40b3370 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 >> v1:172.21.15.97:6789/0 conn(0x7f1bb40b3370 legacy=0x7f1bb40d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ba0003a00 0x7f1ba0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1ba0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ba0005210 0x7f1ba000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40b2f50 con 0x7f1ba0004850 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba000c0b0 con 0x7f1ba0005210 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba000c270 con 0x7f1ba0003a00 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb943c700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1ba0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb9c3d700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ba0003a00 0x7f1ba0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bba43e700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ba0005210 0x7f1ba000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb943c700 1 -- 172.21.15.97:0/3723308897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ba0003a00 msgr2=0x7f1ba0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb943c700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ba0003a00 0x7f1ba0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb943c700 1 -- 172.21.15.97:0/3723308897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ba0005210 msgr2=0x7f1ba000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb943c700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ba0005210 0x7f1ba000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bb943c700 1 -- 172.21.15.97:0/3723308897 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00035b0 con 0x7f1ba0004850 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.219+0000 7f1bba43e700 1 -- 172.21.15.97:0/3723308897 reap_dead start 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bb943c700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1ba0004cd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000c240 tx=0x7f1ba0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0ef56c19eaeb47 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0007c10 con 0x7f1ba0004850 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb000cda0 con 0x7f1ba0004850 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bb8c3b700 1 -- 172.21.15.97:0/3723308897 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0005ba0 con 0x7f1ba0004850 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3723308897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 msgr2=0x7f1ba0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bbbea1700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1ba0004cd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000c240 tx=0x7f1ba0005880 comp rx=0 tx=0).stop 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3723308897 shutdown_connections 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bbbea1700 1 --2- 172.21.15.97:0/3723308897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1ba0004cd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3723308897 >> 172.21.15.97:0/3723308897 conn(0x7f1bb40731a0 msgr2=0x7f1bb409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:05.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3723308897 shutdown_connections 2023-11-24T06:10:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.220+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3723308897 wait complete. 2023-11-24T06:10:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bbbea1700 1 Processor -- start 2023-11-24T06:10:05.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bbbea1700 1 -- start start 2023-11-24T06:10:05.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bbbea1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1bb40ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bbbea1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb40afd00 0x7f1bb40c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bbbea1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb40b0180 0x7f1bb414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bbbea1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41575c0 con 0x7f1ba0004850 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bbbea1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb406b460 con 0x7f1bb40afd00 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bbbea1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb406b740 con 0x7f1bb40b0180 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bb9c3d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1bb40ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bb9c3d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1bb40ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47184/0 (socket says 172.21.15.97:47184) 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bb9c3d700 1 -- 172.21.15.97:0/3612185433 learned_addr learned my addr 172.21.15.97:0/3612185433 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.222+0000 7f1bb943c700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb40afd00 0x7f1bb40c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bba43e700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb40b0180 0x7f1bb414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bb9c3d700 1 -- 172.21.15.97:0/3612185433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb40b0180 msgr2=0x7f1bb414bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bb9c3d700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb40b0180 0x7f1bb414bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bb9c3d700 1 -- 172.21.15.97:0/3612185433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb40afd00 msgr2=0x7f1bb40c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bb9c3d700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb40afd00 0x7f1bb40c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bb9c3d700 1 -- 172.21.15.97:0/3612185433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb406ba20 con 0x7f1ba0004850 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bb9c3d700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1bb40ca7d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400f820 tx=0x7f1ba400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1baa7fc700 1 -- 172.21.15.97:0/3612185433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40052e0 con 0x7f1ba0004850 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1baa7fc700 1 -- 172.21.15.97:0/3612185433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba4005440 con 0x7f1ba0004850 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb406bd00 con 0x7f1ba0004850 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1baa7fc700 1 -- 172.21.15.97:0/3612185433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40038a0 con 0x7f1ba0004850 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.223+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb40cad10 con 0x7f1ba0004850 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.224+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb406c350 con 0x7f1ba0004850 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.224+0000 7f1baa7fc700 1 -- 172.21.15.97:0/3612185433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ba4003a00 con 0x7f1ba0004850 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.225+0000 7f1baa7fc700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.225+0000 7f1bb943c700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.225+0000 7f1baa7fc700 1 -- 172.21.15.97:0/3612185433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ba4089280 con 0x7f1ba0004850 2023-11-24T06:10:05.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.225+0000 7f1bb943c700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b9806aed0 0x7f1b9806d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0001930 tx=0x7f1bb0007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.229+0000 7f1baa7fc700 1 -- 172.21.15.97:0/3612185433 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ba40542a0 con 0x7f1ba0004850 2023-11-24T06:10:05.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.446+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bb41604e0 con 0x7f1b9806aed0 2023-11-24T06:10:05.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.448+0000 7f1baa7fc700 1 -- 172.21.15.97:0/3612185433 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1bb0007b30 con 0x7f1b9806aed0 2023-11-24T06:10:05.450 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:10:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephfs" 2023-11-24T06:10:05.451 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:05.451 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:10:05.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b9806aed0 msgr2=0x7f1b9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b9806aed0 0x7f1b9806d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0001930 tx=0x7f1bb0007000 comp rx=0 tx=0).stop 2023-11-24T06:10:05.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 msgr2=0x7f1bb40ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1bb40ca7d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400f820 tx=0x7f1ba400fb30 comp rx=0 tx=0).stop 2023-11-24T06:10:05.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 shutdown_connections 2023-11-24T06:10:05.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1bb40b0180 0x7f1bb414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ba0004850 0x7f1bb40ca7d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 --2- 172.21.15.97:0/3612185433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1bb40afd00 0x7f1bb40c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 >> 172.21.15.97:0/3612185433 conn(0x7f1bb40731a0 msgr2=0x7f1bb409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:05.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 shutdown_connections 2023-11-24T06:10:05.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.450+0000 7f1bbbea1700 1 -- 172.21.15.97:0/3612185433 wait complete. 2023-11-24T06:10:05.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:10:05.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.584+0000 7f68771e7700 1 Processor -- start 2023-11-24T06:10:05.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.584+0000 7f68771e7700 1 -- start start 2023-11-24T06:10:05.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f68771e7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f687015e160 0x7f68700b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f68771e7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f68700afc90 0x7f6870073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f68771e7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68700b3c60 con 0x7f68700744d0 2023-11-24T06:10:05.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f68771e7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68700b4500 con 0x7f687015e160 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f68771e7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6870074940 con 0x7f68700afc90 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f6874f83700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f687015e160 0x7f68700b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f6875784700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f68700744d0 0x7f687006d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39648/0 (socket says 172.21.15.97:39648) 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f6875784700 1 -- 172.21.15.97:0/197496005 learned_addr learned my addr 172.21.15.97:0/197496005 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.585+0000 7f686ffff700 1 --2- 172.21.15.97:0/197496005 >> v2:172.21.15.183:3300/0 conn(0x7f68700afc90 0x7f6870073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f6874f83700 1 -- 172.21.15.97:0/197496005 >> v2:172.21.15.183:3300/0 conn(0x7f68700afc90 msgr2=0x7f6870073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f6874f83700 1 --2- 172.21.15.97:0/197496005 >> v2:172.21.15.183:3300/0 conn(0x7f68700afc90 0x7f6870073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f6874f83700 1 -- 172.21.15.97:0/197496005 >> v1:172.21.15.97:6789/0 conn(0x7f68700744d0 legacy=0x7f687006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f6874f83700 1 -- 172.21.15.97:0/197496005 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700b58c0 con 0x7f687015e160 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f686f7fe700 1 -- 172.21.15.97:0/197496005 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139091080 0 0) 0x7f68700b3c60 con 0x7f68700744d0 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f6874f83700 1 --2- 172.21.15.97:0/197496005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f687015e160 0x7f68700b5da0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f68700d0880 tx=0x7f6864009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4b3448f57fcae8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f686f7fe700 1 -- 172.21.15.97:0/197496005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864011830 con 0x7f687015e160 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f686f7fe700 1 -- 172.21.15.97:0/197496005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864011990 con 0x7f687015e160 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f686f7fe700 1 -- 172.21.15.97:0/197496005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864011d20 con 0x7f687015e160 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f68771e7700 1 -- 172.21.15.97:0/197496005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f687015e160 msgr2=0x7f68700b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f68771e7700 1 --2- 172.21.15.97:0/197496005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f687015e160 0x7f68700b5da0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f68700d0880 tx=0x7f6864009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f68771e7700 1 -- 172.21.15.97:0/197496005 shutdown_connections 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f68771e7700 1 --2- 172.21.15.97:0/197496005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f687015e160 0x7f68700b5da0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f68771e7700 1 --2- 172.21.15.97:0/197496005 >> v2:172.21.15.183:3300/0 conn(0x7f68700afc90 0x7f6870073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f68771e7700 1 -- 172.21.15.97:0/197496005 >> 172.21.15.97:0/197496005 conn(0x7f68700d0bf0 msgr2=0x7f68700d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f68771e7700 1 -- 172.21.15.97:0/197496005 shutdown_connections 2023-11-24T06:10:05.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.586+0000 7f68771e7700 1 -- 172.21.15.97:0/197496005 wait complete. 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.588+0000 7f68771e7700 1 Processor -- start 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.588+0000 7f68771e7700 1 -- start start 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f68771e7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68700744d0 0x7f68700c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f68771e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68700afc90 0x7f687006df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f68771e7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687015e160 0x7f68701497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f68771e7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c8a50 con 0x7f68700afc90 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f68771e7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687006e450 con 0x7f687015e160 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f68771e7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68700c7dd0 con 0x7f68700744d0 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68700afc90 0x7f687006df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68700afc90 0x7f687006df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47200/0 (socket says 172.21.15.97:47200) 2023-11-24T06:10:05.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 -- 172.21.15.97:0/1433204191 learned_addr learned my addr 172.21.15.97:0/1433204191 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f6875784700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687015e160 0x7f68701497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 -- 172.21.15.97:0/1433204191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68700744d0 msgr2=0x7f68700c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f6874f83700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68700744d0 0x7f68700c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68700744d0 0x7f68700c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 -- 172.21.15.97:0/1433204191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687015e160 msgr2=0x7f68701497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687015e160 0x7f68701497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 -- 172.21.15.97:0/1433204191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68640094a0 con 0x7f68700afc90 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686ffff700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68700afc90 0x7f687006df10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f685c00c7f0 tx=0x7f685c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686d7fa700 1 -- 172.21.15.97:0/1433204191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c011800 con 0x7f68700afc90 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686d7fa700 1 -- 172.21.15.97:0/1433204191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685c011960 con 0x7f68700afc90 2023-11-24T06:10:05.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68700c80b0 con 0x7f68700afc90 2023-11-24T06:10:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f686d7fa700 1 -- 172.21.15.97:0/1433204191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c011cf0 con 0x7f68700afc90 2023-11-24T06:10:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.589+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68700c8670 con 0x7f68700afc90 2023-11-24T06:10:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.590+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6870068fb0 con 0x7f68700afc90 2023-11-24T06:10:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.591+0000 7f686d7fa700 1 -- 172.21.15.97:0/1433204191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f685c018360 con 0x7f68700afc90 2023-11-24T06:10:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.591+0000 7f686d7fa700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f685406aed0 0x7f685406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.591+0000 7f686d7fa700 1 -- 172.21.15.97:0/1433204191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f685c089610 con 0x7f68700afc90 2023-11-24T06:10:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.591+0000 7f6874f83700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f685406aed0 0x7f685406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.592+0000 7f6874f83700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f685406aed0 0x7f685406d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68640097c0 tx=0x7f6864008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.596+0000 7f686d7fa700 1 -- 172.21.15.97:0/1433204191 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f685c0546d0 con 0x7f68700afc90 2023-11-24T06:10:05.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.785+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f687006c140 con 0x7f68700afc90 2023-11-24T06:10:05.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.785+0000 7f686d7fa700 1 -- 172.21.15.97:0/1433204191 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f685c057cf0 con 0x7f68700afc90 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f685406aed0 msgr2=0x7f685406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f685406aed0 0x7f685406d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f68640097c0 tx=0x7f6864008030 comp rx=0 tx=0).stop 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68700afc90 msgr2=0x7f687006df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68700afc90 0x7f687006df10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f685c00c7f0 tx=0x7f685c00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 shutdown_connections 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f687015e160 0x7f68701497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f685406aed0 0x7f685406d390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f68700744d0 0x7f68700c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 --2- 172.21.15.97:0/1433204191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68700afc90 0x7f687006df10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.787+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 >> 172.21.15.97:0/1433204191 conn(0x7f68700d0bf0 msgr2=0x7f6870160550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.788+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 shutdown_connections 2023-11-24T06:10:05.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.788+0000 7f68771e7700 1 -- 172.21.15.97:0/1433204191 wait complete. 2023-11-24T06:10:05.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp7976 2023-11-24T06:10:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.923+0000 7fa1b56ad700 1 Processor -- start 2023-11-24T06:10:05.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.923+0000 7fa1b56ad700 1 -- start start 2023-11-24T06:10:05.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.923+0000 7fa1b56ad700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1b015dfa0 0x7fa1b006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.923+0000 7fa1b56ad700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b00d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.923+0000 7fa1b56ad700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b006bf40 con 0x7fa1b00b6ec0 2023-11-24T06:10:05.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.923+0000 7fa1b56ad700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b00b7a30 con 0x7fa1b015dfa0 2023-11-24T06:10:05.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.923+0000 7fa1b56ad700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b006c210 con 0x7fa1b00cff20 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1aeffd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1b015dfa0 0x7fa1b006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1af7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b00d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1ae7fc700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fa1b00b6ec0 0x7fa1b00b7320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:48982/0 (socket says 172.21.15.97:48982) 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1ae7fc700 1 -- 172.21.15.97:0/84162481 learned_addr learned my addr 172.21.15.97:0/84162481 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1aeffd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1b015dfa0 0x7fa1b006ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:41366/0 (socket says 172.21.15.97:41366) 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1af7fe700 1 -- 172.21.15.97:0/84162481 >> v2:172.21.15.183:3300/0 conn(0x7fa1b015dfa0 msgr2=0x7fa1b006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1af7fe700 1 --2- 172.21.15.97:0/84162481 >> v2:172.21.15.183:3300/0 conn(0x7fa1b015dfa0 0x7fa1b006ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1af7fe700 1 -- 172.21.15.97:0/84162481 >> v1:172.21.15.183:6789/0 conn(0x7fa1b00b6ec0 legacy=0x7fa1b00b7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1af7fe700 1 -- 172.21.15.97:0/84162481 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b00b4b30 con 0x7fa1b00cff20 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.924+0000 7fa1af7fe700 1 --2- 172.21.15.97:0/84162481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b00d0390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009d90 tx=0x7fa1a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef478716c4d1993a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1adffb700 1 -- 172.21.15.97:0/84162481 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800e070 con 0x7fa1b00cff20 2023-11-24T06:10:05.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1adffb700 1 -- 172.21.15.97:0/84162481 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a800bb20 con 0x7fa1b00cff20 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1adffb700 1 -- 172.21.15.97:0/84162481 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8012430 con 0x7fa1b00cff20 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1b56ad700 1 -- 172.21.15.97:0/84162481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 msgr2=0x7fa1b00d0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/84162481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b00d0390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009d90 tx=0x7fa1a800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1b56ad700 1 -- 172.21.15.97:0/84162481 shutdown_connections 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/84162481 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b00d0390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/84162481 >> v2:172.21.15.183:3300/0 conn(0x7fa1b015dfa0 0x7fa1b006ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1b56ad700 1 -- 172.21.15.97:0/84162481 >> 172.21.15.97:0/84162481 conn(0x7fa1b00b3810 msgr2=0x7fa1b00b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1b56ad700 1 -- 172.21.15.97:0/84162481 shutdown_connections 2023-11-24T06:10:05.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.925+0000 7fa1b56ad700 1 -- 172.21.15.97:0/84162481 wait complete. 2023-11-24T06:10:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.926+0000 7fa1b56ad700 1 Processor -- start 2023-11-24T06:10:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.926+0000 7fa1b56ad700 1 -- start start 2023-11-24T06:10:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1b56ad700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b00b6ec0 0x7fa1b00c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1b56ad700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1b56ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b015dfa0 0x7fa1b01553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1b56ad700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b0155b50 con 0x7fa1b015dfa0 2023-11-24T06:10:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1b56ad700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b0155280 con 0x7fa1b00b6ec0 2023-11-24T06:10:05.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1b56ad700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b01f6b80 con 0x7fa1b00cff20 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1af7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b015dfa0 0x7fa1b01553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1af7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b015dfa0 0x7fa1b01553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47216/0 (socket says 172.21.15.97:47216) 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1af7fe700 1 -- 172.21.15.97:0/1706630492 learned_addr learned my addr 172.21.15.97:0/1706630492 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1ae7fc700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.927+0000 7fa1aeffd700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b00b6ec0 0x7fa1b00c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1af7fe700 1 -- 172.21.15.97:0/1706630492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 msgr2=0x7fa1b0154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1af7fe700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b0154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1af7fe700 1 -- 172.21.15.97:0/1706630492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b00b6ec0 msgr2=0x7fa1b00c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1af7fe700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b00b6ec0 0x7fa1b00c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1af7fe700 1 -- 172.21.15.97:0/1706630492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a8009a70 con 0x7fa1b015dfa0 2023-11-24T06:10:05.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1af7fe700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b015dfa0 0x7fa1b01553e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1b006c350 tx=0x7fa1a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1a77fe700 1 -- 172.21.15.97:0/1706630492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8012430 con 0x7fa1b015dfa0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1a77fe700 1 -- 172.21.15.97:0/1706630492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a8020070 con 0x7fa1b015dfa0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1a77fe700 1 -- 172.21.15.97:0/1706630492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8004280 con 0x7fa1b015dfa0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b0155920 con 0x7fa1b015dfa0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.928+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b015fc00 con 0x7fa1b015dfa0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.929+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b0160530 con 0x7fa1b015dfa0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.929+0000 7fa1a77fe700 1 -- 172.21.15.97:0/1706630492 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1a80043e0 con 0x7fa1b015dfa0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.930+0000 7fa1a77fe700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa19006aed0 0x7fa19006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.930+0000 7fa1a77fe700 1 -- 172.21.15.97:0/1706630492 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1a8022070 con 0x7fa1b015dfa0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.930+0000 7fa1aeffd700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa19006aed0 0x7fa19006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:05.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.930+0000 7fa1aeffd700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa19006aed0 0x7fa19006d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009be0 tx=0x7fa1a0009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:05.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:05.934+0000 7fa1a77fe700 1 -- 172.21.15.97:0/1706630492 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1a8056310 con 0x7fa1b015dfa0 2023-11-24T06:10:06.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.155+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp7976", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1b00b53d0 con 0x7fa19006aed0 2023-11-24T06:10:06.162 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:06.159+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:06.162 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:06.159+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:06.162 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:06.159+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:06.162 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:06.159+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:06.162 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:06.159+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:06.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.171+0000 7fa1a77fe700 1 -- 172.21.15.97:0/1706630492 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa1a0004020 con 0x7fa19006aed0 2023-11-24T06:10:06.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.173+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa19006aed0 msgr2=0x7fa19006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.173+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa19006aed0 0x7fa19006d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009be0 tx=0x7fa1a0009210 comp rx=0 tx=0).stop 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.173+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b015dfa0 msgr2=0x7fa1b01553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.173+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b015dfa0 0x7fa1b01553e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa1b006c350 tx=0x7fa1a8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.174+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 shutdown_connections 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.174+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1b015dfa0 0x7fa1b01553e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.174+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1b00b6ec0 0x7fa1b00c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.174+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa19006aed0 0x7fa19006d390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.174+0000 7fa1b56ad700 1 --2- 172.21.15.97:0/1706630492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1b00cff20 0x7fa1b0154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.174+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 >> 172.21.15.97:0/1706630492 conn(0x7fa1b00b3810 msgr2=0x7fa1b006c380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.174+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 shutdown_connections 2023-11-24T06:10:06.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.174+0000 7fa1b56ad700 1 -- 172.21.15.97:0/1706630492 wait complete. 2023-11-24T06:10:06.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create cephfs subvol_grp7976 subvol_snap2221 2023-11-24T06:10:06.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.311+0000 7f9662477700 1 Processor -- start 2023-11-24T06:10:06.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.311+0000 7f9662477700 1 -- start start 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f9662477700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f965c0b18f0 0x7f965c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f9662477700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c161c80 con 0x7f965c15e020 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f9662477700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965c161f40 con 0x7f965c0cf910 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f9662477700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965c162220 con 0x7f965c0b18f0 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f965c15e020 0x7f965c0b11e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39686/0 (socket says 172.21.15.97:39686) 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965bfff700 1 -- 172.21.15.97:0/1392381693 learned_addr learned my addr 172.21.15.97:0/1392381693 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965b7fe700 1 --2- 172.21.15.97:0/1392381693 >> v2:172.21.15.183:3300/0 conn(0x7f965c0b18f0 0x7f965c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485990852 0 0) 0x7f965c161c80 con 0x7f965c15e020 2023-11-24T06:10:06.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96440035b0 con 0x7f965c15e020 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29786999 0 0) 0x7f965c161f40 con 0x7f965c0cf910 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f965c161c80 con 0x7f965c0cf910 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 740231790 0 0) 0x7f96440035b0 con 0x7f965c15e020 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 >> v2:172.21.15.183:3300/0 conn(0x7f965c0b18f0 msgr2=0x7f965c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 --2- 172.21.15.97:0/1392381693 >> v2:172.21.15.183:3300/0 conn(0x7f965c0b18f0 0x7f965c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 >> v1:172.21.15.183:6789/0 conn(0x7f965c0cf910 legacy=0x7f965c0af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c162500 con 0x7f965c15e020 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f9650001670 con 0x7f965c15e020 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 >> v1:172.21.15.97:6789/0 conn(0x7f965c15e020 legacy=0x7f965c0b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.312+0000 7f965affd700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9644003b90 0x7f9644003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965affd700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644004850 0x7f9644004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965affd700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9644005210 0x7f964400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c162220 con 0x7f9644003b90 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964400c140 con 0x7f9644004850 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964400c300 con 0x7f9644005210 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965bfff700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9644003b90 0x7f9644003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965b7fe700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644004850 0x7f9644004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f9660a14700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9644005210 0x7f964400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965bfff700 1 -- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9644005210 msgr2=0x7f964400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965bfff700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9644005210 0x7f964400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965bfff700 1 -- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644004850 msgr2=0x7f9644004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965bfff700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644004850 0x7f9644004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965bfff700 1 -- 172.21.15.97:0/1392381693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440035b0 con 0x7f9644003b90 2023-11-24T06:10:06.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f9660a14700 1 -- 172.21.15.97:0/1392381693 reap_dead start 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965b7fe700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644004850 0x7f9644004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965bfff700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9644003b90 0x7f9644003f90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f965000c210 tx=0x7f9644005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4bab7c84a14d5c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965000cdb0 con 0x7f9644003b90 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9650001af0 con 0x7f9644003b90 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.313+0000 7f965affd700 1 -- 172.21.15.97:0/1392381693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650005ba0 con 0x7f9644003b90 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.314+0000 7f9662477700 1 -- 172.21.15.97:0/1392381693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9644003b90 msgr2=0x7f9644003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.314+0000 7f9662477700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9644003b90 0x7f9644003f90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f965000c210 tx=0x7f9644005880 comp rx=0 tx=0).stop 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.314+0000 7f9662477700 1 -- 172.21.15.97:0/1392381693 shutdown_connections 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.314+0000 7f9662477700 1 --2- 172.21.15.97:0/1392381693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9644003b90 0x7f9644003f90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.314+0000 7f9662477700 1 -- 172.21.15.97:0/1392381693 >> 172.21.15.97:0/1392381693 conn(0x7f965c072710 msgr2=0x7f965c072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.314+0000 7f9662477700 1 -- 172.21.15.97:0/1392381693 shutdown_connections 2023-11-24T06:10:06.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.314+0000 7f9662477700 1 -- 172.21.15.97:0/1392381693 wait complete. 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.315+0000 7f9662477700 1 Processor -- start 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.315+0000 7f9662477700 1 -- start start 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9662477700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644003b90 0x7f965c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9662477700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f965c0c50f0 0x7f965c1f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9662477700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f965c1f6b70 0x7f965c160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9662477700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15f080 con 0x7f965c1f6b70 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9662477700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15e400 con 0x7f9644003b90 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9662477700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965c15e6e0 con 0x7f965c0c50f0 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f965c1f6b70 0x7f965c160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f965c1f6b70 0x7f965c160660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47250/0 (socket says 172.21.15.97:47250) 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 -- 172.21.15.97:0/909783621 learned_addr learned my addr 172.21.15.97:0/909783621 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f965b7fe700 1 --2- 172.21.15.97:0/909783621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f965c0c50f0 0x7f965c1f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f965bfff700 1 --2- 172.21.15.97:0/909783621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644003b90 0x7f965c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 -- 172.21.15.97:0/909783621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f965c0c50f0 msgr2=0x7f965c1f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 --2- 172.21.15.97:0/909783621 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f965c0c50f0 0x7f965c1f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 -- 172.21.15.97:0/909783621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644003b90 msgr2=0x7f965c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 --2- 172.21.15.97:0/909783621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9644003b90 0x7f965c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 -- 172.21.15.97:0/909783621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c15e9c0 con 0x7f965c1f6b70 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9660a14700 1 --2- 172.21.15.97:0/909783621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f965c1f6b70 0x7f965c160660 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f965c0cef10 tx=0x7f965400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9658ff9700 1 -- 172.21.15.97:0/909783621 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654003c10 con 0x7f965c1f6b70 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9658ff9700 1 -- 172.21.15.97:0/909783621 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9654003d90 con 0x7f965c1f6b70 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9658ff9700 1 -- 172.21.15.97:0/909783621 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965400c9b0 con 0x7f965c1f6b70 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.316+0000 7f9662477700 1 -- 172.21.15.97:0/909783621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965c15eca0 con 0x7f965c1f6b70 2023-11-24T06:10:06.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.317+0000 7f9662477700 1 -- 172.21.15.97:0/909783621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965c0c6ce0 con 0x7f965c1f6b70 2023-11-24T06:10:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.318+0000 7f9662477700 1 -- 172.21.15.97:0/909783621 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965c067f60 con 0x7f965c1f6b70 2023-11-24T06:10:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.318+0000 7f9658ff9700 1 -- 172.21.15.97:0/909783621 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f965400cb10 con 0x7f965c1f6b70 2023-11-24T06:10:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.318+0000 7f9658ff9700 1 --2- 172.21.15.97:0/909783621 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f964806af00 0x7f964806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.318+0000 7f965bfff700 1 --2- 172.21.15.97:0/909783621 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f964806af00 0x7f964806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.318+0000 7f9658ff9700 1 -- 172.21.15.97:0/909783621 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96540896b0 con 0x7f965c1f6b70 2023-11-24T06:10:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.319+0000 7f965bfff700 1 --2- 172.21.15.97:0/909783621 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f964806af00 0x7f964806d3c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9650000c00 tx=0x7f965000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.323+0000 7f9658ff9700 1 -- 172.21.15.97:0/909783621 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9654054760 con 0x7f965c1f6b70 2023-11-24T06:10:06.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.541+0000 7f9662477700 1 -- 172.21.15.97:0/909783621 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "cephfs", "group_name": "subvol_grp7976", "snap_name": "subvol_snap2221", "target": ["mon-mgr", ""]}) v1 -- 0x7f965c15e850 con 0x7f964806af00 2023-11-24T06:10:06.546 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:06.543+0000 7fc9c0830700 -1 mgr.server reply reply (38) Function not implemented subvolume group snapshots are not supported 2023-11-24T06:10:06.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.543+0000 7f9658ff9700 1 -- 172.21.15.97:0/909783621 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: -38 subvolume group snapshots are not supported) v1 ==== 51+0+0 (secure 0 0 0) 0x7f965c15e850 con 0x7f964806af00 2023-11-24T06:10:06.548 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENOSYS: subvolume group snapshots are not supported 2023-11-24T06:10:06.551 DEBUG:teuthology.orchestra.run:got remote process result: 38 2023-11-24T06:10:06.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp7976 2023-11-24T06:10:06.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.672+0000 7f9bca24f700 1 Processor -- start 2023-11-24T06:10:06.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.672+0000 7f9bca24f700 1 -- start start 2023-11-24T06:10:06.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bca24f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bc40c6c30 0x7f9bc40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bca24f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc40d0140 con 0x7f9bc415bf50 2023-11-24T06:10:06.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bca24f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc40cf3d0 con 0x7f9bc40cef40 2023-11-24T06:10:06.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bca24f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc415c330 con 0x7f9bc40c6c30 2023-11-24T06:10:06.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc2ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9bc40c6c30 0x7f9bc40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc37fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9bc415bf50 0x7f9bc40ce2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39706/0 (socket says 172.21.15.97:39706) 2023-11-24T06:10:06.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc37fe700 1 -- 172.21.15.97:0/4279554040 learned_addr learned my addr 172.21.15.97:0/4279554040 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc2ffd700 1 -- 172.21.15.97:0/4279554040 >> v1:172.21.15.183:6789/0 conn(0x7f9bc40cef40 legacy=0x7f9bc40cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc2ffd700 1 -- 172.21.15.97:0/4279554040 >> v1:172.21.15.97:6789/0 conn(0x7f9bc415bf50 legacy=0x7f9bc40ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc2ffd700 1 -- 172.21.15.97:0/4279554040 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40d03e0 con 0x7f9bc40c6c30 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc27fc700 1 -- 172.21.15.97:0/4279554040 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171062416 0 0) 0x7f9bc40d0140 con 0x7f9bc415bf50 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc2ffd700 1 --2- 172.21.15.97:0/4279554040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc40cea00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009840 tx=0x7f9bb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d3ce5a3dd56fd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc27fc700 1 -- 172.21.15.97:0/4279554040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4004230 con 0x7f9bc40c6c30 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc27fc700 1 -- 172.21.15.97:0/4279554040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb4004390 con 0x7f9bc40c6c30 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.673+0000 7f9bc27fc700 1 -- 172.21.15.97:0/4279554040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb40036a0 con 0x7f9bc40c6c30 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.674+0000 7f9bca24f700 1 -- 172.21.15.97:0/4279554040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 msgr2=0x7f9bc40cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.674+0000 7f9bca24f700 1 --2- 172.21.15.97:0/4279554040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc40cea00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009840 tx=0x7f9bb4009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.674+0000 7f9bca24f700 1 -- 172.21.15.97:0/4279554040 shutdown_connections 2023-11-24T06:10:06.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.674+0000 7f9bca24f700 1 --2- 172.21.15.97:0/4279554040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc40cea00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.674+0000 7f9bca24f700 1 -- 172.21.15.97:0/4279554040 >> 172.21.15.97:0/4279554040 conn(0x7f9bc40b3780 msgr2=0x7f9bc40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:06.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.674+0000 7f9bca24f700 1 -- 172.21.15.97:0/4279554040 shutdown_connections 2023-11-24T06:10:06.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.674+0000 7f9bca24f700 1 -- 172.21.15.97:0/4279554040 wait complete. 2023-11-24T06:10:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.675+0000 7f9bca24f700 1 Processor -- start 2023-11-24T06:10:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.675+0000 7f9bca24f700 1 -- start start 2023-11-24T06:10:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bca24f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc4070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bca24f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc40cef40 0x7f9bc41e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bca24f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bc415bf50 0x7f9bc41e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bca24f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40c9a90 con 0x7f9bc40c6c30 2023-11-24T06:10:06.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bca24f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4070c40 con 0x7f9bc40cef40 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bca24f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc41e6160 con 0x7f9bc415bf50 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc4070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc4070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47260/0 (socket says 172.21.15.97:47260) 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc37fe700 1 -- 172.21.15.97:0/4152369172 learned_addr learned my addr 172.21.15.97:0/4152369172 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc2ffd700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc40cef40 0x7f9bc41e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc3fff700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bc415bf50 0x7f9bc41e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc37fe700 1 -- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bc415bf50 msgr2=0x7f9bc41e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc37fe700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bc415bf50 0x7f9bc41e5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc37fe700 1 -- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc40cef40 msgr2=0x7f9bc41e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc37fe700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc40cef40 0x7f9bc41e5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc37fe700 1 -- 172.21.15.97:0/4152369172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb40094a0 con 0x7f9bc40c6c30 2023-11-24T06:10:06.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc2ffd700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc40cef40 0x7f9bc41e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.676+0000 7f9bc3fff700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bc415bf50 0x7f9bc41e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.677+0000 7f9bc37fe700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc4070700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800c550 tx=0x7f9bb800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.677+0000 7f9babfff700 1 -- 172.21.15.97:0/4152369172 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8004c80 con 0x7f9bc40c6c30 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.677+0000 7f9babfff700 1 -- 172.21.15.97:0/4152369172 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8004e00 con 0x7f9bc40c6c30 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.677+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc41e6440 con 0x7f9bc40c6c30 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.677+0000 7f9babfff700 1 -- 172.21.15.97:0/4152369172 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8007310 con 0x7f9bc40c6c30 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.677+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc40cba70 con 0x7f9bc40c6c30 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.678+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc4068fb0 con 0x7f9bc40c6c30 2023-11-24T06:10:06.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.678+0000 7f9babfff700 1 -- 172.21.15.97:0/4152369172 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9bb8007470 con 0x7f9bc40c6c30 2023-11-24T06:10:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.679+0000 7f9babfff700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9bb006aed0 0x7f9bb006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.679+0000 7f9bc2ffd700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9bb006aed0 0x7f9bb006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.679+0000 7f9babfff700 1 -- 172.21.15.97:0/4152369172 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bb8089220 con 0x7f9bc40c6c30 2023-11-24T06:10:06.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.679+0000 7f9bc2ffd700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9bb006aed0 0x7f9bb006d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4000c00 tx=0x7f9bb401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.683+0000 7f9babfff700 1 -- 172.21.15.97:0/4152369172 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9bb8054360 con 0x7f9bc40c6c30 2023-11-24T06:10:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.904+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp7976", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bc40ce7f0 con 0x7f9bb006aed0 2023-11-24T06:10:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.908+0000 7f9babfff700 1 -- 172.21.15.97:0/4152369172 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9bc40ce7f0 con 0x7f9bb006aed0 2023-11-24T06:10:06.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9bb006aed0 msgr2=0x7f9bb006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9bb006aed0 0x7f9bb006d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4000c00 tx=0x7f9bb401c040 comp rx=0 tx=0).stop 2023-11-24T06:10:06.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 msgr2=0x7f9bc4070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:06.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc4070700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800c550 tx=0x7f9bb800c910 comp rx=0 tx=0).stop 2023-11-24T06:10:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 shutdown_connections 2023-11-24T06:10:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9bb006aed0 0x7f9bb006d390 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4000c00 tx=0x7f9bb401c040 comp rx=0 tx=0).stop 2023-11-24T06:10:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9bc415bf50 0x7f9bc41e5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bc40c6c30 0x7f9bc4070700 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 --2- 172.21.15.97:0/4152369172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc40cef40 0x7f9bc41e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 >> 172.21.15.97:0/4152369172 conn(0x7f9bc40b3780 msgr2=0x7f9bc40ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 shutdown_connections 2023-11-24T06:10:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:06.910+0000 7f9bca24f700 1 -- 172.21.15.97:0/4152369172 wait complete. 2023-11-24T06:10:06.925 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:10:06.925 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:10:06.970 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:10:07.014 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:10:07.014 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:07.014 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:07.031 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:07.032 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:07.088 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:07.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.246+0000 7f1d653ca700 1 Processor -- start 2023-11-24T06:10:07.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.246+0000 7f1d653ca700 1 -- start start 2023-11-24T06:10:07.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d653ca700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600d0870 0x7f1d600d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d653ca700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d600d1210 con 0x7f1d600b6ef0 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d653ca700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d600d20b0 con 0x7f1d6015e1e0 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d653ca700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600d3150 con 0x7f1d600d0870 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d5f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1d600b6ef0 0x7f1d600b7380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39724/0 (socket says 172.21.15.97:39724) 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/1902076757 learned_addr learned my addr 172.21.15.97:0/1902076757 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d5e7fc700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600d0870 0x7f1d600d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630266247 0 0) 0x7f1d600d1210 con 0x7f1d600b6ef0 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.247+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d480035b0 con 0x7f1d600b6ef0 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2431531099 0 0) 0x7f1d480035b0 con 0x7f1d600b6ef0 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600d0870 msgr2=0x7f1d600d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600d0870 0x7f1d600d0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 >> v1:172.21.15.183:6789/0 conn(0x7f1d6015e1e0 legacy=0x7f1d600d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d600d2390 con 0x7f1d600b6ef0 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f1d580019f0 con 0x7f1d600b6ef0 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 >> v1:172.21.15.97:6789/0 conn(0x7f1d600b6ef0 legacy=0x7f1d600b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d48003a30 0x7f1d48003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600d0870 0x7f1d48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48004d90 0x7f1d48005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600d3150 con 0x7f1d48003a30 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4800bb50 con 0x7f1d48004d90 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4800bd10 con 0x7f1d600d0870 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5effd700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d48003a30 0x7f1d48003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5e7fc700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600d0870 0x7f1d48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48004d90 0x7f1d48005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5effd700 1 -- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600d0870 msgr2=0x7f1d48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5effd700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600d0870 0x7f1d48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5effd700 1 -- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48004d90 msgr2=0x7f1d48005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5effd700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48004d90 0x7f1d48005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5effd700 1 -- 172.21.15.97:0/1902076757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d480035b0 con 0x7f1d48003a30 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48004d90 0x7f1d48005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5effd700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d48003a30 0x7f1d48003e50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1d54007fd0 tx=0x7f1d48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c379912cb671155c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d540052d0 con 0x7f1d48003a30 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d54005470 con 0x7f1d48003a30 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.248+0000 7f1d5dffb700 1 -- 172.21.15.97:0/1902076757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d540038c0 con 0x7f1d48003a30 2023-11-24T06:10:07.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.249+0000 7f1d653ca700 1 -- 172.21.15.97:0/1902076757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d48003a30 msgr2=0x7f1d48003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.249+0000 7f1d653ca700 1 --2- 172.21.15.97:0/1902076757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d48003a30 0x7f1d48003e50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1d54007fd0 tx=0x7f1d48005880 comp rx=0 tx=0).stop 2023-11-24T06:10:07.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.249+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/1902076757 reap_dead start 2023-11-24T06:10:07.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.249+0000 7f1d653ca700 1 -- 172.21.15.97:0/1902076757 shutdown_connections 2023-11-24T06:10:07.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.249+0000 7f1d653ca700 1 -- 172.21.15.97:0/1902076757 >> 172.21.15.97:0/1902076757 conn(0x7f1d600b5b50 msgr2=0x7f1d600b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.249+0000 7f1d653ca700 1 -- 172.21.15.97:0/1902076757 shutdown_connections 2023-11-24T06:10:07.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.249+0000 7f1d653ca700 1 -- 172.21.15.97:0/1902076757 wait complete. 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.250+0000 7f1d653ca700 1 Processor -- start 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.250+0000 7f1d653ca700 1 -- start start 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d653ca700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48003a30 0x7f1d600cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d653ca700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600cb3b0 0x7f1d6014ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d653ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600b2750 0x7f1d600ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d653ca700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6014f3a0 con 0x7f1d600b2750 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d653ca700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d60152780 con 0x7f1d48003a30 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d653ca700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d600cb870 con 0x7f1d600cb3b0 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d5f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600b2750 0x7f1d600ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d5f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600b2750 0x7f1d600ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47282/0 (socket says 172.21.15.97:47282) 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/3381183363 learned_addr learned my addr 172.21.15.97:0/3381183363 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:07.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d5effd700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48003a30 0x7f1d600cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.251+0000 7f1d5e7fc700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600cb3b0 0x7f1d6014ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600cb3b0 msgr2=0x7f1d6014ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600cb3b0 0x7f1d6014ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48003a30 msgr2=0x7f1d600cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48003a30 0x7f1d600cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d5f7fe700 1 -- 172.21.15.97:0/3381183363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d600cbb50 con 0x7f1d600b2750 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d5effd700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48003a30 0x7f1d600cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d5e7fc700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600cb3b0 0x7f1d6014ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d5f7fe700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600b2750 0x7f1d600ad090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d60072830 tx=0x7f1d58007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d477fe700 1 -- 172.21.15.97:0/3381183363 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800ee20 con 0x7f1d600b2750 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d477fe700 1 -- 172.21.15.97:0/3381183363 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d58013420 con 0x7f1d600b2750 2023-11-24T06:10:07.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d477fe700 1 -- 172.21.15.97:0/3381183363 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580137d0 con 0x7f1d600b2750 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d600cbe30 con 0x7f1d600b2750 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.252+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d600cc3f0 con 0x7f1d600b2750 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.253+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d600cce50 con 0x7f1d600b2750 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.253+0000 7f1d477fe700 1 -- 172.21.15.97:0/3381183363 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1d58013930 con 0x7f1d600b2750 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.254+0000 7f1d477fe700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d4c06af60 0x7f1d4c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.254+0000 7f1d477fe700 1 -- 172.21.15.97:0/3381183363 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d58089d00 con 0x7f1d600b2750 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.254+0000 7f1d5effd700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d4c06af60 0x7f1d4c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.254+0000 7f1d5effd700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d4c06af60 0x7f1d4c06d420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1d54007fd0 tx=0x7f1d54009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.258+0000 7f1d477fe700 1 -- 172.21.15.97:0/3381183363 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d5808d030 con 0x7f1d600b2750 2023-11-24T06:10:07.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.450+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d600cc610 con 0x7f1d600b2750 2023-11-24T06:10:07.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.451+0000 7f1d477fe700 1 -- 172.21.15.97:0/3381183363 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f1d58058900 con 0x7f1d600b2750 2023-11-24T06:10:07.454 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:07.454 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:57.042131+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:09:53.319367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:09:54.404860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:07.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d4c06af60 msgr2=0x7f1d4c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d4c06af60 0x7f1d4c06d420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1d54007fd0 tx=0x7f1d54009030 comp rx=0 tx=0).stop 2023-11-24T06:10:07.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600b2750 msgr2=0x7f1d600ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600b2750 0x7f1d600ad090 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d60072830 tx=0x7f1d58007c90 comp rx=0 tx=0).stop 2023-11-24T06:10:07.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 shutdown_connections 2023-11-24T06:10:07.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d600b2750 0x7f1d600ad090 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d48003a30 0x7f1d600cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d4c06af60 0x7f1d4c06d420 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 --2- 172.21.15.97:0/3381183363 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d600cb3b0 0x7f1d6014ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 >> 172.21.15.97:0/3381183363 conn(0x7f1d600b5b50 msgr2=0x7f1d6015c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 shutdown_connections 2023-11-24T06:10:07.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.453+0000 7f1d653ca700 1 -- 172.21.15.97:0/3381183363 wait complete. 2023-11-24T06:10:07.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:09:53.319367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:09:54.404860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:07.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:07.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.588+0000 7f8dd8ba1700 1 Processor -- start 2023-11-24T06:10:07.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.588+0000 7f8dd8ba1700 1 -- start start 2023-11-24T06:10:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.589+0000 7f8dd8ba1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dd40c6c30 0x7f8dd40cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.589+0000 7f8dd8ba1700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40d0140 con 0x7f8dd40cef40 2023-11-24T06:10:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.589+0000 7f8dd8ba1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40cf3d0 con 0x7f8dd415bf50 2023-11-24T06:10:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.589+0000 7f8dd8ba1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd415c330 con 0x7f8dd40c6c30 2023-11-24T06:10:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.589+0000 7f8dd2d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8dd40cef40 0x7f8dd40cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39744/0 (socket says 172.21.15.97:39744) 2023-11-24T06:10:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.589+0000 7f8dd2d9d700 1 -- 172.21.15.97:0/1051071333 learned_addr learned my addr 172.21.15.97:0/1051071333 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd1d9b700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dd40c6c30 0x7f8dd40cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386454098 0 0) 0x7f8dd40d0140 con 0x7f8dd40cef40 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dbc0035b0 con 0x7f8dd40cef40 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 519067884 0 0) 0x7f8dbc0035b0 con 0x7f8dd40cef40 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dd40c6c30 msgr2=0x7f8dd40cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dd40c6c30 0x7f8dd40cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 >> v1:172.21.15.183:6789/0 conn(0x7f8dd415bf50 legacy=0x7f8dd40ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40d03e0 con 0x7f8dd40cef40 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f8dc4001670 con 0x7f8dd40cef40 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 >> v1:172.21.15.97:6789/0 conn(0x7f8dd40cef40 legacy=0x7f8dd40cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dd40c6c30 0x7f8dbc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc003fb0 0x7f8dbc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dbc004d90 0x7f8dbc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd415c330 con 0x7f8dbc003fb0 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc00baf0 con 0x7f8dbc004d90 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc00bcb0 con 0x7f8dd40c6c30 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd1d9b700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc003fb0 0x7f8dbc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd259c700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dd40c6c30 0x7f8dbc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd2d9d700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dbc004d90 0x7f8dbc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd1d9b700 1 -- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dd40c6c30 msgr2=0x7f8dbc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd1d9b700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dd40c6c30 0x7f8dbc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd1d9b700 1 -- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dbc004d90 msgr2=0x7f8dbc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd1d9b700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dbc004d90 0x7f8dbc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.590+0000 7f8dd1d9b700 1 -- 172.21.15.97:0/1051071333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0035b0 con 0x7f8dbc003fb0 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd1d9b700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc003fb0 0x7f8dbc004850 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009fc0 tx=0x7f8dbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411ff192abdb153f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800ec30 con 0x7f8dbc003fb0 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc800f040 con 0x7f8dbc003fb0 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd159a700 1 -- 172.21.15.97:0/1051071333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8014480 con 0x7f8dbc003fb0 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/1051071333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc003fb0 msgr2=0x7f8dbc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd8ba1700 1 --2- 172.21.15.97:0/1051071333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dbc003fb0 0x7f8dbc004850 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009fc0 tx=0x7f8dbc005880 comp rx=0 tx=0).stop 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd2d9d700 1 -- 172.21.15.97:0/1051071333 reap_dead start 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/1051071333 shutdown_connections 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/1051071333 >> 172.21.15.97:0/1051071333 conn(0x7f8dd40b3780 msgr2=0x7f8dd40b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/1051071333 shutdown_connections 2023-11-24T06:10:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.591+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/1051071333 wait complete. 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd8ba1700 1 Processor -- start 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd8ba1700 1 -- start start 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd8ba1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dbc003fb0 0x7f8dd40b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd8ba1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd40b4870 0x7f8dd40d2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd8ba1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dd40b4cf0 0x7f8dd414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd8ba1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40b5170 con 0x7f8dd40b4870 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd8ba1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40b5450 con 0x7f8dd40b4cf0 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd8ba1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40b5730 con 0x7f8dbc003fb0 2023-11-24T06:10:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.593+0000 7f8dd1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd40b4870 0x7f8dd40d2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd40b4870 0x7f8dd40d2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47302/0 (socket says 172.21.15.97:47302) 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd1d9b700 1 -- 172.21.15.97:0/3576365242 learned_addr learned my addr 172.21.15.97:0/3576365242 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd2d9d700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dd40b4cf0 0x7f8dd414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd259c700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dbc003fb0 0x7f8dd40b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd1d9b700 1 -- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dbc003fb0 msgr2=0x7f8dd40b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd1d9b700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dbc003fb0 0x7f8dd40b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd1d9b700 1 -- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dd40b4cf0 msgr2=0x7f8dd414dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd1d9b700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dd40b4cf0 0x7f8dd414dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd1d9b700 1 -- 172.21.15.97:0/3576365242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40b5a10 con 0x7f8dd40b4870 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd259c700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dbc003fb0 0x7f8dd40b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd2d9d700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dd40b4cf0 0x7f8dd414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:07.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd1d9b700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd40b4870 0x7f8dd40d2330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800e700 tx=0x7f8dc8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dc2ffd700 1 -- 172.21.15.97:0/3576365242 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8004450 con 0x7f8dd40b4870 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dc2ffd700 1 -- 172.21.15.97:0/3576365242 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc8004610 con 0x7f8dd40b4870 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd40b5cf0 con 0x7f8dd40b4870 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dc2ffd700 1 -- 172.21.15.97:0/3576365242 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8014480 con 0x7f8dd40b4870 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.594+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd40d2a50 con 0x7f8dd40b4870 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.595+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd4068fb0 con 0x7f8dd40b4870 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.596+0000 7f8dc2ffd700 1 -- 172.21.15.97:0/3576365242 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8dc8024070 con 0x7f8dd40b4870 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.596+0000 7f8dc2ffd700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8db406af60 0x7f8db406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.596+0000 7f8dc2ffd700 1 -- 172.21.15.97:0/3576365242 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dc808b250 con 0x7f8dd40b4870 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.596+0000 7f8dd259c700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8db406af60 0x7f8db406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.596+0000 7f8dd259c700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8db406af60 0x7f8db406d420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc000c00 tx=0x7f8dcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.600+0000 7f8dc2ffd700 1 -- 172.21.15.97:0/3576365242 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8dc80563a0 con 0x7f8dd40b4870 2023-11-24T06:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.795+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8dd40d2a50 con 0x7f8dd40b4870 2023-11-24T06:10:07.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.795+0000 7f8dc2ffd700 1 -- 172.21.15.97:0/3576365242 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dd40d2a50 con 0x7f8dd40b4870 2023-11-24T06:10:07.798 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8db406af60 msgr2=0x7f8db406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8db406af60 0x7f8db406d420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc000c00 tx=0x7f8dcc009040 comp rx=0 tx=0).stop 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd40b4870 msgr2=0x7f8dd40d2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd40b4870 0x7f8dd40d2330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800e700 tx=0x7f8dc8009750 comp rx=0 tx=0).stop 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 shutdown_connections 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8dd40b4cf0 0x7f8dd414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8db406af60 0x7f8db406d420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8dbc003fb0 0x7f8dd40b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 --2- 172.21.15.97:0/3576365242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8dd40b4870 0x7f8dd40d2330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 >> 172.21.15.97:0/3576365242 conn(0x7f8dd40b3780 msgr2=0x7f8dd40cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 shutdown_connections 2023-11-24T06:10:07.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.797+0000 7f8dd8ba1700 1 -- 172.21.15.97:0/3576365242 wait complete. 2023-11-24T06:10:07.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:07.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.932+0000 7fcfefebd700 1 Processor -- start 2023-11-24T06:10:07.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.932+0000 7fcfefebd700 1 -- start start 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.933+0000 7fcfefebd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcfe81587f0 0x7fcfe806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.933+0000 7fcfefebd700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe806b530 con 0x7fcfe80b4370 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.933+0000 7fcfefebd700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe80ae5e0 con 0x7fcfe815b370 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.933+0000 7fcfefebd700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe815be30 con 0x7fcfe81587f0 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.933+0000 7fcfedc59700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcfe81587f0 0x7fcfe806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.933+0000 7fcfee45a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcfe80b4370 0x7fcfe80af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39764/0 (socket says 172.21.15.97:39764) 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.933+0000 7fcfee45a700 1 -- 172.21.15.97:0/940709519 learned_addr learned my addr 172.21.15.97:0/940709519 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.934+0000 7fcfedc59700 1 -- 172.21.15.97:0/940709519 >> v1:172.21.15.183:6789/0 conn(0x7fcfe815b370 legacy=0x7fcfe80b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.934+0000 7fcfedc59700 1 -- 172.21.15.97:0/940709519 >> v1:172.21.15.97:6789/0 conn(0x7fcfe80b4370 legacy=0x7fcfe80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.934+0000 7fcfecc57700 1 -- 172.21.15.97:0/940709519 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697754979 0 0) 0x7fcfe806b530 con 0x7fcfe80b4370 2023-11-24T06:10:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.934+0000 7fcfedc59700 1 -- 172.21.15.97:0/940709519 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe815c110 con 0x7fcfe81587f0 2023-11-24T06:10:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.934+0000 7fcfedc59700 1 --2- 172.21.15.97:0/940709519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe81587f0 0x7fcfe806aff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009840 tx=0x7fcfd8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70713cadf2ca4fe1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.934+0000 7fcfecc57700 1 -- 172.21.15.97:0/940709519 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8004230 con 0x7fcfe81587f0 2023-11-24T06:10:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.934+0000 7fcfecc57700 1 -- 172.21.15.97:0/940709519 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8004390 con 0x7fcfe81587f0 2023-11-24T06:10:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.934+0000 7fcfecc57700 1 -- 172.21.15.97:0/940709519 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80036a0 con 0x7fcfe81587f0 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.935+0000 7fcfefebd700 1 -- 172.21.15.97:0/940709519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe81587f0 msgr2=0x7fcfe806aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.935+0000 7fcfefebd700 1 --2- 172.21.15.97:0/940709519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe81587f0 0x7fcfe806aff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009840 tx=0x7fcfd8009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.935+0000 7fcfefebd700 1 -- 172.21.15.97:0/940709519 shutdown_connections 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.935+0000 7fcfefebd700 1 --2- 172.21.15.97:0/940709519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe81587f0 0x7fcfe806aff0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.935+0000 7fcfefebd700 1 -- 172.21.15.97:0/940709519 >> 172.21.15.97:0/940709519 conn(0x7fcfe80d0bf0 msgr2=0x7fcfe80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.935+0000 7fcfefebd700 1 -- 172.21.15.97:0/940709519 shutdown_connections 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.935+0000 7fcfefebd700 1 -- 172.21.15.97:0/940709519 wait complete. 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.936+0000 7fcfefebd700 1 Processor -- start 2023-11-24T06:10:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.936+0000 7fcfefebd700 1 -- start start 2023-11-24T06:10:07.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfefebd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe80b4370 0x7fcfe809cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfefebd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe81587f0 0x7fcfe815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfefebd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe815b370 0x7fcfe80c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfefebd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe8149e00 con 0x7fcfe80b4370 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfefebd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe809d150 con 0x7fcfe815b370 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfefebd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe814d150 con 0x7fcfe81587f0 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe80b4370 0x7fcfe809cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe80b4370 0x7fcfe809cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47314/0 (socket says 172.21.15.97:47314) 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 -- 172.21.15.97:0/3617906704 learned_addr learned my addr 172.21.15.97:0/3617906704 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 -- 172.21.15.97:0/3617906704 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe81587f0 msgr2=0x7fcfe815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe81587f0 0x7fcfe815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 -- 172.21.15.97:0/3617906704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe815b370 msgr2=0x7fcfe80c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe815b370 0x7fcfe80c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 -- 172.21.15.97:0/3617906704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80094a0 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfedc59700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe80b4370 0x7fcfe809cc10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80090b0 tx=0x7fcfd8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfde7fc700 1 -- 172.21.15.97:0/3617906704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8011830 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfde7fc700 1 -- 172.21.15.97:0/3617906704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd801e070 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfde7fc700 1 -- 172.21.15.97:0/3617906704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8011bb0 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe81491f0 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.937+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe81497b0 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.938+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe8068fb0 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.939+0000 7fcfde7fc700 1 -- 172.21.15.97:0/3617906704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcfd8011d10 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.939+0000 7fcfde7fc700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcfcc06af60 0x7fcfcc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.939+0000 7fcfde7fc700 1 -- 172.21.15.97:0/3617906704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfd808a990 con 0x7fcfe80b4370 2023-11-24T06:10:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.939+0000 7fcfed458700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcfcc06af60 0x7fcfcc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:07.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.939+0000 7fcfed458700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcfcc06af60 0x7fcfcc06d420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80b4030 tx=0x7fcfe4006c50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:07.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:07.943+0000 7fcfde7fc700 1 -- 172.21.15.97:0/3617906704 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfd8055ae0 con 0x7fcfe80b4370 2023-11-24T06:10:08.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.132+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcfe806cb30 con 0x7fcfe80b4370 2023-11-24T06:10:08.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.133+0000 7fcfde7fc700 1 -- 172.21.15.97:0/3617906704 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcfd80200a0 con 0x7fcfe80b4370 2023-11-24T06:10:08.137 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:10:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcfcc06af60 msgr2=0x7fcfcc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcfcc06af60 0x7fcfcc06d420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80b4030 tx=0x7fcfe4006c50 comp rx=0 tx=0).stop 2023-11-24T06:10:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe80b4370 msgr2=0x7fcfe809cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe80b4370 0x7fcfe809cc10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80090b0 tx=0x7fcfd8004750 comp rx=0 tx=0).stop 2023-11-24T06:10:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 shutdown_connections 2023-11-24T06:10:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcfe815b370 0x7fcfe80c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfe80b4370 0x7fcfe809cc10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcfcc06af60 0x7fcfcc06d420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 --2- 172.21.15.97:0/3617906704 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcfe81587f0 0x7fcfe815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 >> 172.21.15.97:0/3617906704 conn(0x7fcfe80d0bf0 msgr2=0x7fcfe815c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 shutdown_connections 2023-11-24T06:10:08.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.135+0000 7fcfefebd700 1 -- 172.21.15.97:0/3617906704 wait complete. 2023-11-24T06:10:08.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:08.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.270+0000 7fdda27b0700 1 Processor -- start 2023-11-24T06:10:08.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.270+0000 7fdda27b0700 1 -- start start 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.271+0000 7fdda27b0700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdd9c15e290 0x7fdd9c0ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.271+0000 7fdda27b0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.271+0000 7fdda27b0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9c06e400 con 0x7fdd9c0aedc0 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.271+0000 7fdda27b0700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9c06e630 con 0x7fdd9c15e290 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.271+0000 7fdda27b0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c06e910 con 0x7fdd9c0ad870 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdda0d4d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdd9b7fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdd9c15e290 0x7fdd9c0ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdda0d4d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:34362/0 (socket says 172.21.15.97:34362) 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdda0d4d700 1 -- 172.21.15.97:0/3274959925 learned_addr learned my addr 172.21.15.97:0/3274959925 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdda0d4d700 1 -- 172.21.15.97:0/3274959925 >> v2:172.21.15.183:3300/0 conn(0x7fdd9c15e290 msgr2=0x7fdd9c0ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdda0d4d700 1 --2- 172.21.15.97:0/3274959925 >> v2:172.21.15.183:3300/0 conn(0x7fdd9c15e290 0x7fdd9c0ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdda0d4d700 1 -- 172.21.15.97:0/3274959925 >> v1:172.21.15.183:6789/0 conn(0x7fdd9c0aedc0 legacy=0x7fdd9c0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdda0d4d700 1 -- 172.21.15.97:0/3274959925 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c06ebf0 con 0x7fdd9c0ad870 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdd9b7fe700 1 --2- 172.21.15.97:0/3274959925 >> v2:172.21.15.183:3300/0 conn(0x7fdd9c15e290 0x7fdd9c0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdda0d4d700 1 --2- 172.21.15.97:0/3274959925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c06f6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009e50 tx=0x7fdd9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ff23024cad2b05 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdd9affd700 1 -- 172.21.15.97:0/3274959925 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400e070 con 0x7fdd9c0ad870 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdd9affd700 1 -- 172.21.15.97:0/3274959925 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9400b910 con 0x7fdd9c0ad870 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.272+0000 7fdd9affd700 1 -- 172.21.15.97:0/3274959925 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400bcc0 con 0x7fdd9c0ad870 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.273+0000 7fdda27b0700 1 -- 172.21.15.97:0/3274959925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 msgr2=0x7fdd9c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.273+0000 7fdda27b0700 1 --2- 172.21.15.97:0/3274959925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c06f6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009e50 tx=0x7fdd9400b330 comp rx=0 tx=0).stop 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.273+0000 7fdda27b0700 1 -- 172.21.15.97:0/3274959925 shutdown_connections 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.273+0000 7fdda27b0700 1 --2- 172.21.15.97:0/3274959925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c06f6d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.273+0000 7fdda27b0700 1 --2- 172.21.15.97:0/3274959925 >> v2:172.21.15.183:3300/0 conn(0x7fdd9c15e290 0x7fdd9c0ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.273+0000 7fdda27b0700 1 -- 172.21.15.97:0/3274959925 >> 172.21.15.97:0/3274959925 conn(0x7fdd9c0b2050 msgr2=0x7fdd9c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.273+0000 7fdda27b0700 1 -- 172.21.15.97:0/3274959925 shutdown_connections 2023-11-24T06:10:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.273+0000 7fdda27b0700 1 -- 172.21.15.97:0/3274959925 wait complete. 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.274+0000 7fdda27b0700 1 Processor -- start 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.274+0000 7fdda27b0700 1 -- start start 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda27b0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda27b0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd9c0aedc0 0x7fdd9c1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda27b0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9c15e290 0x7fdd9c0cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda27b0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c1f4c10 con 0x7fdd9c15e290 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda27b0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c0cce60 con 0x7fdd9c0aedc0 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda27b0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c0cd140 con 0x7fdd9c0ad870 2023-11-24T06:10:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda0d4d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9c15e290 0x7fdd9c0cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda0d4d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9c15e290 0x7fdd9c0cdf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47332/0 (socket says 172.21.15.97:47332) 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda0d4d700 1 -- 172.21.15.97:0/2182293499 learned_addr learned my addr 172.21.15.97:0/2182293499 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdd9bfff700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdd9b7fe700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd9c0aedc0 0x7fdd9c1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda0d4d700 1 -- 172.21.15.97:0/2182293499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 msgr2=0x7fdd9c150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda0d4d700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda0d4d700 1 -- 172.21.15.97:0/2182293499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd9c0aedc0 msgr2=0x7fdd9c1f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda0d4d700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd9c0aedc0 0x7fdd9c1f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.275+0000 7fdda0d4d700 1 -- 172.21.15.97:0/2182293499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c0094a0 con 0x7fdd9c15e290 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.276+0000 7fdda0d4d700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9c15e290 0x7fdd9c0cdf00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c070e40 tx=0x7fdd94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.276+0000 7fdd98ff9700 1 -- 172.21.15.97:0/2182293499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94004800 con 0x7fdd9c15e290 2023-11-24T06:10:08.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.276+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd94009b30 con 0x7fdd9c15e290 2023-11-24T06:10:08.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.276+0000 7fdd98ff9700 1 -- 172.21.15.97:0/2182293499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd94004960 con 0x7fdd9c15e290 2023-11-24T06:10:08.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.276+0000 7fdd98ff9700 1 -- 172.21.15.97:0/2182293499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9401b670 con 0x7fdd9c15e290 2023-11-24T06:10:08.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.276+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c0cd700 con 0x7fdd9c15e290 2023-11-24T06:10:08.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.277+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9c068d90 con 0x7fdd9c15e290 2023-11-24T06:10:08.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.277+0000 7fdd98ff9700 1 -- 172.21.15.97:0/2182293499 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fdd9401b7d0 con 0x7fdd9c15e290 2023-11-24T06:10:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.278+0000 7fdd98ff9700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdd8806af60 0x7fdd8806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.278+0000 7fdd9bfff700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdd8806af60 0x7fdd8806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.278+0000 7fdd98ff9700 1 -- 172.21.15.97:0/2182293499 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd9408ae90 con 0x7fdd9c15e290 2023-11-24T06:10:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.278+0000 7fdd9bfff700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdd8806af60 0x7fdd8806d420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c160370 tx=0x7fdd90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.282+0000 7fdd98ff9700 1 -- 172.21.15.97:0/2182293499 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd94055e50 con 0x7fdd9c15e290 2023-11-24T06:10:08.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.470+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd9c068fb0 con 0x7fdd9c15e290 2023-11-24T06:10:08.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.470+0000 7fdd98ff9700 1 -- 172.21.15.97:0/2182293499 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd94059470 con 0x7fdd9c15e290 2023-11-24T06:10:08.474 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:10:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdd8806af60 msgr2=0x7fdd8806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdd8806af60 0x7fdd8806d420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c160370 tx=0x7fdd90009040 comp rx=0 tx=0).stop 2023-11-24T06:10:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9c15e290 msgr2=0x7fdd9c0cdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9c15e290 0x7fdd9c0cdf00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c070e40 tx=0x7fdd94004ad0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 shutdown_connections 2023-11-24T06:10:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd9c15e290 0x7fdd9c0cdf00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdd8806af60 0x7fdd8806d420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdd9c0ad870 0x7fdd9c150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 --2- 172.21.15.97:0/2182293499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd9c0aedc0 0x7fdd9c1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 >> 172.21.15.97:0/2182293499 conn(0x7fdd9c0b2050 msgr2=0x7fdd9c0adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.472+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 shutdown_connections 2023-11-24T06:10:08.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.473+0000 7fdda27b0700 1 -- 172.21.15.97:0/2182293499 wait complete. 2023-11-24T06:10:08.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:10:08.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.606+0000 7fcd130cd700 1 Processor -- start 2023-11-24T06:10:08.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.606+0000 7fcd130cd700 1 -- start start 2023-11-24T06:10:08.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd130cd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcd0c06c710 0x7fcd0c165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd130cd700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c0af950 con 0x7fcd0c0adf00 2023-11-24T06:10:08.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd130cd700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c0afc30 con 0x7fcd0c1446a0 2023-11-24T06:10:08.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd130cd700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0c144a80 con 0x7fcd0c06c710 2023-11-24T06:10:08.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd0bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcd0c0adf00 0x7fcd0c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39788/0 (socket says 172.21.15.97:39788) 2023-11-24T06:10:08.613 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd0bfff700 1 -- 172.21.15.97:0/1123330266 learned_addr learned my addr 172.21.15.97:0/1123330266 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd1166a700 1 --2- 172.21.15.97:0/1123330266 >> v2:172.21.15.183:3300/0 conn(0x7fcd0c06c710 0x7fcd0c165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389304059 0 0) 0x7fcd0c0af950 con 0x7fcd0c0adf00 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccfc0035b0 con 0x7fcd0c0adf00 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3553484126 0 0) 0x7fccfc0035b0 con 0x7fcd0c0adf00 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 >> v2:172.21.15.183:3300/0 conn(0x7fcd0c06c710 msgr2=0x7fcd0c165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd0b7fe700 1 --2- 172.21.15.97:0/1123330266 >> v2:172.21.15.183:3300/0 conn(0x7fcd0c06c710 0x7fcd0c165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.607+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 >> v1:172.21.15.183:6789/0 conn(0x7fcd0c1446a0 legacy=0x7fcd0c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0aff10 con 0x7fcd0c0adf00 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fccf8001670 con 0x7fcd0c0adf00 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 >> v1:172.21.15.97:6789/0 conn(0x7fcd0c0adf00 legacy=0x7fcd0c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccfc003a00 0x7fccfc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fccfc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccfc005210 0x7fccfc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c144a80 con 0x7fccfc004850 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc00c1b0 con 0x7fccfc005210 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc00c370 con 0x7fccfc003a00 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0bfff700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fccfc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd1166a700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccfc005210 0x7fccfc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd10e69700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccfc003a00 0x7fccfc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0bfff700 1 -- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccfc003a00 msgr2=0x7fccfc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0bfff700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fccfc003a00 0x7fccfc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0bfff700 1 -- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccfc005210 msgr2=0x7fccfc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:08.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0bfff700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccfc005210 0x7fccfc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0bfff700 1 -- 172.21.15.97:0/1123330266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc0035b0 con 0x7fccfc004850 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd1166a700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccfc005210 0x7fccfc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd1166a700 1 -- 172.21.15.97:0/1123330266 reap_dead start 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0bfff700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fccfc004cd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c06f1d0 tx=0x7fccfc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f480d996a0497 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf800cdd0 con 0x7fccfc004850 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8005960 con 0x7fccfc004850 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.608+0000 7fcd0b7fe700 1 -- 172.21.15.97:0/1123330266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8005cd0 con 0x7fccfc004850 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.609+0000 7fcd130cd700 1 -- 172.21.15.97:0/1123330266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 msgr2=0x7fccfc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.609+0000 7fcd130cd700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fccfc004cd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c06f1d0 tx=0x7fccfc005880 comp rx=0 tx=0).stop 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.609+0000 7fcd130cd700 1 -- 172.21.15.97:0/1123330266 shutdown_connections 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.609+0000 7fcd130cd700 1 --2- 172.21.15.97:0/1123330266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fccfc004cd0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.609+0000 7fcd130cd700 1 -- 172.21.15.97:0/1123330266 >> 172.21.15.97:0/1123330266 conn(0x7fcd0c071b90 msgr2=0x7fcd0c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.609+0000 7fcd130cd700 1 -- 172.21.15.97:0/1123330266 shutdown_connections 2023-11-24T06:10:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.609+0000 7fcd130cd700 1 -- 172.21.15.97:0/1123330266 wait complete. 2023-11-24T06:10:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.610+0000 7fcd130cd700 1 Processor -- start 2023-11-24T06:10:08.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.610+0000 7fcd130cd700 1 -- start start 2023-11-24T06:10:08.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd130cd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fcd0c15a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd130cd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0c09de40 0x7fcd0c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd130cd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd0c1f46a0 0x7fcd0c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd130cd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0b5260 con 0x7fccfc004850 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd130cd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0b5500 con 0x7fcd0c09de40 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd130cd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0b57e0 con 0x7fcd0c1f46a0 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fcd0c15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fcd0c15a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47348/0 (socket says 172.21.15.97:47348) 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 -- 172.21.15.97:0/1412738424 learned_addr learned my addr 172.21.15.97:0/1412738424 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd1166a700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd0c1f46a0 0x7fcd0c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd0bfff700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0c09de40 0x7fcd0c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 -- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd0c1f46a0 msgr2=0x7fcd0c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd0c1f46a0 0x7fcd0c0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 -- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0c09de40 msgr2=0x7fcd0c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0c09de40 0x7fcd0c0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 -- 172.21.15.97:0/1412738424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c1f3ea0 con 0x7fccfc004850 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd1166a700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd0c1f46a0 0x7fcd0c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd0bfff700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0c09de40 0x7fcd0c0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.611+0000 7fcd10e69700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fcd0c15a790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000e7b0 tx=0x7fcd0000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.612+0000 7fcd097fa700 1 -- 172.21.15.97:0/1412738424 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000052e0 con 0x7fccfc004850 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.612+0000 7fcd097fa700 1 -- 172.21.15.97:0/1412738424 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd00005440 con 0x7fccfc004850 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.612+0000 7fcd097fa700 1 -- 172.21.15.97:0/1412738424 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000038a0 con 0x7fccfc004850 2023-11-24T06:10:08.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.612+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c1f4180 con 0x7fccfc004850 2023-11-24T06:10:08.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.612+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c1f4b30 con 0x7fccfc004850 2023-11-24T06:10:08.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.613+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0c0753f0 con 0x7fccfc004850 2023-11-24T06:10:08.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.613+0000 7fcd097fa700 1 -- 172.21.15.97:0/1412738424 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcd00003a00 con 0x7fccfc004850 2023-11-24T06:10:08.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.613+0000 7fcd097fa700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fccf006af60 0x7fccf006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:08.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.613+0000 7fcd0bfff700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fccf006af60 0x7fccf006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:08.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.614+0000 7fcd097fa700 1 -- 172.21.15.97:0/1412738424 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd00088cc0 con 0x7fccfc004850 2023-11-24T06:10:08.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.614+0000 7fcd0bfff700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fccf006af60 0x7fccf006d420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c06f1d0 tx=0x7fccf8007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:08.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.618+0000 7fcd097fa700 1 -- 172.21.15.97:0/1412738424 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd00053ec0 con 0x7fccfc004850 2023-11-24T06:10:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.823+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd0c1f4d50 con 0x7fccfc004850 2023-11-24T06:10:08.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.824+0000 7fcd097fa700 1 -- 172.21.15.97:0/1412738424 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd00019020 con 0x7fccfc004850 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fccf006af60 msgr2=0x7fccf006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fccf006af60 0x7fccf006d420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c06f1d0 tx=0x7fccf8007000 comp rx=0 tx=0).stop 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 msgr2=0x7fcd0c15a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fcd0c15a790 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000e7b0 tx=0x7fcd0000eb70 comp rx=0 tx=0).stop 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 shutdown_connections 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fccf006af60 0x7fccf006d420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd0c1f46a0 0x7fcd0c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccfc004850 0x7fcd0c15a790 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 --2- 172.21.15.97:0/1412738424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd0c09de40 0x7fcd0c0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 >> 172.21.15.97:0/1412738424 conn(0x7fcd0c071b90 msgr2=0x7fcd0c15b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 shutdown_connections 2023-11-24T06:10:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:08.826+0000 7fcd130cd700 1 -- 172.21.15.97:0/1412738424 wait complete. 2023-11-24T06:10:08.830 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:10:08.840 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:10:08.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:08.841 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:08.885 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:10:08.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:08.885 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:10:08.902 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:08.902 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:08.923 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:08.923 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:08.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.100+0000 7fe7a05b6700 1 Processor -- start 2023-11-24T06:10:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.100+0000 7fe7a05b6700 1 -- start start 2023-11-24T06:10:09.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe7a05b6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7980b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe7a05b6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe7980af4d0 0x7fe7980afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe7a05b6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7980ae430 con 0x7fe79815e080 2023-11-24T06:10:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe7a05b6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7980af0b0 con 0x7fe7980af4d0 2023-11-24T06:10:09.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe7a05b6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980b05a0 con 0x7fe79806c710 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe79db51700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7980b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe79eb53700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe7980af4d0 0x7fe7980afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe79e352700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fe79815e080 0x7fe79806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:49128/0 (socket says 172.21.15.97:49128) 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe79db51700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7980b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:34392/0 (socket says 172.21.15.97:34392) 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.101+0000 7fe79e352700 1 -- 172.21.15.97:0/1531840229 learned_addr learned my addr 172.21.15.97:0/1531840229 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79db51700 1 -- 172.21.15.97:0/1531840229 >> v2:172.21.15.183:3300/0 conn(0x7fe7980af4d0 msgr2=0x7fe7980afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79db51700 1 --2- 172.21.15.97:0/1531840229 >> v2:172.21.15.183:3300/0 conn(0x7fe7980af4d0 0x7fe7980afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79db51700 1 -- 172.21.15.97:0/1531840229 >> v1:172.21.15.183:6789/0 conn(0x7fe79815e080 legacy=0x7fe79806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79db51700 1 -- 172.21.15.97:0/1531840229 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7980ae710 con 0x7fe79806c710 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79eb53700 1 --2- 172.21.15.97:0/1531840229 >> v2:172.21.15.183:3300/0 conn(0x7fe7980af4d0 0x7fe7980afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:09.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79db51700 1 --2- 172.21.15.97:0/1531840229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7980b5050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe788009840 tx=0x7fe788009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb491380881e3889 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79d350700 1 -- 172.21.15.97:0/1531840229 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788011830 con 0x7fe79806c710 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79d350700 1 -- 172.21.15.97:0/1531840229 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe788011990 con 0x7fe79806c710 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.102+0000 7fe79d350700 1 -- 172.21.15.97:0/1531840229 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788011d20 con 0x7fe79806c710 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.103+0000 7fe7a05b6700 1 -- 172.21.15.97:0/1531840229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 msgr2=0x7fe7980b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.103+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/1531840229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7980b5050 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe788009840 tx=0x7fe788009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.103+0000 7fe7a05b6700 1 -- 172.21.15.97:0/1531840229 shutdown_connections 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.103+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/1531840229 >> v2:172.21.15.183:3300/0 conn(0x7fe7980af4d0 0x7fe7980afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.103+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/1531840229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7980b5050 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.103+0000 7fe7a05b6700 1 -- 172.21.15.97:0/1531840229 >> 172.21.15.97:0/1531840229 conn(0x7fe798071b90 msgr2=0x7fe798071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.103+0000 7fe7a05b6700 1 -- 172.21.15.97:0/1531840229 shutdown_connections 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.103+0000 7fe7a05b6700 1 -- 172.21.15.97:0/1531840229 wait complete. 2023-11-24T06:10:09.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.104+0000 7fe7a05b6700 1 Processor -- start 2023-11-24T06:10:09.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.104+0000 7fe7a05b6700 1 -- start start 2023-11-24T06:10:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe7a05b6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7981603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe7a05b6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7980af4d0 0x7fe798146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe7a05b6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79815e080 0x7fe7981465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe7a05b6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798154180 con 0x7fe79815e080 2023-11-24T06:10:09.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe7a05b6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980d0480 con 0x7fe7980af4d0 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe7a05b6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe798146b10 con 0x7fe79806c710 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79815e080 0x7fe7981465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79815e080 0x7fe7981465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47364/0 (socket says 172.21.15.97:47364) 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 -- 172.21.15.97:0/3115523426 learned_addr learned my addr 172.21.15.97:0/3115523426 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79db51700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7980af4d0 0x7fe798146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79e352700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7981603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 -- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 msgr2=0x7fe7981603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7981603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 -- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7980af4d0 msgr2=0x7fe798146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7980af4d0 0x7fe798146090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 -- 172.21.15.97:0/3115523426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe790009a70 con 0x7fe79815e080 2023-11-24T06:10:09.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79db51700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7980af4d0 0x7fe798146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79e352700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7981603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe79eb53700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79815e080 0x7fe7981465d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe790009f00 tx=0x7fe79000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe78effd700 1 -- 172.21.15.97:0/3115523426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790009d70 con 0x7fe79815e080 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe78effd700 1 -- 172.21.15.97:0/3115523426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79000f040 con 0x7fe79815e080 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7880094a0 con 0x7fe79815e080 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe78effd700 1 -- 172.21.15.97:0/3115523426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790014680 con 0x7fe79815e080 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.105+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7980cfec0 con 0x7fe79815e080 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.106+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7980753f0 con 0x7fe79815e080 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.107+0000 7fe78effd700 1 -- 172.21.15.97:0/3115523426 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe7900147e0 con 0x7fe79815e080 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.107+0000 7fe78effd700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe77c06af60 0x7fe77c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.107+0000 7fe79e352700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe77c06af60 0x7fe77c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.107+0000 7fe78effd700 1 -- 172.21.15.97:0/3115523426 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe79008a500 con 0x7fe79815e080 2023-11-24T06:10:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.108+0000 7fe79e352700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe77c06af60 0x7fe77c06d420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe794005910 tx=0x7fe794009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.112+0000 7fe78effd700 1 -- 172.21.15.97:0/3115523426 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe790055440 con 0x7fe79815e080 2023-11-24T06:10:09.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.301+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7981542c0 con 0x7fe79815e080 2023-11-24T06:10:09.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.302+0000 7fe78effd700 1 -- 172.21.15.97:0/3115523426 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16581 (secure 0 0 0) 0x7fe7981542c0 con 0x7fe79815e080 2023-11-24T06:10:09.307 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:09.307 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:57.042131+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:09:53.319367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:09:54.404860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:09.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.304+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe77c06af60 msgr2=0x7fe77c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.304+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe77c06af60 0x7fe77c06d420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe794005910 tx=0x7fe794009040 comp rx=0 tx=0).stop 2023-11-24T06:10:09.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.304+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79815e080 msgr2=0x7fe7981465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.304+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79815e080 0x7fe7981465d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe790009f00 tx=0x7fe79000c3a0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.305+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 shutdown_connections 2023-11-24T06:10:09.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.305+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe79815e080 0x7fe7981465d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.305+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe77c06af60 0x7fe77c06d420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.305+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe79806c710 0x7fe7981603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.305+0000 7fe7a05b6700 1 --2- 172.21.15.97:0/3115523426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7980af4d0 0x7fe798146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.305+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 >> 172.21.15.97:0/3115523426 conn(0x7fe798071b90 msgr2=0x7fe7980af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.305+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 shutdown_connections 2023-11-24T06:10:09.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.305+0000 7fe7a05b6700 1 -- 172.21.15.97:0/3115523426 wait complete. 2023-11-24T06:10:09.321 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:09:53.319367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:09:54.404860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:09.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:09.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.441+0000 7f4bf8a03700 1 Processor -- start 2023-11-24T06:10:09.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.441+0000 7f4bf8a03700 1 -- start start 2023-11-24T06:10:09.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf8a03700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4bf4158390 0x7f4bf40afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf8a03700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4bf4073f90 0x7f4bf4074370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf8a03700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf40ae610 con 0x7f4bf40ad870 2023-11-24T06:10:09.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf8a03700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf40748b0 con 0x7f4bf4158390 2023-11-24T06:10:09.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf8a03700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf40ae050 con 0x7f4bf4073f90 2023-11-24T06:10:09.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf2d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4bf4158390 0x7f4bf40afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4bf40ad870 0x7f4bf406b970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39820/0 (socket says 172.21.15.97:39820) 2023-11-24T06:10:09.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf259c700 1 -- 172.21.15.97:0/64331192 learned_addr learned my addr 172.21.15.97:0/64331192 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:09.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf2d9d700 1 -- 172.21.15.97:0/64331192 >> v2:172.21.15.183:3300/0 conn(0x7f4bf4073f90 msgr2=0x7f4bf4074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:09.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf2d9d700 1 --2- 172.21.15.97:0/64331192 >> v2:172.21.15.183:3300/0 conn(0x7f4bf4073f90 0x7f4bf4074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf2d9d700 1 -- 172.21.15.97:0/64331192 >> v1:172.21.15.97:6789/0 conn(0x7f4bf40ad870 legacy=0x7f4bf406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.442+0000 7f4bf2d9d700 1 -- 172.21.15.97:0/64331192 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40ae330 con 0x7f4bf4158390 2023-11-24T06:10:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf2d9d700 1 --2- 172.21.15.97:0/64331192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf4158390 0x7f4bf40afe50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40d08c0 tx=0x7f4be8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd36efa46e85d60a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf1d9b700 1 -- 172.21.15.97:0/64331192 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8011830 con 0x7f4bf4158390 2023-11-24T06:10:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf1d9b700 1 -- 172.21.15.97:0/64331192 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be8011990 con 0x7f4bf4158390 2023-11-24T06:10:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf1d9b700 1 -- 172.21.15.97:0/64331192 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8011d20 con 0x7f4bf4158390 2023-11-24T06:10:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf8a03700 1 -- 172.21.15.97:0/64331192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf4158390 msgr2=0x7f4bf40afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/64331192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf4158390 0x7f4bf40afe50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40d08c0 tx=0x7f4be8009b20 comp rx=0 tx=0).stop 2023-11-24T06:10:09.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf8a03700 1 -- 172.21.15.97:0/64331192 shutdown_connections 2023-11-24T06:10:09.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/64331192 >> v2:172.21.15.183:3300/0 conn(0x7f4bf4073f90 0x7f4bf4074370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/64331192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf4158390 0x7f4bf40afe50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf8a03700 1 -- 172.21.15.97:0/64331192 >> 172.21.15.97:0/64331192 conn(0x7f4bf40d0d40 msgr2=0x7f4bf40d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf8a03700 1 -- 172.21.15.97:0/64331192 shutdown_connections 2023-11-24T06:10:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.443+0000 7f4bf8a03700 1 -- 172.21.15.97:0/64331192 wait complete. 2023-11-24T06:10:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.445+0000 7f4bf8a03700 1 Processor -- start 2023-11-24T06:10:09.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.445+0000 7f4bf8a03700 1 -- start start 2023-11-24T06:10:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf8a03700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf4073f90 0x7f4bf4151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf8a03700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf40ad870 0x7f4bf415fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf8a03700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4158390 0x7f4bf414eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf8a03700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf4155770 con 0x7f4bf40ad870 2023-11-24T06:10:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf8a03700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf4155a50 con 0x7f4bf4158390 2023-11-24T06:10:09.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf8a03700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf4155d30 con 0x7f4bf4073f90 2023-11-24T06:10:09.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf40ad870 0x7f4bf415fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf40ad870 0x7f4bf415fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47376/0 (socket says 172.21.15.97:47376) 2023-11-24T06:10:09.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 -- 172.21.15.97:0/505692366 learned_addr learned my addr 172.21.15.97:0/505692366 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:09.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf359e700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4158390 0x7f4bf414eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf2d9d700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf4073f90 0x7f4bf4151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 -- 172.21.15.97:0/505692366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf4073f90 msgr2=0x7f4bf4151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf4073f90 0x7f4bf4151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 -- 172.21.15.97:0/505692366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4158390 msgr2=0x7f4bf414eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4158390 0x7f4bf414eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 -- 172.21.15.97:0/505692366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be80094a0 con 0x7f4bf40ad870 2023-11-24T06:10:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf259c700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf40ad870 0x7f4bf415fae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4be400c520 tx=0x7f4be400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bdb7fe700 1 -- 172.21.15.97:0/505692366 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4005d70 con 0x7f4bf40ad870 2023-11-24T06:10:09.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bdb7fe700 1 -- 172.21.15.97:0/505692366 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40070d0 con 0x7f4bf40ad870 2023-11-24T06:10:09.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf4156010 con 0x7f4bf40ad870 2023-11-24T06:10:09.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bdb7fe700 1 -- 172.21.15.97:0/505692366 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4007440 con 0x7f4bf40ad870 2023-11-24T06:10:09.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.446+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf414f980 con 0x7f4bf40ad870 2023-11-24T06:10:09.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.447+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf4068480 con 0x7f4bf40ad870 2023-11-24T06:10:09.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.448+0000 7f4bdb7fe700 1 -- 172.21.15.97:0/505692366 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4be40075a0 con 0x7f4bf40ad870 2023-11-24T06:10:09.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.448+0000 7f4bdb7fe700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4be006af60 0x7f4be006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.448+0000 7f4bdb7fe700 1 -- 172.21.15.97:0/505692366 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4be4088880 con 0x7f4bf40ad870 2023-11-24T06:10:09.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.448+0000 7f4bf2d9d700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4be006af60 0x7f4be006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.448+0000 7f4bf2d9d700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4be006af60 0x7f4be006d420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4be8000c00 tx=0x7f4be801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.452+0000 7f4bdb7fe700 1 -- 172.21.15.97:0/505692366 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4be40539f0 con 0x7f4bf40ad870 2023-11-24T06:10:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.641+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bf40751e0 con 0x7f4bf40ad870 2023-11-24T06:10:09.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.641+0000 7f4bdb7fe700 1 -- 172.21.15.97:0/505692366 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4be4017020 con 0x7f4bf40ad870 2023-11-24T06:10:09.645 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:09.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4be006af60 msgr2=0x7f4be006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4be006af60 0x7f4be006d420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4be8000c00 tx=0x7f4be801c040 comp rx=0 tx=0).stop 2023-11-24T06:10:09.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf40ad870 msgr2=0x7f4bf415fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf40ad870 0x7f4bf415fae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4be400c520 tx=0x7f4be400c830 comp rx=0 tx=0).stop 2023-11-24T06:10:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 shutdown_connections 2023-11-24T06:10:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4bf4158390 0x7f4bf414eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4be006af60 0x7f4be006d420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4bf4073f90 0x7f4bf4151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 --2- 172.21.15.97:0/505692366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf40ad870 0x7f4bf415fae0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 >> 172.21.15.97:0/505692366 conn(0x7f4bf40d0d40 msgr2=0x7f4bf40afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 shutdown_connections 2023-11-24T06:10:09.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.643+0000 7f4bf8a03700 1 -- 172.21.15.97:0/505692366 wait complete. 2023-11-24T06:10:09.659 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:09.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.778+0000 7f749d90f700 1 Processor -- start 2023-11-24T06:10:09.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.778+0000 7f749d90f700 1 -- start start 2023-11-24T06:10:09.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f749d90f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f749815e1d0 0x7f74980d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f749d90f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980b5950 0x7f74980b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f749d90f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74980b62e0 con 0x7f74980cf240 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f749d90f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980d1aa0 con 0x7f749815e1d0 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f749d90f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980d2ff0 con 0x7f74980b5950 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f74967fc700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f74980cf240 0x7f74980d3430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:49156/0 (socket says 172.21.15.97:49156) 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f7496ffd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f749815e1d0 0x7f74980d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f74967fc700 1 -- 172.21.15.97:0/3742117130 learned_addr learned my addr 172.21.15.97:0/3742117130 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f74977fe700 1 --2- 172.21.15.97:0/3742117130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980b5950 0x7f74980b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f7496ffd700 1 -- 172.21.15.97:0/3742117130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980b5950 msgr2=0x7f74980b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f7496ffd700 1 --2- 172.21.15.97:0/3742117130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980b5950 0x7f74980b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f7496ffd700 1 -- 172.21.15.97:0/3742117130 >> v1:172.21.15.183:6789/0 conn(0x7f74980cf240 legacy=0x7f74980d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f7496ffd700 1 -- 172.21.15.97:0/3742117130 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980d3b40 con 0x7f749815e1d0 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.779+0000 7f7495ffb700 1 -- 172.21.15.97:0/3742117130 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782399268 0 0) 0x7f7488001110 con 0x7f74980cf240 2023-11-24T06:10:09.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f7496ffd700 1 --2- 172.21.15.97:0/3742117130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f749815e1d0 0x7f74980d2ab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f748c009840 tx=0x7f748c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5709580670c02541 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f7495ffb700 1 -- 172.21.15.97:0/3742117130 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c011830 con 0x7f749815e1d0 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f7495ffb700 1 -- 172.21.15.97:0/3742117130 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c011990 con 0x7f749815e1d0 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f7495ffb700 1 -- 172.21.15.97:0/3742117130 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c011d20 con 0x7f749815e1d0 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f749d90f700 1 -- 172.21.15.97:0/3742117130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f749815e1d0 msgr2=0x7f74980d2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f749d90f700 1 --2- 172.21.15.97:0/3742117130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f749815e1d0 0x7f74980d2ab0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f748c009840 tx=0x7f748c009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f749d90f700 1 -- 172.21.15.97:0/3742117130 shutdown_connections 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f749d90f700 1 --2- 172.21.15.97:0/3742117130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980b5950 0x7f74980b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f749d90f700 1 --2- 172.21.15.97:0/3742117130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f749815e1d0 0x7f74980d2ab0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f749d90f700 1 -- 172.21.15.97:0/3742117130 >> 172.21.15.97:0/3742117130 conn(0x7f74980d0950 msgr2=0x7f74980d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f749d90f700 1 -- 172.21.15.97:0/3742117130 shutdown_connections 2023-11-24T06:10:09.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.780+0000 7f749d90f700 1 -- 172.21.15.97:0/3742117130 wait complete. 2023-11-24T06:10:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.782+0000 7f749d90f700 1 Processor -- start 2023-11-24T06:10:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.782+0000 7f749d90f700 1 -- start start 2023-11-24T06:10:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.782+0000 7f749d90f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74980b5950 0x7f74980c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.782+0000 7f749d90f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980cf240 0x7f74980b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.782+0000 7f749d90f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749815e1d0 0x7f74980c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.782+0000 7f749d90f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980c9060 con 0x7f749815e1d0 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.782+0000 7f749d90f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980c8450 con 0x7f74980b5950 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.782+0000 7f749d90f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980c8730 con 0x7f74980cf240 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749815e1d0 0x7f74980c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749815e1d0 0x7f74980c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47392/0 (socket says 172.21.15.97:47392) 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 -- 172.21.15.97:0/893279723 learned_addr learned my addr 172.21.15.97:0/893279723 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f7496ffd700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74980b5950 0x7f74980c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74967fc700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980cf240 0x7f74980b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 -- 172.21.15.97:0/893279723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980cf240 msgr2=0x7f74980b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980cf240 0x7f74980b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 -- 172.21.15.97:0/893279723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74980b5950 msgr2=0x7f74980c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74980b5950 0x7f74980c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 -- 172.21.15.97:0/893279723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0094a0 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f74977fe700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749815e1d0 0x7f74980c7a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f749000efc0 tx=0x7f749000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f747ffff700 1 -- 172.21.15.97:0/893279723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490009ea0 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74980c8a10 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749815bd20 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f747ffff700 1 -- 172.21.15.97:0/893279723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749000f040 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.783+0000 7f747ffff700 1 -- 172.21.15.97:0/893279723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490014480 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.784+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74780052e0 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.785+0000 7f747ffff700 1 -- 172.21.15.97:0/893279723 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f74900145e0 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.785+0000 7f747ffff700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f748406af60 0x7f748406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.785+0000 7f747ffff700 1 -- 172.21.15.97:0/893279723 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f749008a150 con 0x7f749815e1d0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.785+0000 7f7496ffd700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f748406af60 0x7f748406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.786+0000 7f7496ffd700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f748406af60 0x7f748406d420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f748c009fd0 tx=0x7f748c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:09.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.790+0000 7f747ffff700 1 -- 172.21.15.97:0/893279723 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7490055190 con 0x7f749815e1d0 2023-11-24T06:10:09.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.978+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7478005b10 con 0x7f749815e1d0 2023-11-24T06:10:09.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.978+0000 7f747ffff700 1 -- 172.21.15.97:0/893279723 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74900587b0 con 0x7f749815e1d0 2023-11-24T06:10:09.982 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:10:09.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f748406af60 msgr2=0x7f748406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f748406af60 0x7f748406d420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f748c009fd0 tx=0x7f748c01c040 comp rx=0 tx=0).stop 2023-11-24T06:10:09.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749815e1d0 msgr2=0x7f74980c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:09.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749815e1d0 0x7f74980c7a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f749000efc0 tx=0x7f749000c3a0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 shutdown_connections 2023-11-24T06:10:09.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f749815e1d0 0x7f74980c7a00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74980b5950 0x7f74980c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f748406af60 0x7f748406d420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 --2- 172.21.15.97:0/893279723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f74980cf240 0x7f74980b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:09.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 >> 172.21.15.97:0/893279723 conn(0x7f74980d0950 msgr2=0x7f74980b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:09.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 shutdown_connections 2023-11-24T06:10:09.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:09.980+0000 7f749d90f700 1 -- 172.21.15.97:0/893279723 wait complete. 2023-11-24T06:10:09.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:10.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.117+0000 7f77875b2700 1 Processor -- start 2023-11-24T06:10:10.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.117+0000 7f77875b2700 1 -- start start 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f77875b2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f77801587f0 0x7f778006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f77875b2700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778006b530 con 0x7f77800b4370 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f77875b2700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77800ae5e0 con 0x7f778015b370 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f77875b2700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778015be30 con 0x7f77801587f0 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f7785b4f700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f77800b4370 0x7f77800af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39848/0 (socket says 172.21.15.97:39848) 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f7785b4f700 1 -- 172.21.15.97:0/3769927033 learned_addr learned my addr 172.21.15.97:0/3769927033 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815009838 0 0) 0x7f778006b530 con 0x7f77800b4370 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f776c0035b0 con 0x7f77800b4370 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.118+0000 7f778534e700 1 --2- 172.21.15.97:0/3769927033 >> v2:172.21.15.183:3300/0 conn(0x7f77801587f0 0x7f778006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3186590505 0 0) 0x7f776c0035b0 con 0x7f77800b4370 2023-11-24T06:10:10.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 >> v2:172.21.15.183:3300/0 conn(0x7f77801587f0 msgr2=0x7f778006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 --2- 172.21.15.97:0/3769927033 >> v2:172.21.15.183:3300/0 conn(0x7f77801587f0 0x7f778006aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 >> v1:172.21.15.183:6789/0 conn(0x7f778015b370 legacy=0x7f77800b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778015c110 con 0x7f77800b4370 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f77780019f0 con 0x7f77800b4370 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 >> v1:172.21.15.97:6789/0 conn(0x7f77800b4370 legacy=0x7f77800af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f776c003a00 0x7f776c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c004850 0x7f776c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f776c005330 0x7f776c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778015be30 con 0x7f776c003a00 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c00c200 con 0x7f776c004850 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c00c3c0 con 0x7f776c005330 2023-11-24T06:10:10.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f778534e700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f776c003a00 0x7f776c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7784b4d700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c004850 0x7f776c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7785b4f700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f776c005330 0x7f776c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f778534e700 1 -- 172.21.15.97:0/3769927033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f776c005330 msgr2=0x7f776c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f778534e700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f776c005330 0x7f776c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f778534e700 1 -- 172.21.15.97:0/3769927033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c004850 msgr2=0x7f776c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f778534e700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c004850 0x7f776c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f778534e700 1 -- 172.21.15.97:0/3769927033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c0035b0 con 0x7f776c003a00 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7785b4f700 1 -- 172.21.15.97:0/3769927033 reap_dead start 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f778534e700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f776c003a00 0x7f776c003e20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7770009860 tx=0x7f776c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0779c16d775f7ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777000c070 con 0x7f776c003a00 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770013610 con 0x7f776c003a00 2023-11-24T06:10:10.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.119+0000 7f7777fff700 1 -- 172.21.15.97:0/3769927033 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77700139a0 con 0x7f776c003a00 2023-11-24T06:10:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.120+0000 7f77875b2700 1 -- 172.21.15.97:0/3769927033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f776c003a00 msgr2=0x7f776c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.120+0000 7f77875b2700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f776c003a00 0x7f776c003e20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7770009860 tx=0x7f776c005880 comp rx=0 tx=0).stop 2023-11-24T06:10:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.120+0000 7f77875b2700 1 -- 172.21.15.97:0/3769927033 shutdown_connections 2023-11-24T06:10:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.120+0000 7f77875b2700 1 --2- 172.21.15.97:0/3769927033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f776c003a00 0x7f776c003e20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.120+0000 7f77875b2700 1 -- 172.21.15.97:0/3769927033 >> 172.21.15.97:0/3769927033 conn(0x7f77800d0bf0 msgr2=0x7f77800d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.120+0000 7f77875b2700 1 -- 172.21.15.97:0/3769927033 shutdown_connections 2023-11-24T06:10:10.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.120+0000 7f77875b2700 1 -- 172.21.15.97:0/3769927033 wait complete. 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.122+0000 7f77875b2700 1 Processor -- start 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.122+0000 7f77875b2700 1 -- start start 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f77875b2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c003a00 0x7f77801466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f77875b2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f778009b640 0x7f778009bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f77875b2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778009d810 0x7f778009dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f77875b2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77800b0330 con 0x7f778009d810 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f77875b2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7780146c10 con 0x7f776c003a00 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f77875b2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778009c000 con 0x7f778009b640 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778009d810 0x7f778009dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778009d810 0x7f778009dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47410/0 (socket says 172.21.15.97:47410) 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 -- 172.21.15.97:0/236378078 learned_addr learned my addr 172.21.15.97:0/236378078 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7784b4d700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f778009b640 0x7f778009bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f778534e700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c003a00 0x7f77801466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 -- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f778009b640 msgr2=0x7f778009bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f778009b640 0x7f778009bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 -- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c003a00 msgr2=0x7f77801466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c003a00 0x7f77801466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 -- 172.21.15.97:0/236378078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778009ede0 con 0x7f778009d810 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f778534e700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c003a00 0x7f77801466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7785b4f700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778009d810 0x7f778009dc90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7780072de0 tx=0x7f7778007330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7775ffb700 1 -- 172.21.15.97:0/236378078 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778012450 con 0x7f778009d810 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7775ffb700 1 -- 172.21.15.97:0/236378078 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77780125b0 con 0x7f778009d810 2023-11-24T06:10:10.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778009e540 con 0x7f778009d810 2023-11-24T06:10:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.123+0000 7f7775ffb700 1 -- 172.21.15.97:0/236378078 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7778012960 con 0x7f778009d810 2023-11-24T06:10:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.124+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778009eb00 con 0x7f778009d810 2023-11-24T06:10:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.124+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77800b05a0 con 0x7f778009d810 2023-11-24T06:10:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.125+0000 7f7775ffb700 1 -- 172.21.15.97:0/236378078 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7778012ac0 con 0x7f778009d810 2023-11-24T06:10:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.125+0000 7f7775ffb700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f776406af60 0x7f776406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.125+0000 7f7775ffb700 1 -- 172.21.15.97:0/236378078 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f777808bf60 con 0x7f778009d810 2023-11-24T06:10:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.125+0000 7f778534e700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f776406af60 0x7f776406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.126+0000 7f778534e700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f776406af60 0x7f776406d420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7770008010 tx=0x7f777000c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.129+0000 7f7775ffb700 1 -- 172.21.15.97:0/236378078 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7778056f10 con 0x7f778009d810 2023-11-24T06:10:10.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.320+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7780068fb0 con 0x7f778009d810 2023-11-24T06:10:10.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.320+0000 7f7775ffb700 1 -- 172.21.15.97:0/236378078 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f777805a530 con 0x7f778009d810 2023-11-24T06:10:10.324 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:10:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f776406af60 msgr2=0x7f776406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f776406af60 0x7f776406d420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7770008010 tx=0x7f777000c000 comp rx=0 tx=0).stop 2023-11-24T06:10:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778009d810 msgr2=0x7f778009dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778009d810 0x7f778009dc90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7780072de0 tx=0x7f7778007330 comp rx=0 tx=0).stop 2023-11-24T06:10:10.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 shutdown_connections 2023-11-24T06:10:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f778009d810 0x7f778009dc90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f776c003a00 0x7f77801466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f776406af60 0x7f776406d420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 --2- 172.21.15.97:0/236378078 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f778009b640 0x7f778009bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 >> 172.21.15.97:0/236378078 conn(0x7f77800d0bf0 msgr2=0x7f77801571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 shutdown_connections 2023-11-24T06:10:10.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.322+0000 7f77875b2700 1 -- 172.21.15.97:0/236378078 wait complete. 2023-11-24T06:10:10.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:10:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.454+0000 7f13062d2700 1 Processor -- start 2023-11-24T06:10:10.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.454+0000 7f13062d2700 1 -- start start 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f13062d2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f13000b2760 0x7f13000b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f13062d2700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130006d360 con 0x7f1300158000 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f13062d2700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130015b100 con 0x7f130015bab0 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f13062d2700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13000629b0 con 0x7f13000b2760 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f12fffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1300158000 0x7f13000622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39866/0 (socket says 172.21.15.97:39866) 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f12fffff700 1 -- 172.21.15.97:0/1358707781 learned_addr learned my addr 172.21.15.97:0/1358707781 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f12feffd700 1 -- 172.21.15.97:0/1358707781 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395677784 0 0) 0x7f130006d360 con 0x7f1300158000 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f12feffd700 1 -- 172.21.15.97:0/1358707781 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e80035b0 con 0x7f1300158000 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.455+0000 7f130486f700 1 --2- 172.21.15.97:0/1358707781 >> v2:172.21.15.183:3300/0 conn(0x7f13000b2760 0x7f13000b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f12feffd700 1 -- 172.21.15.97:0/1358707781 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013051124 0 0) 0x7f130015b100 con 0x7f130015bab0 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f12feffd700 1 -- 172.21.15.97:0/1358707781 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f130006d360 con 0x7f130015bab0 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f12feffd700 1 -- 172.21.15.97:0/1358707781 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2837693397 0 0) 0x7f12e80035b0 con 0x7f1300158000 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f130486f700 1 -- 172.21.15.97:0/1358707781 >> v1:172.21.15.183:6789/0 conn(0x7f130015bab0 legacy=0x7f13000b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f130486f700 1 -- 172.21.15.97:0/1358707781 >> v1:172.21.15.97:6789/0 conn(0x7f1300158000 legacy=0x7f13000622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f130486f700 1 -- 172.21.15.97:0/1358707781 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130015b3e0 con 0x7f13000b2760 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f130486f700 1 --2- 172.21.15.97:0/1358707781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 0x7f13000b3580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009d90 tx=0x7f12f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e7408b60e020446 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f12feffd700 1 -- 172.21.15.97:0/1358707781 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f800e070 con 0x7f13000b2760 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f12feffd700 1 -- 172.21.15.97:0/1358707781 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8003a20 con 0x7f13000b2760 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.456+0000 7f12feffd700 1 -- 172.21.15.97:0/1358707781 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f40015f0 con 0x7f13000b2760 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.457+0000 7f13062d2700 1 -- 172.21.15.97:0/1358707781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 msgr2=0x7f13000b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.457+0000 7f13062d2700 1 --2- 172.21.15.97:0/1358707781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 0x7f13000b3580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009d90 tx=0x7f12f800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:10.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.457+0000 7f13062d2700 1 -- 172.21.15.97:0/1358707781 shutdown_connections 2023-11-24T06:10:10.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.457+0000 7f13062d2700 1 --2- 172.21.15.97:0/1358707781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 0x7f13000b3580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.457+0000 7f13062d2700 1 -- 172.21.15.97:0/1358707781 >> 172.21.15.97:0/1358707781 conn(0x7f13000743b0 msgr2=0x7f130006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:10.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.457+0000 7f13062d2700 1 -- 172.21.15.97:0/1358707781 shutdown_connections 2023-11-24T06:10:10.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.457+0000 7f13062d2700 1 -- 172.21.15.97:0/1358707781 wait complete. 2023-11-24T06:10:10.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.458+0000 7f13062d2700 1 Processor -- start 2023-11-24T06:10:10.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.458+0000 7f13062d2700 1 -- start start 2023-11-24T06:10:10.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f13062d2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 0x7f1300156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f13062d2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1300158000 0x7f13000ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f13062d2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f130015bab0 0x7f130014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f13062d2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13000addb0 con 0x7f1300158000 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f13062d2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13001f3ce0 con 0x7f13000b2760 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f13062d2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130014cc80 con 0x7f130015bab0 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1300158000 0x7f13000ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1300158000 0x7f13000ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47424/0 (socket says 172.21.15.97:47424) 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 -- 172.21.15.97:0/113664465 learned_addr learned my addr 172.21.15.97:0/113664465 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12fffff700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 0x7f1300156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f130486f700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f130015bab0 0x7f130014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 -- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f130015bab0 msgr2=0x7f130014c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f130015bab0 0x7f130014c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 -- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 msgr2=0x7f1300156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 0x7f1300156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 -- 172.21.15.97:0/113664465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f8009a70 con 0x7f1300158000 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f130486f700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f130015bab0 0x7f130014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.459+0000 7f12ff7fe700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1300158000 0x7f13000ad870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12f0001eb0 tx=0x7f12f000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.460+0000 7f12fcff9700 1 -- 172.21.15.97:0/113664465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00052d0 con 0x7f1300158000 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.460+0000 7f12fcff9700 1 -- 172.21.15.97:0/113664465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0005450 con 0x7f1300158000 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.460+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13000aebb0 con 0x7f1300158000 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.460+0000 7f12fcff9700 1 -- 172.21.15.97:0/113664465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00038c0 con 0x7f1300158000 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.460+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13000adfa0 con 0x7f1300158000 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.461+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13000aed40 con 0x7f1300158000 2023-11-24T06:10:10.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.461+0000 7f12fcff9700 1 -- 172.21.15.97:0/113664465 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f12f0003a20 con 0x7f1300158000 2023-11-24T06:10:10.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.461+0000 7f12fcff9700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f12ec06af60 0x7f12ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.461+0000 7f12fcff9700 1 -- 172.21.15.97:0/113664465 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12f0088a40 con 0x7f1300158000 2023-11-24T06:10:10.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.461+0000 7f12fffff700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f12ec06af60 0x7f12ec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.462+0000 7f12fffff700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f12ec06af60 0x7f12ec06d420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f12f4005990 tx=0x7f12f4001530 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.466+0000 7f12fcff9700 1 -- 172.21.15.97:0/113664465 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12f0053b90 con 0x7f1300158000 2023-11-24T06:10:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.673+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1300067f60 con 0x7f1300158000 2023-11-24T06:10:10.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.674+0000 7f12fcff9700 1 -- 172.21.15.97:0/113664465 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f12f0017080 con 0x7f1300158000 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f12ec06af60 msgr2=0x7f12ec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f12ec06af60 0x7f12ec06d420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f12f4005990 tx=0x7f12f4001530 comp rx=0 tx=0).stop 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1300158000 msgr2=0x7f13000ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1300158000 0x7f13000ad870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f12f0001eb0 tx=0x7f12f000c860 comp rx=0 tx=0).stop 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 shutdown_connections 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f12ec06af60 0x7f12ec06d420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f130015bab0 0x7f130014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13000b2760 0x7f1300156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 --2- 172.21.15.97:0/113664465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1300158000 0x7f13000ad870 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 >> 172.21.15.97:0/113664465 conn(0x7f13000743b0 msgr2=0x7f13000d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:10.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 shutdown_connections 2023-11-24T06:10:10.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.676+0000 7f13062d2700 1 -- 172.21.15.97:0/113664465 wait complete. 2023-11-24T06:10:10.681 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:10:10.691 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:10:10.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:10.692 DEBUG:teuthology.orchestra.run.smithi183:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:10.737 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:10:10.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:10.737 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:10:10.756 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:10.756 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:10.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:10:10.773 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:10.773 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:10.815 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:10:10.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:10.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.931+0000 7fe24831f700 1 Processor -- start 2023-11-24T06:10:10.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.931+0000 7fe24831f700 1 -- start start 2023-11-24T06:10:10.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.932+0000 7fe24831f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe24015bdd0 0x7fe2401604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.932+0000 7fe24831f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.932+0000 7fe24831f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2400b22d0 con 0x7fe2400b3370 2023-11-24T06:10:10.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.932+0000 7fe24831f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400b2f50 con 0x7fe24015bdd0 2023-11-24T06:10:10.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.932+0000 7fe24831f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400d2640 con 0x7fe2400d1c70 2023-11-24T06:10:10.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2460bb700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe24015bdd0 0x7fe2401604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2468bc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2458ba700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fe2400b3370 0x7fe2400b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:49202/0 (socket says 172.21.15.97:49202) 2023-11-24T06:10:10.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2458ba700 1 -- 172.21.15.97:0/3680860739 learned_addr learned my addr 172.21.15.97:0/3680860739 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:10.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2460bb700 1 -- 172.21.15.97:0/3680860739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 msgr2=0x7fe2400d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:10.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2460bb700 1 --2- 172.21.15.97:0/3680860739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2460bb700 1 -- 172.21.15.97:0/3680860739 >> v1:172.21.15.183:6789/0 conn(0x7fe2400b3370 legacy=0x7fe2400b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2460bb700 1 -- 172.21.15.97:0/3680860739 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400b25b0 con 0x7fe24015bdd0 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2468bc700 1 --2- 172.21.15.97:0/3680860739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2460bb700 1 --2- 172.21.15.97:0/3680860739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 0x7fe2401604e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe230009840 tx=0x7fe230009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1020a15b5c3b410 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2450b9700 1 -- 172.21.15.97:0/3680860739 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230011830 con 0x7fe24015bdd0 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2450b9700 1 -- 172.21.15.97:0/3680860739 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe230011990 con 0x7fe24015bdd0 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.933+0000 7fe2450b9700 1 -- 172.21.15.97:0/3680860739 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230011d20 con 0x7fe24015bdd0 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.934+0000 7fe24831f700 1 -- 172.21.15.97:0/3680860739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 msgr2=0x7fe2401604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.934+0000 7fe24831f700 1 --2- 172.21.15.97:0/3680860739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 0x7fe2401604e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe230009840 tx=0x7fe230009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.934+0000 7fe24831f700 1 -- 172.21.15.97:0/3680860739 shutdown_connections 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.934+0000 7fe24831f700 1 --2- 172.21.15.97:0/3680860739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.934+0000 7fe24831f700 1 --2- 172.21.15.97:0/3680860739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 0x7fe2401604e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.934+0000 7fe24831f700 1 -- 172.21.15.97:0/3680860739 >> 172.21.15.97:0/3680860739 conn(0x7fe2400731a0 msgr2=0x7fe24009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:10.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.934+0000 7fe24831f700 1 -- 172.21.15.97:0/3680860739 shutdown_connections 2023-11-24T06:10:10.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.934+0000 7fe24831f700 1 -- 172.21.15.97:0/3680860739 wait complete. 2023-11-24T06:10:10.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.935+0000 7fe24831f700 1 Processor -- start 2023-11-24T06:10:10.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.935+0000 7fe24831f700 1 -- start start 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe24831f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400b3370 0x7fe240156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe24831f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe24831f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 0x7fe2400cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe24831f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2401e7190 con 0x7fe2400b3370 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe24831f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2401f4140 con 0x7fe24015bdd0 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe24831f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2401e67f0 con 0x7fe2400d1c70 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2460bb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400b3370 0x7fe240156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2460bb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400b3370 0x7fe240156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47434/0 (socket says 172.21.15.97:47434) 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2460bb700 1 -- 172.21.15.97:0/291912751 learned_addr learned my addr 172.21.15.97:0/291912751 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2458ba700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2468bc700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 0x7fe2400cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2460bb700 1 -- 172.21.15.97:0/291912751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 msgr2=0x7fe2400cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2460bb700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2460bb700 1 -- 172.21.15.97:0/291912751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 msgr2=0x7fe2400cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2460bb700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 0x7fe2400cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.936+0000 7fe2460bb700 1 -- 172.21.15.97:0/291912751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe238009a70 con 0x7fe2400b3370 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.937+0000 7fe2460bb700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400b3370 0x7fe240156d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe230009fd0 tx=0x7fe230004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.937+0000 7fe236ffd700 1 -- 172.21.15.97:0/291912751 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300045d0 con 0x7fe2400b3370 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.937+0000 7fe236ffd700 1 -- 172.21.15.97:0/291912751 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23001e070 con 0x7fe2400b3370 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.937+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2300094a0 con 0x7fe2400b3370 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.937+0000 7fe236ffd700 1 -- 172.21.15.97:0/291912751 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300038e0 con 0x7fe2400b3370 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.937+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2401e6db0 con 0x7fe2400b3370 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.938+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe240068a50 con 0x7fe2400b3370 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.938+0000 7fe236ffd700 1 -- 172.21.15.97:0/291912751 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe230003a40 con 0x7fe2400b3370 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.938+0000 7fe236ffd700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe22406af60 0x7fe22406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.939+0000 7fe236ffd700 1 -- 172.21.15.97:0/291912751 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe23008a4c0 con 0x7fe2400b3370 2023-11-24T06:10:10.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.939+0000 7fe2458ba700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe22406af60 0x7fe22406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:10.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.939+0000 7fe2458ba700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe22406af60 0x7fe22406d420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2400c7320 tx=0x7fe23c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:10.943+0000 7fe236ffd700 1 -- 172.21.15.97:0/291912751 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe230055590 con 0x7fe2400b3370 2023-11-24T06:10:11.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.185+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2401e6930 con 0x7fe2400b3370 2023-11-24T06:10:11.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.185+0000 7fe236ffd700 1 -- 172.21.15.97:0/291912751 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fe2401e6930 con 0x7fe2400b3370 2023-11-24T06:10:11.189 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:11.189 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1778230784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1778230784},{"type":"v1","addr":"172.21.15.97:6836","nonce":1778230784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2183416217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2183416217},{"type":"v1","addr":"172.21.15.183:6834","nonce":2183416217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3650123299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3650123299},{"type":"v1","addr":"172.21.15.183:6835","nonce":3650123299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:09:55.467969+0000","modified":"2023-11-24T06:09:58.451611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5070},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5070":{"gid":5070,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3767182596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3767182596},{"type":"v1","addr":"172.21.15.97:6837","nonce":3767182596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe22406af60 msgr2=0x7fe22406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe22406af60 0x7fe22406d420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2400c7320 tx=0x7fe23c009040 comp rx=0 tx=0).stop 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400b3370 msgr2=0x7fe240156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400b3370 0x7fe240156d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe230009fd0 tx=0x7fe230004750 comp rx=0 tx=0).stop 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 shutdown_connections 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe24015bdd0 0x7fe2400cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2400b3370 0x7fe240156d80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe22406af60 0x7fe22406d420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 --2- 172.21.15.97:0/291912751 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2400d1c70 0x7fe2400cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 >> 172.21.15.97:0/291912751 conn(0x7fe2400731a0 msgr2=0x7fe2400b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:11.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 shutdown_connections 2023-11-24T06:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.188+0000 7fe24831f700 1 -- 172.21.15.97:0/291912751 wait complete. 2023-11-24T06:10:11.193 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2023-11-24T06:10:11.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:11.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.320+0000 7f4c91757700 1 Processor -- start 2023-11-24T06:10:11.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.320+0000 7f4c91757700 1 -- start start 2023-11-24T06:10:11.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.320+0000 7f4c91757700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0b1390 0x7f4c8c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.324 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.320+0000 7f4c91757700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4c8c0d2390 0x7f4c8c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.320+0000 7f4c91757700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c8c0b1c60 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.320+0000 7f4c91757700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8c0d0450 con 0x7f4c8c0d2390 2023-11-24T06:10:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.320+0000 7f4c91757700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0d0720 con 0x7f4c8c0b1390 2023-11-24T06:10:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8a7fc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0b1390 0x7f4c8c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8b7fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4c8c0d2390 0x7f4c8c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8affd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f4c8c0b0fb0 0x7f4c8c0cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:49216/0 (socket says 172.21.15.97:49216) 2023-11-24T06:10:11.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8affd700 1 -- 172.21.15.97:0/2333465671 learned_addr learned my addr 172.21.15.97:0/2333465671 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c89ffb700 1 -- 172.21.15.97:0/2333465671 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024666802 0 0) 0x7f4c8c0b1c60 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c89ffb700 1 -- 172.21.15.97:0/2333465671 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c740035b0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8b7fe700 1 -- 172.21.15.97:0/2333465671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0b1390 msgr2=0x7f4c8c06b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8b7fe700 1 --2- 172.21.15.97:0/2333465671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0b1390 0x7f4c8c06b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8b7fe700 1 -- 172.21.15.97:0/2333465671 >> v1:172.21.15.183:6789/0 conn(0x7f4c8c0b0fb0 legacy=0x7f4c8c0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8b7fe700 1 -- 172.21.15.97:0/2333465671 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0d0a00 con 0x7f4c8c0d2390 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8a7fc700 1 --2- 172.21.15.97:0/2333465671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0b1390 0x7f4c8c06b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.321+0000 7f4c8b7fe700 1 --2- 172.21.15.97:0/2333465671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0d2390 0x7f4c8c06bc90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009d90 tx=0x7f4c8400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a197f61178da86db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c89ffb700 1 -- 172.21.15.97:0/2333465671 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400e070 con 0x7f4c8c0d2390 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c89ffb700 1 -- 172.21.15.97:0/2333465671 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8400bb20 con 0x7f4c8c0d2390 2023-11-24T06:10:11.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c89ffb700 1 -- 172.21.15.97:0/2333465671 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84012430 con 0x7f4c8c0d2390 2023-11-24T06:10:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c91757700 1 -- 172.21.15.97:0/2333465671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0d2390 msgr2=0x7f4c8c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c91757700 1 --2- 172.21.15.97:0/2333465671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0d2390 0x7f4c8c06bc90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009d90 tx=0x7f4c8400b330 comp rx=0 tx=0).stop 2023-11-24T06:10:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c91757700 1 -- 172.21.15.97:0/2333465671 shutdown_connections 2023-11-24T06:10:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c91757700 1 --2- 172.21.15.97:0/2333465671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0d2390 0x7f4c8c06bc90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c91757700 1 --2- 172.21.15.97:0/2333465671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0b1390 0x7f4c8c06b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c91757700 1 -- 172.21.15.97:0/2333465671 >> 172.21.15.97:0/2333465671 conn(0x7f4c8c09aa00 msgr2=0x7f4c8c09ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c91757700 1 -- 172.21.15.97:0/2333465671 shutdown_connections 2023-11-24T06:10:11.328 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.322+0000 7f4c91757700 1 -- 172.21.15.97:0/2333465671 wait complete. 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c91757700 1 Processor -- start 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c91757700 1 -- start start 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c91757700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c0b0fb0 0x7f4c8c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c91757700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0b1390 0x7f4c8c158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c91757700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0d2390 0x7f4c8c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c91757700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0c6cb0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c91757700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c1f4630 con 0x7f4c8c0b1390 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c91757700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0b0020 con 0x7f4c8c0d2390 2023-11-24T06:10:11.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.324+0000 7f4c8affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c0b0fb0 0x7f4c8c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c0b0fb0 0x7f4c8c0afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47448/0 (socket says 172.21.15.97:47448) 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8affd700 1 -- 172.21.15.97:0/1451833023 learned_addr learned my addr 172.21.15.97:0/1451833023 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8a7fc700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0b1390 0x7f4c8c158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8b7fe700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0d2390 0x7f4c8c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8affd700 1 -- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0d2390 msgr2=0x7f4c8c0ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8affd700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0d2390 0x7f4c8c0ce8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8affd700 1 -- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0b1390 msgr2=0x7f4c8c158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8affd700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0b1390 0x7f4c8c158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8affd700 1 -- 172.21.15.97:0/1451833023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c800094a0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8b7fe700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0d2390 0x7f4c8c0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c8affd700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c0b0fb0 0x7f4c8c0afae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00c700 tx=0x7f4c7c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c737fe700 1 -- 172.21.15.97:0/1451833023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c005d70 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c737fe700 1 -- 172.21.15.97:0/1451833023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7c0070d0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c84009a70 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c737fe700 1 -- 172.21.15.97:0/1451833023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c007460 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.325+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c1f3ad0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.326+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c068480 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.326+0000 7f4c737fe700 1 -- 172.21.15.97:0/1451833023 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4c7c0075c0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.327+0000 7f4c737fe700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4c7806af60 0x7f4c7806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.327+0000 7f4c8a7fc700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4c7806af60 0x7f4c7806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.327+0000 7f4c737fe700 1 -- 172.21.15.97:0/1451833023 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c7c088be0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.327+0000 7f4c8a7fc700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4c7806af60 0x7f4c7806d420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0b7ce0 tx=0x7f4c80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:11.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.332+0000 7f4c737fe700 1 -- 172.21.15.97:0/1451833023 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c7c053cb0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.522+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c8c06c460 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.523+0000 7f4c737fe700 1 -- 172.21.15.97:0/1451833023 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f4c7c0572d0 con 0x7f4c8c0b0fb0 2023-11-24T06:10:11.527 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:11.527 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:57.042131+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:09:53.319367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:09:54.404860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4c7806af60 msgr2=0x7f4c7806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4c7806af60 0x7f4c7806d420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0b7ce0 tx=0x7f4c80008040 comp rx=0 tx=0).stop 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c0b0fb0 msgr2=0x7f4c8c0afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c0b0fb0 0x7f4c8c0afae0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00c700 tx=0x7f4c7c00cac0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 shutdown_connections 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4c7806af60 0x7f4c7806d420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4c8c0d2390 0x7f4c8c0ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c8c0b0fb0 0x7f4c8c0afae0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 --2- 172.21.15.97:0/1451833023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c8c0b1390 0x7f4c8c158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.525+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 >> 172.21.15.97:0/1451833023 conn(0x7f4c8c09aa00 msgr2=0x7f4c8c0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:11.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.526+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 shutdown_connections 2023-11-24T06:10:11.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.526+0000 7f4c91757700 1 -- 172.21.15.97:0/1451833023 wait complete. 2023-11-24T06:10:11.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:09:53.319367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:09:54.404860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:11.542 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.661+0000 7f9af4ecc700 1 Processor -- start 2023-11-24T06:10:11.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.661+0000 7f9af4ecc700 1 -- start start 2023-11-24T06:10:11.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9af4ecc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9af00cccd0 0x7f9af00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9af4ecc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9af015c320 0x7f9af015c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9af4ecc700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af015ccb0 con 0x7f9af015bf40 2023-11-24T06:10:11.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9af4ecc700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af00cd6a0 con 0x7f9af00cccd0 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9af4ecc700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af006fd30 con 0x7f9af015c320 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aedd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9af00cccd0 0x7f9af00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aee59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9af015bf40 0x7f9af00af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39908/0 (socket says 172.21.15.97:39908) 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aedd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9af00cccd0 0x7f9af00cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47458/0 (socket says 172.21.15.97:47458) 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aee59c700 1 -- 172.21.15.97:0/797561407 learned_addr learned my addr 172.21.15.97:0/797561407 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aeed9d700 1 --2- 172.21.15.97:0/797561407 >> v2:172.21.15.183:3300/0 conn(0x7f9af015c320 0x7f9af015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aedd9b700 1 -- 172.21.15.97:0/797561407 >> v2:172.21.15.183:3300/0 conn(0x7f9af015c320 msgr2=0x7f9af015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aedd9b700 1 --2- 172.21.15.97:0/797561407 >> v2:172.21.15.183:3300/0 conn(0x7f9af015c320 0x7f9af015c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aedd9b700 1 -- 172.21.15.97:0/797561407 >> v1:172.21.15.97:6789/0 conn(0x7f9af015bf40 legacy=0x7f9af00af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aed59a700 1 -- 172.21.15.97:0/797561407 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1873414259 0 0) 0x7f9af015ccb0 con 0x7f9af015bf40 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aedd9b700 1 -- 172.21.15.97:0/797561407 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af006f770 con 0x7f9af00cccd0 2023-11-24T06:10:11.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aedd9b700 1 --2- 172.21.15.97:0/797561407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af00cccd0 0x7f9af00cd160 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9af006bff0 tx=0x7f9ae4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83dcc7621812c0e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.662+0000 7f9aed59a700 1 -- 172.21.15.97:0/797561407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4011830 con 0x7f9af00cccd0 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9aed59a700 1 -- 172.21.15.97:0/797561407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae4011990 con 0x7f9af00cccd0 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9aed59a700 1 -- 172.21.15.97:0/797561407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4011d20 con 0x7f9af00cccd0 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9af4ecc700 1 -- 172.21.15.97:0/797561407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af00cccd0 msgr2=0x7f9af00cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/797561407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af00cccd0 0x7f9af00cd160 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9af006bff0 tx=0x7f9ae4009b20 comp rx=0 tx=0).stop 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9af4ecc700 1 -- 172.21.15.97:0/797561407 shutdown_connections 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/797561407 >> v2:172.21.15.183:3300/0 conn(0x7f9af015c320 0x7f9af015c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/797561407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af00cccd0 0x7f9af00cd160 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9af4ecc700 1 -- 172.21.15.97:0/797561407 >> 172.21.15.97:0/797561407 conn(0x7f9af00b2050 msgr2=0x7f9af00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9af4ecc700 1 -- 172.21.15.97:0/797561407 shutdown_connections 2023-11-24T06:10:11.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.663+0000 7f9af4ecc700 1 -- 172.21.15.97:0/797561407 wait complete. 2023-11-24T06:10:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.664+0000 7f9af4ecc700 1 Processor -- start 2023-11-24T06:10:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.664+0000 7f9af4ecc700 1 -- start start 2023-11-24T06:10:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9af4ecc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af00cccd0 0x7f9af01f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9af4ecc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af015bf40 0x7f9af013ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9af4ecc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af015c320 0x7f9af013f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9af4ecc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af013faf0 con 0x7f9af015bf40 2023-11-24T06:10:11.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9af4ecc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af013fdd0 con 0x7f9af00cccd0 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9af4ecc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af01400b0 con 0x7f9af015c320 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9aedd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af015bf40 0x7f9af013ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9aedd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af015bf40 0x7f9af013ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47464/0 (socket says 172.21.15.97:47464) 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9aedd9b700 1 -- 172.21.15.97:0/227871921 learned_addr learned my addr 172.21.15.97:0/227871921 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9aedd9b700 1 -- 172.21.15.97:0/227871921 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af015c320 msgr2=0x7f9af013f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9aedd9b700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af015c320 0x7f9af013f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9aedd9b700 1 -- 172.21.15.97:0/227871921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af00cccd0 msgr2=0x7f9af01f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9aedd9b700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af00cccd0 0x7f9af01f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.665+0000 7f9aedd9b700 1 -- 172.21.15.97:0/227871921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae40094a0 con 0x7f9af015bf40 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.666+0000 7f9aedd9b700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af015bf40 0x7f9af013ebb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4000c00 tx=0x7f9ae4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.666+0000 7f9adeffd700 1 -- 172.21.15.97:0/227871921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40038a0 con 0x7f9af015bf40 2023-11-24T06:10:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.666+0000 7f9adeffd700 1 -- 172.21.15.97:0/227871921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae401f070 con 0x7f9af015bf40 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.666+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af0140390 con 0x7f9af015bf40 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.666+0000 7f9adeffd700 1 -- 172.21.15.97:0/227871921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4003c40 con 0x7f9af015bf40 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.666+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af01f55b0 con 0x7f9af015bf40 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.667+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af0068d90 con 0x7f9af015bf40 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.667+0000 7f9adeffd700 1 -- 172.21.15.97:0/227871921 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9ae4026480 con 0x7f9af015bf40 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.667+0000 7f9adeffd700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9ad006af60 0x7f9ad006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.667+0000 7f9adeffd700 1 -- 172.21.15.97:0/227871921 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ae4021070 con 0x7f9af015bf40 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.668+0000 7f9aee59c700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9ad006af60 0x7f9ad006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:11.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.668+0000 7f9aee59c700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9ad006af60 0x7f9ad006d420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00011e0 tx=0x7f9ae00079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:11.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.672+0000 7f9adeffd700 1 -- 172.21.15.97:0/227871921 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ae40561b0 con 0x7f9af015bf40 2023-11-24T06:10:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.860+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9af013fc30 con 0x7f9af015bf40 2023-11-24T06:10:11.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.860+0000 7f9adeffd700 1 -- 172.21.15.97:0/227871921 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9af013fc30 con 0x7f9af015bf40 2023-11-24T06:10:11.865 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.862+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9ad006af60 msgr2=0x7f9ad006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.862+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9ad006af60 0x7f9ad006d420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00011e0 tx=0x7f9ae00079b0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.862+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af015bf40 msgr2=0x7f9af013ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.862+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af015bf40 0x7f9af013ebb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4000c00 tx=0x7f9ae4004750 comp rx=0 tx=0).stop 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.862+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 shutdown_connections 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.862+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9ad006af60 0x7f9ad006d420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.862+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9af015c320 0x7f9af013f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.862+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af00cccd0 0x7f9af01f5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.863+0000 7f9af4ecc700 1 --2- 172.21.15.97:0/227871921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af015bf40 0x7f9af013ebb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.863+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 >> 172.21.15.97:0/227871921 conn(0x7f9af00b2050 msgr2=0x7f9af015a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:11.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.863+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 shutdown_connections 2023-11-24T06:10:11.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.863+0000 7f9af4ecc700 1 -- 172.21.15.97:0/227871921 wait complete. 2023-11-24T06:10:11.879 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:12.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.998+0000 7f51f0d01700 1 Processor -- start 2023-11-24T06:10:12.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.998+0000 7f51f0d01700 1 -- start start 2023-11-24T06:10:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51f0d01700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51f0d01700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f51ec06c540 0x7f51ec15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51f0d01700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec0b46d0 con 0x7f51ec15f2d0 2023-11-24T06:10:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51f0d01700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec06c9c0 con 0x7f51ec06c540 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51f0d01700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec15faf0 con 0x7f51ec143b80 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51e9d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f51ec06c540 0x7f51ec15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51ead9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f51ec15f2d0 0x7f51ec06db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39926/0 (socket says 172.21.15.97:39926) 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51ead9d700 1 -- 172.21.15.97:0/2101337605 learned_addr learned my addr 172.21.15.97:0/2101337605 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51ea59c700 1 --2- 172.21.15.97:0/2101337605 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51e959a700 1 -- 172.21.15.97:0/2101337605 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926296768 0 0) 0x7f51ec0b46d0 con 0x7f51ec15f2d0 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51e959a700 1 -- 172.21.15.97:0/2101337605 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d40035b0 con 0x7f51ec15f2d0 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51e9d9b700 1 -- 172.21.15.97:0/2101337605 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 msgr2=0x7f51ec06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51e9d9b700 1 --2- 172.21.15.97:0/2101337605 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec06c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51e9d9b700 1 -- 172.21.15.97:0/2101337605 >> v1:172.21.15.97:6789/0 conn(0x7f51ec15f2d0 legacy=0x7f51ec06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51e9d9b700 1 -- 172.21.15.97:0/2101337605 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec06fe10 con 0x7f51ec06c540 2023-11-24T06:10:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51ea59c700 1 --2- 172.21.15.97:0/2101337605 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:11.999+0000 7f51e9d9b700 1 --2- 172.21.15.97:0/2101337605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec15ed90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009840 tx=0x7f51e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d124765222fbf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51e959a700 1 -- 172.21.15.97:0/2101337605 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0011830 con 0x7f51ec06c540 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51e959a700 1 -- 172.21.15.97:0/2101337605 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0011990 con 0x7f51ec06c540 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51e959a700 1 -- 172.21.15.97:0/2101337605 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0011d20 con 0x7f51ec06c540 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51f0d01700 1 -- 172.21.15.97:0/2101337605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 msgr2=0x7f51ec15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51f0d01700 1 --2- 172.21.15.97:0/2101337605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec15ed90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009840 tx=0x7f51e0009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51f0d01700 1 -- 172.21.15.97:0/2101337605 shutdown_connections 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51f0d01700 1 --2- 172.21.15.97:0/2101337605 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51f0d01700 1 --2- 172.21.15.97:0/2101337605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec15ed90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51f0d01700 1 -- 172.21.15.97:0/2101337605 >> 172.21.15.97:0/2101337605 conn(0x7f51ec071ac0 msgr2=0x7f51ec071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51f0d01700 1 -- 172.21.15.97:0/2101337605 shutdown_connections 2023-11-24T06:10:12.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.000+0000 7f51f0d01700 1 -- 172.21.15.97:0/2101337605 wait complete. 2023-11-24T06:10:12.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.001+0000 7f51f0d01700 1 Processor -- start 2023-11-24T06:10:12.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51f0d01700 1 -- start start 2023-11-24T06:10:12.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51f0d01700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51f0d01700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec0aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51f0d01700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51ec15f2d0 0x7f51ec0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51f0d01700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec15a270 con 0x7f51ec06c540 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51f0d01700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0abac0 con 0x7f51ec15f2d0 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51f0d01700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0ab7e0 con 0x7f51ec143b80 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51ea59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51ea59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47476/0 (socket says 172.21.15.97:47476) 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.002+0000 7f51ea59c700 1 -- 172.21.15.97:0/3291947550 learned_addr learned my addr 172.21.15.97:0/3291947550 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51ead9d700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51ec15f2d0 0x7f51ec0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51e9d9b700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec0aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51ea59c700 1 -- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 msgr2=0x7f51ec0aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51ea59c700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec0aa9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51ea59c700 1 -- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51ec15f2d0 msgr2=0x7f51ec0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.010 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51ea59c700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51ec15f2d0 0x7f51ec0afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51ea59c700 1 -- 172.21.15.97:0/3291947550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00094a0 con 0x7f51ec06c540 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51ead9d700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51ec15f2d0 0x7f51ec0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51ea59c700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec159780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00e810 tx=0x7f51dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51daffd700 1 -- 172.21.15.97:0/3291947550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc013610 con 0x7f51ec06c540 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51daffd700 1 -- 172.21.15.97:0/3291947550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc013770 con 0x7f51ec06c540 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec0a9f10 con 0x7f51ec06c540 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51daffd700 1 -- 172.21.15.97:0/3291947550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc013b20 con 0x7f51ec06c540 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.003+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec0aa4d0 con 0x7f51ec06c540 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.004+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec068f10 con 0x7f51ec06c540 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.004+0000 7f51daffd700 1 -- 172.21.15.97:0/3291947550 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f51dc013c80 con 0x7f51ec06c540 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.005+0000 7f51daffd700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51cc06af60 0x7f51cc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.005+0000 7f51daffd700 1 -- 172.21.15.97:0/3291947550 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51dc0893f0 con 0x7f51ec06c540 2023-11-24T06:10:12.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.005+0000 7f51e9d9b700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51cc06af60 0x7f51cc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.005+0000 7f51e9d9b700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51cc06af60 0x7f51cc06d420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f51e00090b0 tx=0x7f51e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.009+0000 7f51daffd700 1 -- 172.21.15.97:0/3291947550 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51dc0543a0 con 0x7f51ec06c540 2023-11-24T06:10:12.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.205+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51ec0b52d0 con 0x7f51ec06c540 2023-11-24T06:10:12.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.206+0000 7f51daffd700 1 -- 172.21.15.97:0/3291947550 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51dc0579c0 con 0x7f51ec06c540 2023-11-24T06:10:12.210 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:10:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51cc06af60 msgr2=0x7f51cc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51cc06af60 0x7f51cc06d420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f51e00090b0 tx=0x7f51e0008030 comp rx=0 tx=0).stop 2023-11-24T06:10:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 msgr2=0x7f51ec159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec159780 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00e810 tx=0x7f51dc00ebd0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 shutdown_connections 2023-11-24T06:10:12.212 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51ec15f2d0 0x7f51ec0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51ec06c540 0x7f51ec159780 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51cc06af60 0x7f51cc06d420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 --2- 172.21.15.97:0/3291947550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51ec143b80 0x7f51ec0aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 >> 172.21.15.97:0/3291947550 conn(0x7f51ec071ac0 msgr2=0x7f51ec06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 shutdown_connections 2023-11-24T06:10:12.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.208+0000 7f51f0d01700 1 -- 172.21.15.97:0/3291947550 wait complete. 2023-11-24T06:10:12.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:12.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.342+0000 7f10892be700 1 Processor -- start 2023-11-24T06:10:12.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.342+0000 7f10892be700 1 -- start start 2023-11-24T06:10:12.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.343+0000 7f10892be700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10840d0680 0x7f10840b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.343+0000 7f10892be700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108415e880 con 0x7f10840b3420 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.343+0000 7f10892be700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108415dc70 con 0x7f10841404a0 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.343+0000 7f10892be700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108415df50 con 0x7f10840d0680 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.343+0000 7f10827fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10840d0680 0x7f10840b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.343+0000 7f10837fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f10840b3420 0x7f1084071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39940/0 (socket says 172.21.15.97:39940) 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.343+0000 7f10837fe700 1 -- 172.21.15.97:0/3870216463 learned_addr learned my addr 172.21.15.97:0/3870216463 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f10827fc700 1 -- 172.21.15.97:0/3870216463 >> v1:172.21.15.183:6789/0 conn(0x7f10841404a0 legacy=0x7f10840cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f10827fc700 1 -- 172.21.15.97:0/3870216463 >> v1:172.21.15.97:6789/0 conn(0x7f10840b3420 legacy=0x7f1084071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f10827fc700 1 -- 172.21.15.97:0/3870216463 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108415e230 con 0x7f10840d0680 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f1081ffb700 1 -- 172.21.15.97:0/3870216463 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948083916 0 0) 0x7f108415e880 con 0x7f10840b3420 2023-11-24T06:10:12.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f10827fc700 1 --2- 172.21.15.97:0/3870216463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f10840b2ee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f107400b150 tx=0x7f107400b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c13bd8d57d6bd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f1081ffb700 1 -- 172.21.15.97:0/3870216463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400e070 con 0x7f10840d0680 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f1081ffb700 1 -- 172.21.15.97:0/3870216463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107400bde0 con 0x7f10840d0680 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f1081ffb700 1 -- 172.21.15.97:0/3870216463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074004270 con 0x7f10840d0680 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.344+0000 7f10892be700 1 -- 172.21.15.97:0/3870216463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 msgr2=0x7f10840b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.345+0000 7f10892be700 1 --2- 172.21.15.97:0/3870216463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f10840b2ee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f107400b150 tx=0x7f107400b460 comp rx=0 tx=0).stop 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.345+0000 7f10892be700 1 -- 172.21.15.97:0/3870216463 shutdown_connections 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.345+0000 7f10892be700 1 --2- 172.21.15.97:0/3870216463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f10840b2ee0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.345+0000 7f10892be700 1 -- 172.21.15.97:0/3870216463 >> 172.21.15.97:0/3870216463 conn(0x7f108406b410 msgr2=0x7f108406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.345+0000 7f10892be700 1 -- 172.21.15.97:0/3870216463 shutdown_connections 2023-11-24T06:10:12.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.345+0000 7f10892be700 1 -- 172.21.15.97:0/3870216463 wait complete. 2023-11-24T06:10:12.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.346+0000 7f10892be700 1 Processor -- start 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.346+0000 7f10892be700 1 -- start start 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10892be700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10840b3420 0x7f10840aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10892be700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f108413db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10892be700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10841404a0 0x7f108413e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10892be700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840ae450 con 0x7f10840d0680 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10892be700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1084134e40 con 0x7f10840b3420 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10892be700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1084141650 con 0x7f10841404a0 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f108413db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f108413db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47492/0 (socket says 172.21.15.97:47492) 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 -- 172.21.15.97:0/3054522828 learned_addr learned my addr 172.21.15.97:0/3054522828 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10837fe700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10841404a0 0x7f108413e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f1082ffd700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10840b3420 0x7f10840aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 -- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10841404a0 msgr2=0x7f108413e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10841404a0 0x7f108413e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 -- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10840b3420 msgr2=0x7f10840aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10840b3420 0x7f10840aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 -- 172.21.15.97:0/3054522828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1074009dc0 con 0x7f10840d0680 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f1082ffd700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10840b3420 0x7f10840aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10837fe700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10841404a0 0x7f108413e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10827fc700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f108413db30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f107400b860 tx=0x7f1074004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f106b7fe700 1 -- 172.21.15.97:0/3054522828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074003ab0 con 0x7f10840d0680 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f106b7fe700 1 -- 172.21.15.97:0/3054522828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074003c10 con 0x7f10840d0680 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1084140d20 con 0x7f10840d0680 2023-11-24T06:10:12.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f106b7fe700 1 -- 172.21.15.97:0/3054522828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10740041f0 con 0x7f10840d0680 2023-11-24T06:10:12.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.347+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10841412e0 con 0x7f10840d0680 2023-11-24T06:10:12.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.348+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1084068480 con 0x7f10840d0680 2023-11-24T06:10:12.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.350+0000 7f106b7fe700 1 -- 172.21.15.97:0/3054522828 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1074030480 con 0x7f10840d0680 2023-11-24T06:10:12.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.350+0000 7f106b7fe700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f107006af60 0x7f107006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.350+0000 7f106b7fe700 1 -- 172.21.15.97:0/3054522828 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f107408b350 con 0x7f10840d0680 2023-11-24T06:10:12.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.350+0000 7f1082ffd700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f107006af60 0x7f107006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.350+0000 7f1082ffd700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f107006af60 0x7f107006d420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1078006fd0 tx=0x7f1078008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.354+0000 7f106b7fe700 1 -- 172.21.15.97:0/3054522828 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1074056310 con 0x7f10840d0680 2023-11-24T06:10:12.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.545+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1084141470 con 0x7f10840d0680 2023-11-24T06:10:12.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.545+0000 7f106b7fe700 1 -- 172.21.15.97:0/3054522828 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f107405ee60 con 0x7f10840d0680 2023-11-24T06:10:12.549 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.547+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f107006af60 msgr2=0x7f107006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.547+0000 7f10892be700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f107006af60 0x7f107006d420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1078006fd0 tx=0x7f1078008040 comp rx=0 tx=0).stop 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.547+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 msgr2=0x7f108413db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.547+0000 7f10892be700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f108413db30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f107400b860 tx=0x7f1074004750 comp rx=0 tx=0).stop 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.548+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 shutdown_connections 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.548+0000 7f10892be700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f107006af60 0x7f107006d420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.548+0000 7f10892be700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10841404a0 0x7f108413e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.548+0000 7f10892be700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10840b3420 0x7f10840aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.548+0000 7f10892be700 1 --2- 172.21.15.97:0/3054522828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10840d0680 0x7f108413db30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.548+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 >> 172.21.15.97:0/3054522828 conn(0x7f108406b410 msgr2=0x7f108413f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:12.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.548+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 shutdown_connections 2023-11-24T06:10:12.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.548+0000 7f10892be700 1 -- 172.21.15.97:0/3054522828 wait complete. 2023-11-24T06:10:12.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:12.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.679+0000 7f46e11c6700 1 Processor -- start 2023-11-24T06:10:12.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.679+0000 7f46e11c6700 1 -- start start 2023-11-24T06:10:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.680+0000 7f46e11c6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc1446a0 0x7f46dc0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.680+0000 7f46e11c6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f46dc06c710 0x7f46dc0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.680+0000 7f46e11c6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46dc0af950 con 0x7f46dc0adf00 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.680+0000 7f46e11c6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46dc144a80 con 0x7f46dc06c710 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.680+0000 7f46e11c6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0afc30 con 0x7f46dc1446a0 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.680+0000 7f46db59e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f46dc06c710 0x7f46dc0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.680+0000 7f46db59e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f46dc06c710 0x7f46dc0b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47502/0 (socket says 172.21.15.97:47502) 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46db59e700 1 -- 172.21.15.97:0/1092992836 learned_addr learned my addr 172.21.15.97:0/1092992836 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46dad9d700 1 --2- 172.21.15.97:0/1092992836 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc1446a0 0x7f46dc0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46db59e700 1 -- 172.21.15.97:0/1092992836 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc1446a0 msgr2=0x7f46dc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46db59e700 1 --2- 172.21.15.97:0/1092992836 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc1446a0 0x7f46dc0b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46db59e700 1 -- 172.21.15.97:0/1092992836 >> v1:172.21.15.183:6789/0 conn(0x7f46dc0adf00 legacy=0x7f46dc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46db59e700 1 -- 172.21.15.97:0/1092992836 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc0aff10 con 0x7f46dc06c710 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46db59e700 1 --2- 172.21.15.97:0/1092992836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc06c710 0x7f46dc0b3d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009e50 tx=0x7f46d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fc0dc609eb06bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46d9d9b700 1 -- 172.21.15.97:0/1092992836 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d400e070 con 0x7f46dc06c710 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46d9d9b700 1 -- 172.21.15.97:0/1092992836 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d4003a20 con 0x7f46dc06c710 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.681+0000 7f46d9d9b700 1 -- 172.21.15.97:0/1092992836 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4003dd0 con 0x7f46dc06c710 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.682+0000 7f46e11c6700 1 -- 172.21.15.97:0/1092992836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc06c710 msgr2=0x7f46dc0b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.682+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1092992836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc06c710 0x7f46dc0b3d40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009e50 tx=0x7f46d400b330 comp rx=0 tx=0).stop 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.682+0000 7f46e11c6700 1 -- 172.21.15.97:0/1092992836 shutdown_connections 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.682+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1092992836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc06c710 0x7f46dc0b3d40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.682+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1092992836 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc1446a0 0x7f46dc0b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.682+0000 7f46e11c6700 1 -- 172.21.15.97:0/1092992836 >> 172.21.15.97:0/1092992836 conn(0x7f46dc071b90 msgr2=0x7f46dc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.682+0000 7f46e11c6700 1 -- 172.21.15.97:0/1092992836 shutdown_connections 2023-11-24T06:10:12.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.682+0000 7f46e11c6700 1 -- 172.21.15.97:0/1092992836 wait complete. 2023-11-24T06:10:12.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.683+0000 7f46e11c6700 1 Processor -- start 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.683+0000 7f46e11c6700 1 -- start start 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46e11c6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc06c710 0x7f46dc15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46e11c6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc0adf00 0x7f46dc09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46e11c6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc1446a0 0x7f46dc09af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46e11c6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc13e560 con 0x7f46dc0adf00 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46e11c6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0cf8e0 con 0x7f46dc1446a0 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46e11c6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc15aa90 con 0x7f46dc06c710 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc0adf00 0x7f46dc09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc0adf00 0x7f46dc09aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47506/0 (socket says 172.21.15.97:47506) 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 -- 172.21.15.97:0/1007193847 learned_addr learned my addr 172.21.15.97:0/1007193847 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:12.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46dad9d700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc06c710 0x7f46dc15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46db59e700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc1446a0 0x7f46dc09af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 -- 172.21.15.97:0/1007193847 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc06c710 msgr2=0x7f46dc15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc06c710 0x7f46dc15a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 -- 172.21.15.97:0/1007193847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc1446a0 msgr2=0x7f46dc09af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc1446a0 0x7f46dc09af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 -- 172.21.15.97:0/1007193847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d4009b30 con 0x7f46dc0adf00 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.684+0000 7f46da59c700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc0adf00 0x7f46dc09aa00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c4a0 tx=0x7f46cc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.685+0000 7f46c37fe700 1 -- 172.21.15.97:0/1007193847 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc011790 con 0x7f46dc0adf00 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.685+0000 7f46c37fe700 1 -- 172.21.15.97:0/1007193847 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc0118f0 con 0x7f46dc0adf00 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.685+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc0cf600 con 0x7f46dc0adf00 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.685+0000 7f46c37fe700 1 -- 172.21.15.97:0/1007193847 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc011ca0 con 0x7f46dc0adf00 2023-11-24T06:10:12.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.685+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46dc15be90 con 0x7f46dc0adf00 2023-11-24T06:10:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.686+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46dc0753f0 con 0x7f46dc0adf00 2023-11-24T06:10:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.686+0000 7f46c37fe700 1 -- 172.21.15.97:0/1007193847 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f46cc01c460 con 0x7f46dc0adf00 2023-11-24T06:10:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.686+0000 7f46c37fe700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46c806af60 0x7f46c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.686+0000 7f46c37fe700 1 -- 172.21.15.97:0/1007193847 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46cc089390 con 0x7f46dc0adf00 2023-11-24T06:10:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.687+0000 7f46dad9d700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46c806af60 0x7f46c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:12.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.687+0000 7f46dad9d700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46c806af60 0x7f46c806d420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46dc15b850 tx=0x7f46d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:12.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.691+0000 7f46c37fe700 1 -- 172.21.15.97:0/1007193847 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46cc0544e0 con 0x7f46dc0adf00 2023-11-24T06:10:12.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.922+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46dc06ef20 con 0x7f46dc0adf00 2023-11-24T06:10:12.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.923+0000 7f46c37fe700 1 -- 172.21.15.97:0/1007193847 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f46cc057b00 con 0x7f46dc0adf00 2023-11-24T06:10:12.927 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:12.927 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1778230784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1778230784},{"type":"v1","addr":"172.21.15.97:6836","nonce":1778230784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2183416217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2183416217},{"type":"v1","addr":"172.21.15.183:6834","nonce":2183416217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3650123299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3650123299},{"type":"v1","addr":"172.21.15.183:6835","nonce":3650123299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:09:55.467969+0000","modified":"2023-11-24T06:09:58.451611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5070},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5070":{"gid":5070,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3767182596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3767182596},{"type":"v1","addr":"172.21.15.97:6837","nonce":3767182596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:10:12.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46c806af60 msgr2=0x7f46c806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46c806af60 0x7f46c806d420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f46dc15b850 tx=0x7f46d0008040 comp rx=0 tx=0).stop 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc0adf00 msgr2=0x7f46dc09aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc0adf00 0x7f46dc09aa00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00c4a0 tx=0x7f46cc00c860 comp rx=0 tx=0).stop 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 shutdown_connections 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46dc1446a0 0x7f46dc09af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46c806af60 0x7f46c806d420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46dc06c710 0x7f46dc15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 --2- 172.21.15.97:0/1007193847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46dc0adf00 0x7f46dc09aa00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 >> 172.21.15.97:0/1007193847 conn(0x7f46dc071b90 msgr2=0x7f46dc0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 shutdown_connections 2023-11-24T06:10:12.930 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:12.925+0000 7f46e11c6700 1 -- 172.21.15.97:0/1007193847 wait complete. 2023-11-24T06:10:12.931 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2023-11-24T06:10:12.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:13.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.057+0000 7f00b7d94700 1 Processor -- start 2023-11-24T06:10:13.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.058+0000 7f00b7d94700 1 -- start start 2023-11-24T06:10:13.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.058+0000 7f00b7d94700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b00adf00 0x7f00b006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.058+0000 7f00b7d94700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00b006c540 0x7f00b00b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.058+0000 7f00b7d94700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b00af950 con 0x7f00b01446a0 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.058+0000 7f00b7d94700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b0144a80 con 0x7f00b006c540 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.058+0000 7f00b7d94700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b006c9b0 con 0x7f00b00adf00 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b6331700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00b006c540 0x7f00b00b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b5b30700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f00b01446a0 0x7f00b00b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39964/0 (socket says 172.21.15.97:39964) 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b6331700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f00b006c540 0x7f00b00b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47516/0 (socket says 172.21.15.97:47516) 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b5b30700 1 -- 172.21.15.97:0/2158516402 learned_addr learned my addr 172.21.15.97:0/2158516402 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b532f700 1 --2- 172.21.15.97:0/2158516402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b00adf00 0x7f00b006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b6331700 1 -- 172.21.15.97:0/2158516402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b00adf00 msgr2=0x7f00b006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:13.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b6331700 1 --2- 172.21.15.97:0/2158516402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b00adf00 0x7f00b006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b6331700 1 -- 172.21.15.97:0/2158516402 >> v1:172.21.15.97:6789/0 conn(0x7f00b01446a0 legacy=0x7f00b00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b6331700 1 -- 172.21.15.97:0/2158516402 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00afc30 con 0x7f00b006c540 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b4b2e700 1 -- 172.21.15.97:0/2158516402 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054179786 0 0) 0x7f00b00af950 con 0x7f00b01446a0 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b6331700 1 --2- 172.21.15.97:0/2158516402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b006c540 0x7f00b00b3d40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009dc0 tx=0x7f00a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3523d9cbb4d112c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b4b2e700 1 -- 172.21.15.97:0/2158516402 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a800e070 con 0x7f00b006c540 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b4b2e700 1 -- 172.21.15.97:0/2158516402 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a800bb90 con 0x7f00b006c540 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.059+0000 7f00b4b2e700 1 -- 172.21.15.97:0/2158516402 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8012430 con 0x7f00b006c540 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.060+0000 7f00b7d94700 1 -- 172.21.15.97:0/2158516402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b006c540 msgr2=0x7f00b00b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.060+0000 7f00b7d94700 1 --2- 172.21.15.97:0/2158516402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b006c540 0x7f00b00b3d40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009dc0 tx=0x7f00a800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.060+0000 7f00b7d94700 1 -- 172.21.15.97:0/2158516402 shutdown_connections 2023-11-24T06:10:13.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.060+0000 7f00b7d94700 1 --2- 172.21.15.97:0/2158516402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b006c540 0x7f00b00b3d40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.060+0000 7f00b7d94700 1 --2- 172.21.15.97:0/2158516402 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b00adf00 0x7f00b006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.060+0000 7f00b7d94700 1 -- 172.21.15.97:0/2158516402 >> 172.21.15.97:0/2158516402 conn(0x7f00b0071b90 msgr2=0x7f00b0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.060+0000 7f00b7d94700 1 -- 172.21.15.97:0/2158516402 shutdown_connections 2023-11-24T06:10:13.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.060+0000 7f00b7d94700 1 -- 172.21.15.97:0/2158516402 wait complete. 2023-11-24T06:10:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.061+0000 7f00b7d94700 1 Processor -- start 2023-11-24T06:10:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.061+0000 7f00b7d94700 1 -- start start 2023-11-24T06:10:13.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b7d94700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b006c540 0x7f00b00cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b7d94700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b00adf00 0x7f00b013fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b7d94700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b01446a0 0x7f00b0140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b7d94700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b01581d0 con 0x7f00b01446a0 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b7d94700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b01584b0 con 0x7f00b00adf00 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b7d94700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b0158790 con 0x7f00b006c540 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b01446a0 0x7f00b0140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b01446a0 0x7f00b0140240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47522/0 (socket says 172.21.15.97:47522) 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 -- 172.21.15.97:0/133990163 learned_addr learned my addr 172.21.15.97:0/133990163 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 -- 172.21.15.97:0/133990163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b006c540 msgr2=0x7f00b00cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b006c540 0x7f00b00cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 -- 172.21.15.97:0/133990163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b00adf00 msgr2=0x7f00b013fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b00adf00 0x7f00b013fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 -- 172.21.15.97:0/133990163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a8009a70 con 0x7f00b01446a0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b6331700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b01446a0 0x7f00b0140240 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f00a8014040 tx=0x7f00a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00a67fc700 1 -- 172.21.15.97:0/133990163 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a801b810 con 0x7f00b01446a0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00a67fc700 1 -- 172.21.15.97:0/133990163 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a8020070 con 0x7f00b01446a0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b0158a70 con 0x7f00b01446a0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00a67fc700 1 -- 172.21.15.97:0/133990163 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a801bbd0 con 0x7f00b01446a0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.062+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b0159030 con 0x7f00b01446a0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.063+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b00753f0 con 0x7f00b01446a0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.064+0000 7f00a67fc700 1 -- 172.21.15.97:0/133990163 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f00a801bd30 con 0x7f00b01446a0 2023-11-24T06:10:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.064+0000 7f00a67fc700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f009406af60 0x7f009406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.064+0000 7f00a67fc700 1 -- 172.21.15.97:0/133990163 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00a8022070 con 0x7f00b01446a0 2023-11-24T06:10:13.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.064+0000 7f00b5b30700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f009406af60 0x7f009406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.065+0000 7f00b5b30700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f009406af60 0x7f009406d420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f00a0001610 tx=0x7f00a00079f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.069+0000 7f00a67fc700 1 -- 172.21.15.97:0/133990163 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00a805aa70 con 0x7f00b01446a0 2023-11-24T06:10:13.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.263+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00b0138ce0 con 0x7f00b01446a0 2023-11-24T06:10:13.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.265+0000 7f00a67fc700 1 -- 172.21.15.97:0/133990163 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f00a805a600 con 0x7f00b01446a0 2023-11-24T06:10:13.269 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:13.269 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:57.042131+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:09:53.319367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:09:54.404860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:13.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f009406af60 msgr2=0x7f009406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f009406af60 0x7f009406d420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f00a0001610 tx=0x7f00a00079f0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b01446a0 msgr2=0x7f00b0140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b01446a0 0x7f00b0140240 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f00a8014040 tx=0x7f00a8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 shutdown_connections 2023-11-24T06:10:13.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00b01446a0 0x7f00b0140240 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f009406af60 0x7f009406d420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00b006c540 0x7f00b00cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 --2- 172.21.15.97:0/133990163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b00adf00 0x7f00b013fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 >> 172.21.15.97:0/133990163 conn(0x7f00b0071b90 msgr2=0x7f00b00b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:13.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 shutdown_connections 2023-11-24T06:10:13.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.267+0000 7f00b7d94700 1 -- 172.21.15.97:0/133990163 wait complete. 2023-11-24T06:10:13.283 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:10:13.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.400+0000 7f780d51b700 1 Processor -- start 2023-11-24T06:10:13.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.400+0000 7f780d51b700 1 -- start start 2023-11-24T06:10:13.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f780d51b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f780815e1e0 0x7f78080d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f780d51b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f78080d3a00 0x7f78080d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f780d51b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78080d0db0 0x7f780806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f780d51b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78080d3150 con 0x7f780815e1e0 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f780d51b700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78080d1240 con 0x7f78080d3a00 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f780d51b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780815e5c0 con 0x7f78080d0db0 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f780815e1e0 0x7f78080d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f780815e1e0 0x7f78080d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47526/0 (socket says 172.21.15.97:47526) 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 -- 172.21.15.97:0/1385947596 learned_addr learned my addr 172.21.15.97:0/1385947596 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f78077fe700 1 --2- 172.21.15.97:0/1385947596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78080d0db0 0x7f780806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f78067fc700 1 --2- 172.21.15.97:0/1385947596 >> v2:172.21.15.183:3300/0 conn(0x7f78080d3a00 0x7f78080d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 -- 172.21.15.97:0/1385947596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78080d0db0 msgr2=0x7f780806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 --2- 172.21.15.97:0/1385947596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78080d0db0 0x7f780806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 -- 172.21.15.97:0/1385947596 >> v2:172.21.15.183:3300/0 conn(0x7f78080d3a00 msgr2=0x7f78080d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 --2- 172.21.15.97:0/1385947596 >> v2:172.21.15.183:3300/0 conn(0x7f78080d3a00 0x7f78080d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 -- 172.21.15.97:0/1385947596 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78080d20b0 con 0x7f780815e1e0 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.401+0000 7f7806ffd700 1 --2- 172.21.15.97:0/1385947596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f780815e1e0 0x7f78080d34c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7808161ba0 tx=0x7f77f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602c9b24a04a30f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f7805ffb700 1 -- 172.21.15.97:0/1385947596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f800b860 con 0x7f780815e1e0 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f7805ffb700 1 -- 172.21.15.97:0/1385947596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f800b9c0 con 0x7f780815e1e0 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f7805ffb700 1 -- 172.21.15.97:0/1385947596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f800bd50 con 0x7f780815e1e0 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 -- 172.21.15.97:0/1385947596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f780815e1e0 msgr2=0x7f78080d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 --2- 172.21.15.97:0/1385947596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f780815e1e0 0x7f78080d34c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7808161ba0 tx=0x7f77f8009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:13.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 -- 172.21.15.97:0/1385947596 shutdown_connections 2023-11-24T06:10:13.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 --2- 172.21.15.97:0/1385947596 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78080d0db0 0x7f780806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 --2- 172.21.15.97:0/1385947596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f780815e1e0 0x7f78080d34c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 --2- 172.21.15.97:0/1385947596 >> v2:172.21.15.183:3300/0 conn(0x7f78080d3a00 0x7f78080d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 -- 172.21.15.97:0/1385947596 >> 172.21.15.97:0/1385947596 conn(0x7f78080b5b50 msgr2=0x7f78080b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:13.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 -- 172.21.15.97:0/1385947596 shutdown_connections 2023-11-24T06:10:13.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.402+0000 7f780d51b700 1 -- 172.21.15.97:0/1385947596 wait complete. 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.403+0000 7f780d51b700 1 Processor -- start 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.403+0000 7f780d51b700 1 -- start start 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f780d51b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78080d0db0 0x7f78081f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f780d51b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78080d3a00 0x7f78081562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f780d51b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f780815e1e0 0x7f7808142f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f780d51b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78080cde80 con 0x7f78080d0db0 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f780d51b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78080cb870 con 0x7f78080d3a00 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f780d51b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78081567e0 con 0x7f780815e1e0 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78080d0db0 0x7f78081f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78080d0db0 0x7f78081f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47532/0 (socket says 172.21.15.97:47532) 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f7806ffd700 1 -- 172.21.15.97:0/140459228 learned_addr learned my addr 172.21.15.97:0/140459228 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f78077fe700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f780815e1e0 0x7f7808142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.404+0000 7f78067fc700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78080d3a00 0x7f78081562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f7806ffd700 1 -- 172.21.15.97:0/140459228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f780815e1e0 msgr2=0x7f7808142f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f7806ffd700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f780815e1e0 0x7f7808142f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f7806ffd700 1 -- 172.21.15.97:0/140459228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78080d3a00 msgr2=0x7f78081562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f7806ffd700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78080d3a00 0x7f78081562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f7806ffd700 1 -- 172.21.15.97:0/140459228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f8009550 con 0x7f78080d0db0 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f7806ffd700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78080d0db0 0x7f78081f3420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009130 tx=0x7f77f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f77ef7fe700 1 -- 172.21.15.97:0/140459228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f80044d0 con 0x7f78080d0db0 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f77ef7fe700 1 -- 172.21.15.97:0/140459228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f801e070 con 0x7f78080d0db0 2023-11-24T06:10:13.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f77ef7fe700 1 -- 172.21.15.97:0/140459228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f80038e0 con 0x7f78080d0db0 2023-11-24T06:10:13.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78081599c0 con 0x7f78080d0db0 2023-11-24T06:10:13.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.405+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7808159f80 con 0x7f78080d0db0 2023-11-24T06:10:13.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.406+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780815a5d0 con 0x7f78080d0db0 2023-11-24T06:10:13.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.406+0000 7f77ef7fe700 1 -- 172.21.15.97:0/140459228 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f77f8003a40 con 0x7f78080d0db0 2023-11-24T06:10:13.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.406+0000 7f77ef7fe700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77f406af60 0x7f77f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.407+0000 7f77ef7fe700 1 -- 172.21.15.97:0/140459228 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77f8020070 con 0x7f78080d0db0 2023-11-24T06:10:13.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.407+0000 7f78067fc700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77f406af60 0x7f77f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.407+0000 7f78067fc700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77f406af60 0x7f77f406d420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f78081601b0 tx=0x7f77fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.411+0000 7f77ef7fe700 1 -- 172.21.15.97:0/140459228 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77f8055bb0 con 0x7f78080d0db0 2023-11-24T06:10:13.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.644+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78080751e0 con 0x7f78080d0db0 2023-11-24T06:10:13.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.644+0000 7f77ef7fe700 1 -- 172.21.15.97:0/140459228 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f77f802a020 con 0x7f78080d0db0 2023-11-24T06:10:13.648 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:13.648 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1778230784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1778230784},{"type":"v1","addr":"172.21.15.97:6836","nonce":1778230784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2183416217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2183416217},{"type":"v1","addr":"172.21.15.183:6834","nonce":2183416217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3650123299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3650123299},{"type":"v1","addr":"172.21.15.183:6835","nonce":3650123299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:09:55.467969+0000","modified":"2023-11-24T06:09:58.451611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5070},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5070":{"gid":5070,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3767182596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3767182596},{"type":"v1","addr":"172.21.15.97:6837","nonce":3767182596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77f406af60 msgr2=0x7f77f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77f406af60 0x7f77f406d420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f78081601b0 tx=0x7f77fc008040 comp rx=0 tx=0).stop 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78080d0db0 msgr2=0x7f78081f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78080d0db0 0x7f78081f3420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009130 tx=0x7f77f8004750 comp rx=0 tx=0).stop 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 shutdown_connections 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77f406af60 0x7f77f406d420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f780815e1e0 0x7f7808142f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78080d0db0 0x7f78081f3420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 --2- 172.21.15.97:0/140459228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78080d3a00 0x7f78081562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 >> 172.21.15.97:0/140459228 conn(0x7f78080b5b50 msgr2=0x7f78080b7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:13.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 shutdown_connections 2023-11-24T06:10:13.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.646+0000 7f780d51b700 1 -- 172.21.15.97:0/140459228 wait complete. 2023-11-24T06:10:13.652 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2023-11-24T06:10:13.662 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:13.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.778+0000 7f42aefb5700 1 Processor -- start 2023-11-24T06:10:13.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.778+0000 7f42aefb5700 1 -- start start 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42aefb5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f42a806c710 0x7f42a80cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42aefb5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f42a80b49a0 0x7f42a80d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42aefb5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a80d09a0 con 0x7f42a8152cc0 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42aefb5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a815c4b0 con 0x7f42a806c710 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42aefb5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a815c790 con 0x7f42a80b49a0 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42a7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f42a806c710 0x7f42a80cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42acd51700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f42a8152cc0 0x7f42a806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:39992/0 (socket says 172.21.15.97:39992) 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42acd51700 1 -- 172.21.15.97:0/1229607403 learned_addr learned my addr 172.21.15.97:0/1229607403 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.779+0000 7f42ad552700 1 --2- 172.21.15.97:0/1229607403 >> v2:172.21.15.183:3300/0 conn(0x7f42a80b49a0 0x7f42a80d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a7fff700 1 -- 172.21.15.97:0/1229607403 >> v2:172.21.15.183:3300/0 conn(0x7f42a80b49a0 msgr2=0x7f42a80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a7fff700 1 --2- 172.21.15.97:0/1229607403 >> v2:172.21.15.183:3300/0 conn(0x7f42a80b49a0 0x7f42a80d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a7fff700 1 -- 172.21.15.97:0/1229607403 >> v1:172.21.15.97:6789/0 conn(0x7f42a8152cc0 legacy=0x7f42a806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a77fe700 1 -- 172.21.15.97:0/1229607403 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198167857 0 0) 0x7f42a80d09a0 con 0x7f42a8152cc0 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a7fff700 1 -- 172.21.15.97:0/1229607403 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a815ca70 con 0x7f42a806c710 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a7fff700 1 --2- 172.21.15.97:0/1229607403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a80cff20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42a80b6300 tx=0x7f4294009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81b023189f0c0d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a77fe700 1 -- 172.21.15.97:0/1229607403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294011830 con 0x7f42a806c710 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a77fe700 1 -- 172.21.15.97:0/1229607403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4294011990 con 0x7f42a806c710 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42a77fe700 1 -- 172.21.15.97:0/1229607403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294011d20 con 0x7f42a806c710 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42aefb5700 1 -- 172.21.15.97:0/1229607403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 msgr2=0x7f42a80cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.780+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1229607403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a80cff20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f42a80b6300 tx=0x7f4294009b20 comp rx=0 tx=0).stop 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.781+0000 7f42aefb5700 1 -- 172.21.15.97:0/1229607403 shutdown_connections 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.781+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1229607403 >> v2:172.21.15.183:3300/0 conn(0x7f42a80b49a0 0x7f42a80d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.781+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1229607403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a80cff20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.781+0000 7f42aefb5700 1 -- 172.21.15.97:0/1229607403 >> 172.21.15.97:0/1229607403 conn(0x7f42a8071b90 msgr2=0x7f42a8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:13.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.781+0000 7f42aefb5700 1 -- 172.21.15.97:0/1229607403 shutdown_connections 2023-11-24T06:10:13.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.781+0000 7f42aefb5700 1 -- 172.21.15.97:0/1229607403 wait complete. 2023-11-24T06:10:13.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.782+0000 7f42aefb5700 1 Processor -- start 2023-11-24T06:10:13.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.782+0000 7f42aefb5700 1 -- start start 2023-11-24T06:10:13.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42aefb5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a8153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42aefb5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42a80b49a0 0x7f42a81f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42aefb5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8152cc0 0x7f42a813d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42aefb5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a814b270 con 0x7f42a806c710 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42aefb5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a81f5da0 con 0x7f42a8152cc0 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42aefb5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a81480f0 con 0x7f42a80b49a0 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a8153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a8153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47544/0 (socket says 172.21.15.97:47544) 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 -- 172.21.15.97:0/1006077137 learned_addr learned my addr 172.21.15.97:0/1006077137 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42ad552700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8152cc0 0x7f42a813d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42a7fff700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42a80b49a0 0x7f42a81f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 -- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42a80b49a0 msgr2=0x7f42a81f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42a80b49a0 0x7f42a81f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 -- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8152cc0 msgr2=0x7f42a813d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8152cc0 0x7f42a813d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 -- 172.21.15.97:0/1006077137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42940094a0 con 0x7f42a806c710 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42ad552700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8152cc0 0x7f42a813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42acd51700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a8153db0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c580 tx=0x7f429c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42a57fa700 1 -- 172.21.15.97:0/1006077137 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c004c80 con 0x7f42a806c710 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42a57fa700 1 -- 172.21.15.97:0/1006077137 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c004e00 con 0x7f42a806c710 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a813d970 con 0x7f42a806c710 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42a57fa700 1 -- 172.21.15.97:0/1006077137 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c007310 con 0x7f42a806c710 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.783+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a8148340 con 0x7f42a806c710 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.784+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a80753f0 con 0x7f42a806c710 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.785+0000 7f42a57fa700 1 -- 172.21.15.97:0/1006077137 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f429c007470 con 0x7f42a806c710 2023-11-24T06:10:13.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.785+0000 7f42a57fa700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f428c06af60 0x7f428c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:13.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.785+0000 7f42a57fa700 1 -- 172.21.15.97:0/1006077137 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f429c089250 con 0x7f42a806c710 2023-11-24T06:10:13.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.785+0000 7f42a7fff700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f428c06af60 0x7f428c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:13.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.786+0000 7f42a7fff700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f428c06af60 0x7f428c06d420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f42940090b0 tx=0x7f4294008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:13.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:13.790+0000 7f42a57fa700 1 -- 172.21.15.97:0/1006077137 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f429c0543a0 con 0x7f42a806c710 2023-11-24T06:10:14.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.020+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42a815f9e0 con 0x7f42a806c710 2023-11-24T06:10:14.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.021+0000 7f42a57fa700 1 -- 172.21.15.97:0/1006077137 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f42a815f9e0 con 0x7f42a806c710 2023-11-24T06:10:14.025 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:14.025 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1778230784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1778230784},{"type":"v1","addr":"172.21.15.97:6836","nonce":1778230784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2183416217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2183416217},{"type":"v1","addr":"172.21.15.183:6834","nonce":2183416217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3650123299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3650123299},{"type":"v1","addr":"172.21.15.183:6835","nonce":3650123299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:09:55.467969+0000","modified":"2023-11-24T06:09:58.451611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5070},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5070":{"gid":5070,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/3767182596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3767182596},{"type":"v1","addr":"172.21.15.97:6837","nonce":3767182596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:10:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f428c06af60 msgr2=0x7f428c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f428c06af60 0x7f428c06d420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f42940090b0 tx=0x7f4294008030 comp rx=0 tx=0).stop 2023-11-24T06:10:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 msgr2=0x7f42a8153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a8153db0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c580 tx=0x7f429c00c940 comp rx=0 tx=0).stop 2023-11-24T06:10:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 shutdown_connections 2023-11-24T06:10:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8152cc0 0x7f42a813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a806c710 0x7f42a8153db0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f428c06af60 0x7f428c06d420 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 --2- 172.21.15.97:0/1006077137 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42a80b49a0 0x7f42a81f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 >> 172.21.15.97:0/1006077137 conn(0x7f42a8071b90 msgr2=0x7f42a80b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 shutdown_connections 2023-11-24T06:10:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.023+0000 7f42aefb5700 1 -- 172.21.15.97:0/1006077137 wait complete. 2023-11-24T06:10:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 18 2023-11-24T06:10:14.039 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:14.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.156+0000 7fbfa1922700 1 Processor -- start 2023-11-24T06:10:14.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.156+0000 7fbfa1922700 1 -- start start 2023-11-24T06:10:14.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.157+0000 7fbfa1922700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf9c15e1e0 0x7fbf9c0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.157+0000 7fbfa1922700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbf9c0d3a00 0x7fbf9c0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.157+0000 7fbfa1922700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c0d0db0 0x7fbf9c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.157+0000 7fbfa1922700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c0d3150 con 0x7fbf9c15e1e0 2023-11-24T06:10:14.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.157+0000 7fbfa1922700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9c0d1240 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.157+0000 7fbfa1922700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c15e5c0 con 0x7fbf9c0d0db0 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf9c15e1e0 0x7fbf9c0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf9c15e1e0 0x7fbf9c0d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47552/0 (socket says 172.21.15.97:47552) 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 -- 172.21.15.97:0/3336681633 learned_addr learned my addr 172.21.15.97:0/3336681633 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9a7fc700 1 --2- 172.21.15.97:0/3336681633 >> v2:172.21.15.183:3300/0 conn(0x7fbf9c0d3a00 0x7fbf9c0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9b7fe700 1 --2- 172.21.15.97:0/3336681633 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c0d0db0 0x7fbf9c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 -- 172.21.15.97:0/3336681633 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c0d0db0 msgr2=0x7fbf9c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 --2- 172.21.15.97:0/3336681633 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c0d0db0 0x7fbf9c06aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 -- 172.21.15.97:0/3336681633 >> v2:172.21.15.183:3300/0 conn(0x7fbf9c0d3a00 msgr2=0x7fbf9c0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 --2- 172.21.15.97:0/3336681633 >> v2:172.21.15.183:3300/0 conn(0x7fbf9c0d3a00 0x7fbf9c0d0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 -- 172.21.15.97:0/3336681633 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9c0d20b0 con 0x7fbf9c15e1e0 2023-11-24T06:10:14.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf9affd700 1 --2- 172.21.15.97:0/3336681633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c15e1e0 0x7fbf9c0d34c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c161ba0 tx=0x7fbf90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f81daaf4e72d044 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf99ffb700 1 -- 172.21.15.97:0/3336681633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000b860 con 0x7fbf9c15e1e0 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf99ffb700 1 -- 172.21.15.97:0/3336681633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9000b9c0 con 0x7fbf9c15e1e0 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.158+0000 7fbf99ffb700 1 -- 172.21.15.97:0/3336681633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000bd50 con 0x7fbf9c15e1e0 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 -- 172.21.15.97:0/3336681633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c15e1e0 msgr2=0x7fbf9c0d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 --2- 172.21.15.97:0/3336681633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c15e1e0 0x7fbf9c0d34c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c161ba0 tx=0x7fbf90009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 -- 172.21.15.97:0/3336681633 shutdown_connections 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 --2- 172.21.15.97:0/3336681633 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c0d0db0 0x7fbf9c06aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 --2- 172.21.15.97:0/3336681633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c15e1e0 0x7fbf9c0d34c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 --2- 172.21.15.97:0/3336681633 >> v2:172.21.15.183:3300/0 conn(0x7fbf9c0d3a00 0x7fbf9c0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 -- 172.21.15.97:0/3336681633 >> 172.21.15.97:0/3336681633 conn(0x7fbf9c0b5b50 msgr2=0x7fbf9c0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:14.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 -- 172.21.15.97:0/3336681633 shutdown_connections 2023-11-24T06:10:14.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.159+0000 7fbfa1922700 1 -- 172.21.15.97:0/3336681633 wait complete. 2023-11-24T06:10:14.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.160+0000 7fbfa1922700 1 Processor -- start 2023-11-24T06:10:14.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.161+0000 7fbfa1922700 1 -- start start 2023-11-24T06:10:14.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.161+0000 7fbfa1922700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9c0d0db0 0x7fbf9c0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.161+0000 7fbfa1922700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0d3a00 0x7fbf9c15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.161+0000 7fbfa1922700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c15e1e0 0x7fbf9c14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.161+0000 7fbfa1922700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1e8a60 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.161+0000 7fbfa1922700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1e6ca0 con 0x7fbf9c0d0db0 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.161+0000 7fbfa1922700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c1e6f80 con 0x7fbf9c15e1e0 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.161+0000 7fbf9a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0d3a00 0x7fbf9c15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0d3a00 0x7fbf9c15f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47560/0 (socket says 172.21.15.97:47560) 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9a7fc700 1 -- 172.21.15.97:0/4030603216 learned_addr learned my addr 172.21.15.97:0/4030603216 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9affd700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9c0d0db0 0x7fbf9c0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9b7fe700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c15e1e0 0x7fbf9c14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9a7fc700 1 -- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c15e1e0 msgr2=0x7fbf9c14e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9a7fc700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c15e1e0 0x7fbf9c14e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9a7fc700 1 -- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9c0d0db0 msgr2=0x7fbf9c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9a7fc700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9c0d0db0 0x7fbf9c0b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9a7fc700 1 -- 172.21.15.97:0/4030603216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf90009550 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9affd700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9c0d0db0 0x7fbf9c0b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf9a7fc700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0d3a00 0x7fbf9c15f8a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00c7f0 tx=0x7fbf8c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf83fff700 1 -- 172.21.15.97:0/4030603216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c011800 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf83fff700 1 -- 172.21.15.97:0/4030603216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c011960 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c1e7260 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbf83fff700 1 -- 172.21.15.97:0/4030603216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c011cf0 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.162+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c1e7820 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.163+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9c0751e0 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.163+0000 7fbf83fff700 1 -- 172.21.15.97:0/4030603216 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fbf8c018360 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.164+0000 7fbf83fff700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf8806af90 0x7fbf8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.164+0000 7fbf9affd700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf8806af90 0x7fbf8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.164+0000 7fbf83fff700 1 -- 172.21.15.97:0/4030603216 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf8c089650 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.164+0000 7fbf9affd700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf8806af90 0x7fbf8806d450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf90014040 tx=0x7fbf90008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.168+0000 7fbf83fff700 1 -- 172.21.15.97:0/4030603216 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf8c054570 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.360+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf9c0b5f30 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.361+0000 7fbf83fff700 1 -- 172.21.15.97:0/4030603216 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16581 (secure 0 0 0) 0x7fbf8c057b90 con 0x7fbf9c0d3a00 2023-11-24T06:10:14.365 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:14.365 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":33,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:09:57.042131+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:09:53.319367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:09:54.404860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:14.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf8806af90 msgr2=0x7fbf8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf8806af90 0x7fbf8806d450 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbf90014040 tx=0x7fbf90008030 comp rx=0 tx=0).stop 2023-11-24T06:10:14.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0d3a00 msgr2=0x7fbf9c15f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0d3a00 0x7fbf9c15f8a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00c7f0 tx=0x7fbf8c00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 shutdown_connections 2023-11-24T06:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf8806af90 0x7fbf8806d450 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf9c15e1e0 0x7fbf9c14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf9c0d0db0 0x7fbf9c0b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 --2- 172.21.15.97:0/4030603216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf9c0d3a00 0x7fbf9c15f8a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 >> 172.21.15.97:0/4030603216 conn(0x7fbf9c0b5b50 msgr2=0x7fbf9c15d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 shutdown_connections 2023-11-24T06:10:14.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.363+0000 7fbfa1922700 1 -- 172.21.15.97:0/4030603216 wait complete. 2023-11-24T06:10:14.379 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:10:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.497+0000 7f92d39e8700 1 Processor -- start 2023-11-24T06:10:14.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.497+0000 7f92d39e8700 1 -- start start 2023-11-24T06:10:14.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d39e8700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f92cc15bdd0 0x7f92cc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d39e8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92cc0b3370 0x7f92cc0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d39e8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc0d1c70 0x7f92cc0d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d39e8700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0b2f50 con 0x7f92cc0b3370 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d39e8700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0d2660 con 0x7f92cc15bdd0 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d39e8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0b2030 con 0x7f92cc0d1c70 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92cc0b3370 0x7f92cc0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92cc0b3370 0x7f92cc0b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47566/0 (socket says 172.21.15.97:47566) 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 -- 172.21.15.97:0/4133458440 learned_addr learned my addr 172.21.15.97:0/4133458440 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 -- 172.21.15.97:0/4133458440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc0d1c70 msgr2=0x7f92cc0d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 --2- 172.21.15.97:0/4133458440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc0d1c70 0x7f92cc0d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d1784700 1 --2- 172.21.15.97:0/4133458440 >> v2:172.21.15.183:3300/0 conn(0x7f92cc15bdd0 0x7f92cc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 -- 172.21.15.97:0/4133458440 >> v2:172.21.15.183:3300/0 conn(0x7f92cc15bdd0 msgr2=0x7f92cc1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 --2- 172.21.15.97:0/4133458440 >> v2:172.21.15.183:3300/0 conn(0x7f92cc15bdd0 0x7f92cc1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 -- 172.21.15.97:0/4133458440 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0b22d0 con 0x7f92cc0b3370 2023-11-24T06:10:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92d0f83700 1 --2- 172.21.15.97:0/4133458440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0b3370 0x7f92cc0b1af0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f92cc06ec10 tx=0x7f92c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1412ec637ca2df4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.498+0000 7f92c3fff700 1 -- 172.21.15.97:0/4133458440 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c800b870 con 0x7f92cc0b3370 2023-11-24T06:10:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92c3fff700 1 -- 172.21.15.97:0/4133458440 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c800b9d0 con 0x7f92cc0b3370 2023-11-24T06:10:14.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92c3fff700 1 -- 172.21.15.97:0/4133458440 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c800bd60 con 0x7f92cc0b3370 2023-11-24T06:10:14.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 -- 172.21.15.97:0/4133458440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0b3370 msgr2=0x7f92cc0b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 --2- 172.21.15.97:0/4133458440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0b3370 0x7f92cc0b1af0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f92cc06ec10 tx=0x7f92c8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 -- 172.21.15.97:0/4133458440 shutdown_connections 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 --2- 172.21.15.97:0/4133458440 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc0d1c70 0x7f92cc0d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 --2- 172.21.15.97:0/4133458440 >> v2:172.21.15.183:3300/0 conn(0x7f92cc15bdd0 0x7f92cc1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 --2- 172.21.15.97:0/4133458440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0b3370 0x7f92cc0b1af0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 -- 172.21.15.97:0/4133458440 >> 172.21.15.97:0/4133458440 conn(0x7f92cc0731a0 msgr2=0x7f92cc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 -- 172.21.15.97:0/4133458440 shutdown_connections 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.499+0000 7f92d39e8700 1 -- 172.21.15.97:0/4133458440 wait complete. 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.500+0000 7f92d39e8700 1 Processor -- start 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.500+0000 7f92d39e8700 1 -- start start 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d39e8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92cc0b3370 0x7f92cc0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d39e8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0d1c70 0x7f92cc0b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d39e8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc15bdd0 0x7f92cc0b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d39e8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0cf600 con 0x7f92cc0d1c70 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d39e8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0cc720 con 0x7f92cc0b3370 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d39e8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0cca00 con 0x7f92cc15bdd0 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d0f83700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0d1c70 0x7f92cc0b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d0f83700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0d1c70 0x7f92cc0b67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47574/0 (socket says 172.21.15.97:47574) 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d0f83700 1 -- 172.21.15.97:0/2724545912 learned_addr learned my addr 172.21.15.97:0/2724545912 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d1784700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92cc0b3370 0x7f92cc0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d1f85700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc15bdd0 0x7f92cc0b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d0f83700 1 -- 172.21.15.97:0/2724545912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc15bdd0 msgr2=0x7f92cc0b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d0f83700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc15bdd0 0x7f92cc0b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d0f83700 1 -- 172.21.15.97:0/2724545912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92cc0b3370 msgr2=0x7f92cc0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d0f83700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92cc0b3370 0x7f92cc0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.501+0000 7f92d0f83700 1 -- 172.21.15.97:0/2724545912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c8009550 con 0x7f92cc0d1c70 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.502+0000 7f92d0f83700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0d1c70 0x7f92cc0b67d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f92c8013000 tx=0x7f92c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.502+0000 7f92c1ffb700 1 -- 172.21.15.97:0/2724545912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004310 con 0x7f92cc0d1c70 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.502+0000 7f92c1ffb700 1 -- 172.21.15.97:0/2724545912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c801f070 con 0x7f92cc0d1c70 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.502+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc0ccce0 con 0x7f92cc0d1c70 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.502+0000 7f92c1ffb700 1 -- 172.21.15.97:0/2724545912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c80036a0 con 0x7f92cc0d1c70 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.502+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc1592c0 con 0x7f92cc0d1c70 2023-11-24T06:10:14.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.503+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc0cf450 con 0x7f92cc0d1c70 2023-11-24T06:10:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.503+0000 7f92c1ffb700 1 -- 172.21.15.97:0/2724545912 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f92c8021030 con 0x7f92cc0d1c70 2023-11-24T06:10:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.504+0000 7f92c1ffb700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f92b006af60 0x7f92b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.504+0000 7f92d1784700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f92b006af60 0x7f92b006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.504+0000 7f92c1ffb700 1 -- 172.21.15.97:0/2724545912 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92c808a8e0 con 0x7f92cc0d1c70 2023-11-24T06:10:14.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.504+0000 7f92d1784700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f92b006af60 0x7f92b006d420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f92bc006fd0 tx=0x7f92bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.508+0000 7f92c1ffb700 1 -- 172.21.15.97:0/2724545912 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92c8055a30 con 0x7f92cc0d1c70 2023-11-24T06:10:14.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.742+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f92cc0cf2c0 con 0x7f92cc0d1c70 2023-11-24T06:10:14.746 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-11-24T06:10:14.742+0000 7fbb14a83700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:10:14.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.744+0000 7f92c1ffb700 1 -- 172.21.15.97:0/2724545912 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f92c8059050 con 0x7f92cc0d1c70 2023-11-24T06:10:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f92b006af60 msgr2=0x7f92b006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f92b006af60 0x7f92b006d420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f92bc006fd0 tx=0x7f92bc008040 comp rx=0 tx=0).stop 2023-11-24T06:10:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0d1c70 msgr2=0x7f92cc0b67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0d1c70 0x7f92cc0b67d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f92c8013000 tx=0x7f92c8004750 comp rx=0 tx=0).stop 2023-11-24T06:10:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 shutdown_connections 2023-11-24T06:10:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f92b006af60 0x7f92b006d420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92cc15bdd0 0x7f92cc0b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f92cc0b3370 0x7f92cc0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 --2- 172.21.15.97:0/2724545912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92cc0d1c70 0x7f92cc0b67d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 >> 172.21.15.97:0/2724545912 conn(0x7f92cc0731a0 msgr2=0x7f92cc06f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 shutdown_connections 2023-11-24T06:10:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.746+0000 7f92d39e8700 1 -- 172.21.15.97:0/2724545912 wait complete. 2023-11-24T06:10:14.751 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:10:14.752 INFO:tasks.ceph.mds.a.smithi097.stderr: -6479> 2023-11-24T06:09:48.145+0000 7f7c7168d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-24T06:10:14.753 INFO:tasks.ceph.mds.a.smithi097.stderr: -5270> 2023-11-24T06:10:00.008+0000 7f7c6b681700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:10:14.763 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:10:14.784 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-11-24T06:10:14.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.882+0000 7f0915076700 1 Processor -- start 2023-11-24T06:10:14.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.882+0000 7f0915076700 1 -- start start 2023-11-24T06:10:14.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f0915076700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f091015e090 0x7f09100d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f0915076700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100d2a60 con 0x7f09100d24e0 2023-11-24T06:10:14.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f0915076700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100d2d40 con 0x7f0910071d20 2023-11-24T06:10:14.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f0915076700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100d3020 con 0x7f091015e090 2023-11-24T06:10:14.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f091015e090 0x7f09100d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090e59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f09100d24e0 0x7f0910071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40046/0 (socket says 172.21.15.97:40046) 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090e59c700 1 -- 172.21.15.97:0/321004126 learned_addr learned my addr 172.21.15.97:0/321004126 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090ed9d700 1 -- 172.21.15.97:0/321004126 >> v1:172.21.15.183:6789/0 conn(0x7f0910071d20 legacy=0x7f09100b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090ed9d700 1 -- 172.21.15.97:0/321004126 >> v1:172.21.15.97:6789/0 conn(0x7f09100d24e0 legacy=0x7f0910071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090ed9d700 1 -- 172.21.15.97:0/321004126 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100d3300 con 0x7f091015e090 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090dd9b700 1 -- 172.21.15.97:0/321004126 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405445781 0 0) 0x7f09100d2a60 con 0x7f09100d24e0 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090ed9d700 1 --2- 172.21.15.97:0/321004126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091015e090 0x7f09100d1fa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0904009840 tx=0x7f0904009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf9cea36af5d14c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.883+0000 7f090dd9b700 1 -- 172.21.15.97:0/321004126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904011830 con 0x7f091015e090 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f090dd9b700 1 -- 172.21.15.97:0/321004126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904011990 con 0x7f091015e090 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f090dd9b700 1 -- 172.21.15.97:0/321004126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904011d20 con 0x7f091015e090 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f0915076700 1 -- 172.21.15.97:0/321004126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091015e090 msgr2=0x7f09100d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f0915076700 1 --2- 172.21.15.97:0/321004126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091015e090 0x7f09100d1fa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0904009840 tx=0x7f0904009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:14.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f0915076700 1 -- 172.21.15.97:0/321004126 shutdown_connections 2023-11-24T06:10:14.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f0915076700 1 --2- 172.21.15.97:0/321004126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091015e090 0x7f09100d1fa0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f0915076700 1 -- 172.21.15.97:0/321004126 >> 172.21.15.97:0/321004126 conn(0x7f091006b410 msgr2=0x7f091006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:14.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f0915076700 1 -- 172.21.15.97:0/321004126 shutdown_connections 2023-11-24T06:10:14.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.884+0000 7f0915076700 1 -- 172.21.15.97:0/321004126 wait complete. 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.885+0000 7f0915076700 1 Processor -- start 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f0915076700 1 -- start start 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f0915076700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0910071d20 0x7f091014bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f0915076700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09100d24e0 0x7f09100cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f0915076700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091015e090 0x7f09100ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f0915076700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091014c690 con 0x7f09100d24e0 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f0915076700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0910160e00 con 0x7f091015e090 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f0915076700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091014c930 con 0x7f0910071d20 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f090e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09100d24e0 0x7f09100cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f090e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09100d24e0 0x7f09100cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47600/0 (socket says 172.21.15.97:47600) 2023-11-24T06:10:14.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.886+0000 7f090e59c700 1 -- 172.21.15.97:0/3737709254 learned_addr learned my addr 172.21.15.97:0/3737709254 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090ed9d700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0910071d20 0x7f091014bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090f59e700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091015e090 0x7f09100ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090e59c700 1 -- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0910071d20 msgr2=0x7f091014bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090e59c700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0910071d20 0x7f091014bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090e59c700 1 -- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091015e090 msgr2=0x7f09100ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090e59c700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091015e090 0x7f09100ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090e59c700 1 -- 172.21.15.97:0/3737709254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09040094a0 con 0x7f09100d24e0 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090ed9d700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0910071d20 0x7f091014bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090f59e700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091015e090 0x7f09100ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f090e59c700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09100d24e0 0x7f09100cdd40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0900001eb0 tx=0x7f090000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f08f77fe700 1 -- 172.21.15.97:0/3737709254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900004c80 con 0x7f09100d24e0 2023-11-24T06:10:14.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f08f77fe700 1 -- 172.21.15.97:0/3737709254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0900004de0 con 0x7f09100d24e0 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09100d3c80 con 0x7f09100d24e0 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f08f77fe700 1 -- 172.21.15.97:0/3737709254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900007310 con 0x7f09100d24e0 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.887+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091014cef0 con 0x7f09100d24e0 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.888+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091014d260 con 0x7f09100d24e0 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.888+0000 7f08f77fe700 1 -- 172.21.15.97:0/3737709254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0900007470 con 0x7f09100d24e0 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.889+0000 7f08f77fe700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f08fc06af60 0x7f08fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.889+0000 7f08f77fe700 1 -- 172.21.15.97:0/3737709254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f09000893b0 con 0x7f09100d24e0 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.889+0000 7f090ed9d700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f08fc06af60 0x7f08fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:14.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.889+0000 7f090ed9d700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f08fc06af60 0x7f08fc06d420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0904000c00 tx=0x7f090401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:14.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:14.893+0000 7f08f77fe700 1 -- 172.21.15.97:0/3737709254 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0900054410 con 0x7f09100d24e0 2023-11-24T06:10:15.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.124+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0910068fb0 con 0x7f09100d24e0 2023-11-24T06:10:15.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.745+0000 7f08f77fe700 1 -- 172.21.15.97:0/3737709254 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0900057a30 con 0x7f09100d24e0 2023-11-24T06:10:15.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.748+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f08fc06af60 msgr2=0x7f08fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.748+0000 7f0915076700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f08fc06af60 0x7f08fc06d420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0904000c00 tx=0x7f090401c040 comp rx=0 tx=0).stop 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.748+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09100d24e0 msgr2=0x7f09100cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.748+0000 7f0915076700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09100d24e0 0x7f09100cdd40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0900001eb0 tx=0x7f090000c8b0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.749+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 shutdown_connections 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.749+0000 7f0915076700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f091015e090 0x7f09100ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.749+0000 7f0915076700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f08fc06af60 0x7f08fc06d420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.749+0000 7f0915076700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0910071d20 0x7f091014bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.749+0000 7f0915076700 1 --2- 172.21.15.97:0/3737709254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09100d24e0 0x7f09100cdd40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.749+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 >> 172.21.15.97:0/3737709254 conn(0x7f091006b410 msgr2=0x7f09100b8870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:15.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.749+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 shutdown_connections 2023-11-24T06:10:15.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.749+0000 7f0915076700 1 -- 172.21.15.97:0/3737709254 wait complete. 2023-11-24T06:10:15.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:10:15.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.935+0000 7fd94b67f700 1 Processor -- start 2023-11-24T06:10:15.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.935+0000 7fd94b67f700 1 -- start start 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.936+0000 7fd94b67f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 0x7fd94406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.936+0000 7fd94b67f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9440b5e00 con 0x7fd9440b01e0 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.936+0000 7fd94b67f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9440b05c0 con 0x7fd9440b59c0 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.936+0000 7fd94b67f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441470a0 con 0x7fd9440746a0 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.936+0000 7fd94941b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd9440b01e0 0x7fd944073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40060/0 (socket says 172.21.15.97:40060) 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.936+0000 7fd94941b700 1 -- 172.21.15.97:0/2445912348 learned_addr learned my addr 172.21.15.97:0/2445912348 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4065865878 0 0) 0x7fd9440b5e00 con 0x7fd9440b01e0 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd948c1a700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 0x7fd94406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9300035b0 con 0x7fd9440b01e0 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017876799 0 0) 0x7fd9300035b0 con 0x7fd9440b01e0 2023-11-24T06:10:15.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 msgr2=0x7fd94406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 0x7fd94406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 >> v1:172.21.15.183:6789/0 conn(0x7fd9440b59c0 legacy=0x7fd94406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9440b4250 con 0x7fd9440b01e0 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fd934001670 con 0x7fd9440b01e0 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 >> v1:172.21.15.97:6789/0 conn(0x7fd9440b01e0 legacy=0x7fd944073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd930003a30 0x7fd930003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9300043e0 0x7fd9300048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 0x7fd9300051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441470a0 con 0x7fd930003a30 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93000baf0 con 0x7fd9300043e0 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93000bcb0 con 0x7fd9440746a0 2023-11-24T06:10:15.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd94941b700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd930003a30 0x7fd930003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd948c1a700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9300043e0 0x7fd9300048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd949c1c700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 0x7fd9300051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd94941b700 1 -- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 msgr2=0x7fd9300051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd94941b700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 0x7fd9300051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd94941b700 1 -- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9300043e0 msgr2=0x7fd9300048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd94941b700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9300043e0 0x7fd9300048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd94941b700 1 -- 172.21.15.97:0/2445912348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9300035b0 con 0x7fd930003a30 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.937+0000 7fd949c1c700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd9440746a0 0x7fd9300051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd94941b700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd930003a30 0x7fd930003e10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c240 tx=0x7fd930005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52bce79c954e44b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400cde0 con 0x7fd930003a30 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd934007ba0 con 0x7fd930003a30 2023-11-24T06:10:15.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd93bfff700 1 -- 172.21.15.97:0/2445912348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934005b80 con 0x7fd930003a30 2023-11-24T06:10:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd94b67f700 1 -- 172.21.15.97:0/2445912348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd930003a30 msgr2=0x7fd930003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd94b67f700 1 --2- 172.21.15.97:0/2445912348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd930003a30 0x7fd930003e10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c240 tx=0x7fd930005880 comp rx=0 tx=0).stop 2023-11-24T06:10:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd949c1c700 1 -- 172.21.15.97:0/2445912348 reap_dead start 2023-11-24T06:10:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd94b67f700 1 -- 172.21.15.97:0/2445912348 shutdown_connections 2023-11-24T06:10:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd94b67f700 1 -- 172.21.15.97:0/2445912348 >> 172.21.15.97:0/2445912348 conn(0x7fd9440d0b40 msgr2=0x7fd9440d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd94b67f700 1 -- 172.21.15.97:0/2445912348 shutdown_connections 2023-11-24T06:10:15.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.938+0000 7fd94b67f700 1 -- 172.21.15.97:0/2445912348 wait complete. 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.940+0000 7fd94b67f700 1 Processor -- start 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.940+0000 7fd94b67f700 1 -- start start 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd94b67f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd930003a30 0x7fd944140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd94b67f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd94413b4a0 0x7fd94413d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd94b67f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9441f3840 0x7fd9440ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd94b67f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94413dbd0 con 0x7fd9441f3840 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd94b67f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94413de00 con 0x7fd930003a30 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd94b67f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94413e0e0 con 0x7fd94413b4a0 2023-11-24T06:10:15.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9441f3840 0x7fd9440ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9441f3840 0x7fd9440ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47624/0 (socket says 172.21.15.97:47624) 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 -- 172.21.15.97:0/2865031737 learned_addr learned my addr 172.21.15.97:0/2865031737 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd948c1a700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd94413b4a0 0x7fd94413d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd94941b700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd930003a30 0x7fd944140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 -- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd94413b4a0 msgr2=0x7fd94413d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd94413b4a0 0x7fd94413d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 -- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd930003a30 msgr2=0x7fd944140c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd930003a30 0x7fd944140c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 -- 172.21.15.97:0/2865031737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94413e3c0 con 0x7fd9441f3840 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd94941b700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd930003a30 0x7fd944140c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd948c1a700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd94413b4a0 0x7fd94413d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:15.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.941+0000 7fd949c1c700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9441f3840 0x7fd9440ae760 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd944074fe0 tx=0x7fd93c00c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.942+0000 7fd939ffb700 1 -- 172.21.15.97:0/2865031737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c003c10 con 0x7fd9441f3840 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.942+0000 7fd939ffb700 1 -- 172.21.15.97:0/2865031737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c003d70 con 0x7fd9441f3840 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.942+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94413bc90 con 0x7fd9441f3840 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.942+0000 7fd939ffb700 1 -- 172.21.15.97:0/2865031737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00c980 con 0x7fd9441f3840 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.942+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94413e6a0 con 0x7fd9441f3840 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.943+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94413c000 con 0x7fd9441f3840 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.943+0000 7fd939ffb700 1 -- 172.21.15.97:0/2865031737 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fd93c00cae0 con 0x7fd9441f3840 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.943+0000 7fd939ffb700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd92806af60 0x7fd92806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.943+0000 7fd939ffb700 1 -- 172.21.15.97:0/2865031737 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd93c089860 con 0x7fd9441f3840 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.943+0000 7fd94941b700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd92806af60 0x7fd92806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:15.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.944+0000 7fd94941b700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd92806af60 0x7fd92806d420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd934000c00 tx=0x7fd934007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:15.948+0000 7fd939ffb700 1 -- 172.21.15.97:0/2865031737 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd93c0548c0 con 0x7fd9441f3840 2023-11-24T06:10:16.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:16.138+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd944067f60 con 0x7fd9441f3840 2023-11-24T06:10:17.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.131+0000 7fd939ffb700 1 -- 172.21.15.97:0/2865031737 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd93c057ee0 con 0x7fd9441f3840 2023-11-24T06:10:17.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.135+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd92806af60 msgr2=0x7fd92806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.135+0000 7fd94b67f700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd92806af60 0x7fd92806d420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd934000c00 tx=0x7fd934007000 comp rx=0 tx=0).stop 2023-11-24T06:10:17.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.135+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9441f3840 msgr2=0x7fd9440ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.135+0000 7fd94b67f700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9441f3840 0x7fd9440ae760 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd944074fe0 tx=0x7fd93c00c0e0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.136+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 shutdown_connections 2023-11-24T06:10:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.136+0000 7fd94b67f700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9441f3840 0x7fd9440ae760 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.136+0000 7fd94b67f700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd930003a30 0x7fd944140c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.136+0000 7fd94b67f700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd92806af60 0x7fd92806d420 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd934000c00 tx=0x7fd934007000 comp rx=0 tx=0).stop 2023-11-24T06:10:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.136+0000 7fd94b67f700 1 --2- 172.21.15.97:0/2865031737 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd94413b4a0 0x7fd94413d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.136+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 >> 172.21.15.97:0/2865031737 conn(0x7fd9440d0b40 msgr2=0x7fd94406e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.136+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 shutdown_connections 2023-11-24T06:10:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.136+0000 7fd94b67f700 1 -- 172.21.15.97:0/2865031737 wait complete. 2023-11-24T06:10:17.143 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:10:17.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:10:17.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.281+0000 7f77f342d700 1 Processor -- start 2023-11-24T06:10:17.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.281+0000 7f77f342d700 1 -- start start 2023-11-24T06:10:17.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.282+0000 7f77f342d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec158800 0x7f77ec06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.282+0000 7f77f342d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77ec06e8b0 0x7f77ec0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.282+0000 7f77f342d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ec158be0 con 0x7f77ec0b4d20 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.282+0000 7f77f342d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ec0b51d0 con 0x7f77ec06e8b0 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.282+0000 7f77f342d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec06f9d0 con 0x7f77ec158800 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.282+0000 7f77f09c8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77ec06e8b0 0x7f77ec0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.282+0000 7f77f19ca700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f77ec0b4d20 0x7f77ec15e760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40082/0 (socket says 172.21.15.97:40082) 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.282+0000 7f77f09c8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f77ec06e8b0 0x7f77ec0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47630/0 (socket says 172.21.15.97:47630) 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77f19ca700 1 -- 172.21.15.97:0/3123376971 learned_addr learned my addr 172.21.15.97:0/3123376971 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77f11c9700 1 --2- 172.21.15.97:0/3123376971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec158800 0x7f77ec06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77f09c8700 1 -- 172.21.15.97:0/3123376971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec158800 msgr2=0x7f77ec06e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77f09c8700 1 --2- 172.21.15.97:0/3123376971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec158800 0x7f77ec06e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77f09c8700 1 -- 172.21.15.97:0/3123376971 >> v1:172.21.15.97:6789/0 conn(0x7f77ec0b4d20 legacy=0x7f77ec15e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77f09c8700 1 -- 172.21.15.97:0/3123376971 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec06edc0 con 0x7f77ec06e8b0 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77f11c9700 1 --2- 172.21.15.97:0/3123376971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec158800 0x7f77ec06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77f09c8700 1 --2- 172.21.15.97:0/3123376971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec0b47e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009840 tx=0x7f77e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846b86854ced44d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77e3fff700 1 -- 172.21.15.97:0/3123376971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8011830 con 0x7f77ec06e8b0 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77e3fff700 1 -- 172.21.15.97:0/3123376971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e8011990 con 0x7f77ec06e8b0 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.283+0000 7f77e3fff700 1 -- 172.21.15.97:0/3123376971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e8011d20 con 0x7f77ec06e8b0 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.284+0000 7f77f342d700 1 -- 172.21.15.97:0/3123376971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 msgr2=0x7f77ec0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.284+0000 7f77f342d700 1 --2- 172.21.15.97:0/3123376971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec0b47e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009840 tx=0x7f77e8009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.284+0000 7f77f342d700 1 -- 172.21.15.97:0/3123376971 shutdown_connections 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.284+0000 7f77f342d700 1 --2- 172.21.15.97:0/3123376971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec158800 0x7f77ec06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.284+0000 7f77f342d700 1 --2- 172.21.15.97:0/3123376971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec0b47e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.284+0000 7f77f342d700 1 -- 172.21.15.97:0/3123376971 >> 172.21.15.97:0/3123376971 conn(0x7f77ec0d0b40 msgr2=0x7f77ec0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:17.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.284+0000 7f77f342d700 1 -- 172.21.15.97:0/3123376971 shutdown_connections 2023-11-24T06:10:17.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.284+0000 7f77f342d700 1 -- 172.21.15.97:0/3123376971 wait complete. 2023-11-24T06:10:17.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.285+0000 7f77f342d700 1 Processor -- start 2023-11-24T06:10:17.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.285+0000 7f77f342d700 1 -- start start 2023-11-24T06:10:17.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f342d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f342d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec0b4d20 0x7f77ec0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f342d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77ec158800 0x7f77ec0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f342d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec14f690 con 0x7f77ec06e8b0 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f342d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0cb250 con 0x7f77ec158800 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f342d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ec0cdf60 con 0x7f77ec0b4d20 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47632/0 (socket says 172.21.15.97:47632) 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 -- 172.21.15.97:0/520090457 learned_addr learned my addr 172.21.15.97:0/520090457 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f09c8700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec0b4d20 0x7f77ec0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f19ca700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77ec158800 0x7f77ec0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 -- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec0b4d20 msgr2=0x7f77ec0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec0b4d20 0x7f77ec0ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 -- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77ec158800 msgr2=0x7f77ec0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77ec158800 0x7f77ec0cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 -- 172.21.15.97:0/520090457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e80094a0 con 0x7f77ec06e8b0 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f19ca700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77ec158800 0x7f77ec0cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f11c9700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec146550 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00c7f0 tx=0x7f77dc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77e1ffb700 1 -- 172.21.15.97:0/520090457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc011800 con 0x7f77ec06e8b0 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77e1ffb700 1 -- 172.21.15.97:0/520090457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77dc011960 con 0x7f77ec06e8b0 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77e1ffb700 1 -- 172.21.15.97:0/520090457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77dc011d10 con 0x7f77ec06e8b0 2023-11-24T06:10:17.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.286+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec0ce240 con 0x7f77ec06e8b0 2023-11-24T06:10:17.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.287+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ec0ce800 con 0x7f77ec06e8b0 2023-11-24T06:10:17.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.287+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ec067f60 con 0x7f77ec06e8b0 2023-11-24T06:10:17.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.288+0000 7f77e1ffb700 1 -- 172.21.15.97:0/520090457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f77dc018360 con 0x7f77ec06e8b0 2023-11-24T06:10:17.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.288+0000 7f77e1ffb700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77d006af60 0x7f77d006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:17.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.288+0000 7f77e1ffb700 1 -- 172.21.15.97:0/520090457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f77dc0895d0 con 0x7f77ec06e8b0 2023-11-24T06:10:17.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.288+0000 7f77f09c8700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77d006af60 0x7f77d006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:17.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.288+0000 7f77f09c8700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77d006af60 0x7f77d006d420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77ec06ecf0 tx=0x7f77e8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:17.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.292+0000 7f77e1ffb700 1 -- 172.21.15.97:0/520090457 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77dc0546e0 con 0x7f77ec06e8b0 2023-11-24T06:10:17.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:17.485+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77ec0cea20 con 0x7f77ec06e8b0 2023-11-24T06:10:18.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.138+0000 7f77e1ffb700 1 -- 172.21.15.97:0/520090457 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f77dc057d00 con 0x7f77ec06e8b0 2023-11-24T06:10:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.141+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77d006af60 msgr2=0x7f77d006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.141+0000 7f77f342d700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77d006af60 0x7f77d006d420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77ec06ecf0 tx=0x7f77e8008030 comp rx=0 tx=0).stop 2023-11-24T06:10:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.141+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 msgr2=0x7f77ec146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.141+0000 7f77f342d700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec146550 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00c7f0 tx=0x7f77dc00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.142+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 shutdown_connections 2023-11-24T06:10:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.142+0000 7f77f342d700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77ec158800 0x7f77ec0cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.142+0000 7f77f342d700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77ec06e8b0 0x7f77ec146550 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.142+0000 7f77f342d700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f77d006af60 0x7f77d006d420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.142+0000 7f77f342d700 1 --2- 172.21.15.97:0/520090457 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f77ec0b4d20 0x7f77ec0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.142+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 >> 172.21.15.97:0/520090457 conn(0x7f77ec0d0b40 msgr2=0x7f77ec15e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.142+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 shutdown_connections 2023-11-24T06:10:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.142+0000 7f77f342d700 1 -- 172.21.15.97:0/520090457 wait complete. 2023-11-24T06:10:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-11-24T06:10:18.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:10:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.280+0000 7f1ec6a09700 1 Processor -- start 2023-11-24T06:10:18.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.280+0000 7f1ec6a09700 1 -- start start 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.281+0000 7f1ec6a09700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ec0144760 0x7f1ec00b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.281+0000 7f1ec6a09700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec00b4300 con 0x7f1ec006c000 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.281+0000 7f1ec6a09700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec0144b40 con 0x7f1ec00b4cd0 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.281+0000 7f1ec6a09700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec015ec80 con 0x7f1ec0144760 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.281+0000 7f1ebffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ec0144760 0x7f1ec00b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.281+0000 7f1ec4fa6700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1ec006c000 0x7f1ec006c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40096/0 (socket says 172.21.15.97:40096) 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.281+0000 7f1ec4fa6700 1 -- 172.21.15.97:0/3501019277 learned_addr learned my addr 172.21.15.97:0/3501019277 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ebffff700 1 -- 172.21.15.97:0/3501019277 >> v1:172.21.15.183:6789/0 conn(0x7f1ec00b4cd0 legacy=0x7f1ec006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ebffff700 1 -- 172.21.15.97:0/3501019277 >> v1:172.21.15.97:6789/0 conn(0x7f1ec006c000 legacy=0x7f1ec006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ebffff700 1 -- 172.21.15.97:0/3501019277 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec006e6e0 con 0x7f1ec0144760 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ebeffd700 1 -- 172.21.15.97:0/3501019277 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011997331 0 0) 0x7f1ec00b4300 con 0x7f1ec006c000 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ebffff700 1 --2- 172.21.15.97:0/3501019277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec0144760 0x7f1ec00b4790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009840 tx=0x7f1eb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b20b4f3f44623f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ebeffd700 1 -- 172.21.15.97:0/3501019277 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0004230 con 0x7f1ec0144760 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ebeffd700 1 -- 172.21.15.97:0/3501019277 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0004390 con 0x7f1ec0144760 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ebeffd700 1 -- 172.21.15.97:0/3501019277 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00036a0 con 0x7f1ec0144760 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ec6a09700 1 -- 172.21.15.97:0/3501019277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec0144760 msgr2=0x7f1ec00b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ec6a09700 1 --2- 172.21.15.97:0/3501019277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec0144760 0x7f1ec00b4790 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009840 tx=0x7f1eb0009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.282+0000 7f1ec6a09700 1 -- 172.21.15.97:0/3501019277 shutdown_connections 2023-11-24T06:10:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.283+0000 7f1ec6a09700 1 --2- 172.21.15.97:0/3501019277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec0144760 0x7f1ec00b4790 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.283+0000 7f1ec6a09700 1 -- 172.21.15.97:0/3501019277 >> 172.21.15.97:0/3501019277 conn(0x7f1ec0071a70 msgr2=0x7f1ec0071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.283+0000 7f1ec6a09700 1 -- 172.21.15.97:0/3501019277 shutdown_connections 2023-11-24T06:10:18.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.283+0000 7f1ec6a09700 1 -- 172.21.15.97:0/3501019277 wait complete. 2023-11-24T06:10:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.284+0000 7f1ec6a09700 1 Processor -- start 2023-11-24T06:10:18.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.284+0000 7f1ec6a09700 1 -- start start 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec6a09700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec006c000 0x7f1ec01401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec6a09700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec00b4cd0 0x7f1ec00cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec6a09700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ec0144760 0x7f1ec015edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec6a09700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00ccbe0 con 0x7f1ec006c000 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec6a09700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01f1610 con 0x7f1ec00b4cd0 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec6a09700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0140710 con 0x7f1ec0144760 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec006c000 0x7f1ec01401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec006c000 0x7f1ec01401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47646/0 (socket says 172.21.15.97:47646) 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 -- 172.21.15.97:0/590505071 learned_addr learned my addr 172.21.15.97:0/590505071 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebf7fe700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec00b4cd0 0x7f1ec00cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec4fa6700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ec0144760 0x7f1ec015edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 -- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ec0144760 msgr2=0x7f1ec015edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ec0144760 0x7f1ec015edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 -- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec00b4cd0 msgr2=0x7f1ec00cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec00b4cd0 0x7f1ec00cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 -- 172.21.15.97:0/590505071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00094a0 con 0x7f1ec006c000 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebf7fe700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec00b4cd0 0x7f1ec00cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebffff700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec006c000 0x7f1ec01401d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1ec006cf40 tx=0x7f1eb00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebcff9700 1 -- 172.21.15.97:0/590505071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0011630 con 0x7f1ec006c000 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebcff9700 1 -- 172.21.15.97:0/590505071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0011790 con 0x7f1ec006c000 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec00cc340 con 0x7f1ec006c000 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ebcff9700 1 -- 172.21.15.97:0/590505071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0011b40 con 0x7f1ec006c000 2023-11-24T06:10:18.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.285+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec00cc900 con 0x7f1ec006c000 2023-11-24T06:10:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.286+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec0068fb0 con 0x7f1ec006c000 2023-11-24T06:10:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.287+0000 7f1ebcff9700 1 -- 172.21.15.97:0/590505071 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1eb0011ca0 con 0x7f1ec006c000 2023-11-24T06:10:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.287+0000 7f1ebcff9700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ea006af60 0x7f1ea006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.287+0000 7f1ebcff9700 1 -- 172.21.15.97:0/590505071 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1eb008aba0 con 0x7f1ec006c000 2023-11-24T06:10:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.287+0000 7f1ebf7fe700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ea006af60 0x7f1ea006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.287+0000 7f1ebf7fe700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ea006af60 0x7f1ea006d420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00b0b90 tx=0x7f1ea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.291+0000 7f1ebcff9700 1 -- 172.21.15.97:0/590505071 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1eb0055ff0 con 0x7f1ec006c000 2023-11-24T06:10:18.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.485+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1ec0068fb0 con 0x7f1ec006c000 2023-11-24T06:10:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.488+0000 7f1ebcff9700 1 -- 172.21.15.97:0/590505071 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1eb0059610 con 0x7f1ec006c000 2023-11-24T06:10:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ea006af60 msgr2=0x7f1ea006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ea006af60 0x7f1ea006d420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00b0b90 tx=0x7f1ea8009040 comp rx=0 tx=0).stop 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec006c000 msgr2=0x7f1ec01401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec006c000 0x7f1ec01401d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1ec006cf40 tx=0x7f1eb00047c0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 shutdown_connections 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1ea006af60 0x7f1ea006d420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ec0144760 0x7f1ec015edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ec006c000 0x7f1ec01401d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 --2- 172.21.15.97:0/590505071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ec00b4cd0 0x7f1ec00cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 >> 172.21.15.97:0/590505071 conn(0x7f1ec0071a70 msgr2=0x7f1ec00b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 shutdown_connections 2023-11-24T06:10:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.491+0000 7f1ec6a09700 1 -- 172.21.15.97:0/590505071 wait complete. 2023-11-24T06:10:18.506 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T06:10:18.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.629+0000 7ff083486700 1 Processor -- start 2023-11-24T06:10:18.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.629+0000 7ff083486700 1 -- start start 2023-11-24T06:10:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff083486700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff07c06c710 0x7ff07c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff083486700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff07c0b49a0 0x7ff07c0d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff083486700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07c0d09a0 con 0x7ff07c152cc0 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff083486700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07c15c4b0 con 0x7ff07c0b49a0 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff083486700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07c15c790 con 0x7ff07c06c710 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff081a23700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff07c0b49a0 0x7ff07c0d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff081a23700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff07c0b49a0 0x7ff07c0d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47658/0 (socket says 172.21.15.97:47658) 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff081a23700 1 -- 172.21.15.97:0/829718999 learned_addr learned my addr 172.21.15.97:0/829718999 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff080a21700 1 --2- 172.21.15.97:0/829718999 >> v2:172.21.15.183:3300/0 conn(0x7ff07c06c710 0x7ff07c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff081a23700 1 -- 172.21.15.97:0/829718999 >> v2:172.21.15.183:3300/0 conn(0x7ff07c06c710 msgr2=0x7ff07c0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff081a23700 1 --2- 172.21.15.97:0/829718999 >> v2:172.21.15.183:3300/0 conn(0x7ff07c06c710 0x7ff07c0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff081a23700 1 -- 172.21.15.97:0/829718999 >> v1:172.21.15.183:6789/0 conn(0x7ff07c152cc0 legacy=0x7ff07c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.630+0000 7ff081a23700 1 -- 172.21.15.97:0/829718999 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c15ca70 con 0x7ff07c0b49a0 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff081a23700 1 --2- 172.21.15.97:0/829718999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c0b49a0 0x7ff07c0d0460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff07c074fa0 tx=0x7ff07400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3443a287b7b94c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff073fff700 1 -- 172.21.15.97:0/829718999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07400e070 con 0x7ff07c0b49a0 2023-11-24T06:10:18.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff073fff700 1 -- 172.21.15.97:0/829718999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07400b9c0 con 0x7ff07c0b49a0 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff073fff700 1 -- 172.21.15.97:0/829718999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07400bd70 con 0x7ff07c0b49a0 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff083486700 1 -- 172.21.15.97:0/829718999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c0b49a0 msgr2=0x7ff07c0d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff083486700 1 --2- 172.21.15.97:0/829718999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c0b49a0 0x7ff07c0d0460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff07c074fa0 tx=0x7ff07400b330 comp rx=0 tx=0).stop 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff083486700 1 -- 172.21.15.97:0/829718999 shutdown_connections 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff083486700 1 --2- 172.21.15.97:0/829718999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c0b49a0 0x7ff07c0d0460 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff083486700 1 --2- 172.21.15.97:0/829718999 >> v2:172.21.15.183:3300/0 conn(0x7ff07c06c710 0x7ff07c0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff083486700 1 -- 172.21.15.97:0/829718999 >> 172.21.15.97:0/829718999 conn(0x7ff07c071b90 msgr2=0x7ff07c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff083486700 1 -- 172.21.15.97:0/829718999 shutdown_connections 2023-11-24T06:10:18.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.631+0000 7ff083486700 1 -- 172.21.15.97:0/829718999 wait complete. 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.632+0000 7ff083486700 1 Processor -- start 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.632+0000 7ff083486700 1 -- start start 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff083486700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c06c710 0x7ff07c143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff083486700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff07c0b49a0 0x7ff07c09b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff083486700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff07c152cc0 0x7ff07c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff083486700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c09e630 con 0x7ff07c06c710 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff083486700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c09da60 con 0x7ff07c0b49a0 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff083486700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c09dd00 con 0x7ff07c152cc0 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff081222700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c06c710 0x7ff07c143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff081222700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c06c710 0x7ff07c143380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47660/0 (socket says 172.21.15.97:47660) 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.633+0000 7ff081222700 1 -- 172.21.15.97:0/127828556 learned_addr learned my addr 172.21.15.97:0/127828556 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff080a21700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff07c0b49a0 0x7ff07c09b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff081222700 1 -- 172.21.15.97:0/127828556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff07c152cc0 msgr2=0x7ff07c0b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff081222700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff07c152cc0 0x7ff07c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff081222700 1 -- 172.21.15.97:0/127828556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff07c0b49a0 msgr2=0x7ff07c09b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff081222700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff07c0b49a0 0x7ff07c09b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff081222700 1 -- 172.21.15.97:0/127828556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074009b30 con 0x7ff07c06c710 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff081222700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c06c710 0x7ff07c143380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c450 tx=0x7ff06c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff071ffb700 1 -- 172.21.15.97:0/127828556 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0052d0 con 0x7ff07c06c710 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff071ffb700 1 -- 172.21.15.97:0/127828556 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c005430 con 0x7ff07c06c710 2023-11-24T06:10:18.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07c09dfe0 con 0x7ff07c06c710 2023-11-24T06:10:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff071ffb700 1 -- 172.21.15.97:0/127828556 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0038c0 con 0x7ff07c06c710 2023-11-24T06:10:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.634+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c1438c0 con 0x7ff07c06c710 2023-11-24T06:10:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.635+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07c0753f0 con 0x7ff07c06c710 2023-11-24T06:10:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.635+0000 7ff071ffb700 1 -- 172.21.15.97:0/127828556 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff06c003a20 con 0x7ff07c06c710 2023-11-24T06:10:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.636+0000 7ff071ffb700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff06006af60 0x7ff06006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.636+0000 7ff071ffb700 1 -- 172.21.15.97:0/127828556 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff06c0886d0 con 0x7ff07c06c710 2023-11-24T06:10:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.636+0000 7ff080a21700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff06006af60 0x7ff06006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:18.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.636+0000 7ff080a21700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff06006af60 0x7ff06006d420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff078009950 tx=0x7ff078008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:18.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.640+0000 7ff071ffb700 1 -- 172.21.15.97:0/127828556 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff06c053b20 con 0x7ff07c06c710 2023-11-24T06:10:18.913 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:18.909+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7ff07c067f60 con 0x7ff07c06c710 2023-11-24T06:10:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.146+0000 7ff071ffb700 1 -- 172.21.15.97:0/127828556 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff06c057140 con 0x7ff07c06c710 2023-11-24T06:10:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff06006af60 msgr2=0x7ff06006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff06006af60 0x7ff06006d420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff078009950 tx=0x7ff078008040 comp rx=0 tx=0).stop 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c06c710 msgr2=0x7ff07c143380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c06c710 0x7ff07c143380 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c450 tx=0x7ff06c00c810 comp rx=0 tx=0).stop 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 shutdown_connections 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff06006af60 0x7ff06006d420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff07c152cc0 0x7ff07c0b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff07c06c710 0x7ff07c143380 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 --2- 172.21.15.97:0/127828556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff07c0b49a0 0x7ff07c09b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 >> 172.21.15.97:0/127828556 conn(0x7ff07c071b90 msgr2=0x7ff07c06b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 shutdown_connections 2023-11-24T06:10:19.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.148+0000 7ff083486700 1 -- 172.21.15.97:0/127828556 wait complete. 2023-11-24T06:10:19.165 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... ok 2023-11-24T06:10:19.165 INFO:tasks.cephfs_test_runner:Starting test: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) Validate cleaning of stale subvolume snapshot metadata. 2023-11-24T06:10:19.165 INFO:tasks.cephfs_test_runner:test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:10:19.166 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:19.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.282+0000 7f8bb911d700 1 Processor -- start 2023-11-24T06:10:19.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.282+0000 7f8bb911d700 1 -- start start 2023-11-24T06:10:19.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.282+0000 7f8bb911d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8bb40b04b0 0x7f8bb40622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.282+0000 7f8bb911d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb40af1e0 0x7f8bb40b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.282+0000 7f8bb911d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb415c540 con 0x7f8bb415c160 2023-11-24T06:10:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.282+0000 7f8bb911d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40b2ee0 con 0x7f8bb40b04b0 2023-11-24T06:10:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.282+0000 7f8bb911d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40add40 con 0x7f8bb40af1e0 2023-11-24T06:10:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb2d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8bb415c160 0x7f8bb40afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40120/0 (socket says 172.21.15.97:40120) 2023-11-24T06:10:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb2d9d700 1 -- 172.21.15.97:0/542952758 learned_addr learned my addr 172.21.15.97:0/542952758 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62771665 0 0) 0x7f8bb415c540 con 0x7f8bb415c160 2023-11-24T06:10:19.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba00035b0 con 0x7f8bb415c160 2023-11-24T06:10:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb359e700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb40af1e0 0x7f8bb40b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb259c700 1 --2- 172.21.15.97:0/542952758 >> v2:172.21.15.183:3300/0 conn(0x7f8bb40b04b0 0x7f8bb40622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58940188 0 0) 0x7f8ba00035b0 con 0x7f8bb415c160 2023-11-24T06:10:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb40af1e0 msgr2=0x7f8bb40b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb40af1e0 0x7f8bb40b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 >> v2:172.21.15.183:3300/0 conn(0x7f8bb40b04b0 msgr2=0x7f8bb40622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 --2- 172.21.15.97:0/542952758 >> v2:172.21.15.183:3300/0 conn(0x7f8bb40b04b0 0x7f8bb40622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40ad410 con 0x7f8bb415c160 2023-11-24T06:10:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f8ba4001670 con 0x7f8bb415c160 2023-11-24T06:10:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 >> v1:172.21.15.97:6789/0 conn(0x7f8bb415c160 legacy=0x7f8bb40afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba0003a00 0x7f8ba0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb40af1e0 0x7f8ba0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8ba0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40b2ee0 con 0x7f8ba0004d90 2023-11-24T06:10:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40add40 con 0x7f8ba0003a00 2023-11-24T06:10:19.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000ba10 con 0x7f8bb40af1e0 2023-11-24T06:10:19.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb359e700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8ba0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb2d9d700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba0003a00 0x7f8ba0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb259c700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb40af1e0 0x7f8ba0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb359e700 1 -- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb40af1e0 msgr2=0x7f8ba0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.283+0000 7f8bb359e700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb40af1e0 0x7f8ba0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb359e700 1 -- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba0003a00 msgr2=0x7f8ba0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb359e700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba0003a00 0x7f8ba0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb359e700 1 -- 172.21.15.97:0/542952758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba00035b0 con 0x7f8ba0004d90 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb359e700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8ba0005210 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009f90 tx=0x7f8ba0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90eb62a357868df5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00ebd0 con 0x7f8ba0004d90 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00f040 con 0x7f8ba0004d90 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb1d9b700 1 -- 172.21.15.97:0/542952758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac014480 con 0x7f8ba0004d90 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb911d700 1 -- 172.21.15.97:0/542952758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 msgr2=0x7f8ba0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb911d700 1 --2- 172.21.15.97:0/542952758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8ba0005210 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009f90 tx=0x7f8ba0005880 comp rx=0 tx=0).stop 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb359e700 1 -- 172.21.15.97:0/542952758 reap_dead start 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb911d700 1 -- 172.21.15.97:0/542952758 shutdown_connections 2023-11-24T06:10:19.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb911d700 1 -- 172.21.15.97:0/542952758 >> 172.21.15.97:0/542952758 conn(0x7f8bb40b57b0 msgr2=0x7f8bb40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb911d700 1 -- 172.21.15.97:0/542952758 shutdown_connections 2023-11-24T06:10:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.284+0000 7f8bb911d700 1 -- 172.21.15.97:0/542952758 wait complete. 2023-11-24T06:10:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.286+0000 7f8bb911d700 1 Processor -- start 2023-11-24T06:10:19.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.286+0000 7f8bb911d700 1 -- start start 2023-11-24T06:10:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb911d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8bb41f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb911d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb40c9610 0x7f8bb40c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb911d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb415dd40 0x7f8bb415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb911d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40c9fb0 con 0x7f8ba0004d90 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb911d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb415e710 con 0x7f8bb40c9610 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb911d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb415c540 con 0x7f8bb415dd40 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8bb41f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8bb41f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47680/0 (socket says 172.21.15.97:47680) 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 -- 172.21.15.97:0/668279891 learned_addr learned my addr 172.21.15.97:0/668279891 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb259c700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb40c9610 0x7f8bb40c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb359e700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb415dd40 0x7f8bb415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 -- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb415dd40 msgr2=0x7f8bb415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb415dd40 0x7f8bb415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 -- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb40c9610 msgr2=0x7f8bb40c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb40c9610 0x7f8bb40c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 -- 172.21.15.97:0/668279891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb415d550 con 0x7f8ba0004d90 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb359e700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb415dd40 0x7f8bb415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb2d9d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8bb41f4590 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4007340 tx=0x7f8ba4007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bab7fe700 1 -- 172.21.15.97:0/668279891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400f660 con 0x7f8ba0004d90 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bab7fe700 1 -- 172.21.15.97:0/668279891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba400f800 con 0x7f8ba0004d90 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb415c790 con 0x7f8ba0004d90 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bab7fe700 1 -- 172.21.15.97:0/668279891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400fbb0 con 0x7f8ba0004d90 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.287+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb415cd50 con 0x7f8ba0004d90 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.289+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb4067f60 con 0x7f8ba0004d90 2023-11-24T06:10:19.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.289+0000 7f8bab7fe700 1 -- 172.21.15.97:0/668279891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8ba400fd10 con 0x7f8ba0004d90 2023-11-24T06:10:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.289+0000 7f8bab7fe700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b9406af60 0x7f8b9406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.289+0000 7f8bb259c700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b9406af60 0x7f8b9406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.289+0000 7f8bab7fe700 1 -- 172.21.15.97:0/668279891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8ba4089280 con 0x7f8ba0004d90 2023-11-24T06:10:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.290+0000 7f8bb259c700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b9406af60 0x7f8b9406d420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009ce0 tx=0x7f8b9c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.294+0000 7f8bab7fe700 1 -- 172.21.15.97:0/668279891 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ba40546d0 con 0x7f8ba0004d90 2023-11-24T06:10:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.485+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bb406d850 con 0x7f8ba0004d90 2023-11-24T06:10:19.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.486+0000 7f8bab7fe700 1 -- 172.21.15.97:0/668279891 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+12998 (secure 0 0 0) 0x7f8ba4057cf0 con 0x7f8ba0004d90 2023-11-24T06:10:19.490 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:19.491 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:18.134044+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6837/3767182596":"2023-11-25T06:10:14.744461+0000","172.21.15.97:6835/3767182596":"2023-11-25T06:10:14.744461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b9406af60 msgr2=0x7f8b9406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b9406af60 0x7f8b9406d420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009ce0 tx=0x7f8b9c009210 comp rx=0 tx=0).stop 2023-11-24T06:10:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 msgr2=0x7f8bb41f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8bb41f4590 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4007340 tx=0x7f8ba4007650 comp rx=0 tx=0).stop 2023-11-24T06:10:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 shutdown_connections 2023-11-24T06:10:19.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b9406af60 0x7f8b9406d420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8bb415dd40 0x7f8bb415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba0004d90 0x7f8bb41f4590 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 --2- 172.21.15.97:0/668279891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bb40c9610 0x7f8bb40c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 >> 172.21.15.97:0/668279891 conn(0x7f8bb40b57b0 msgr2=0x7f8bb40b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:19.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 shutdown_connections 2023-11-24T06:10:19.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.488+0000 7f8bb911d700 1 -- 172.21.15.97:0/668279891 wait complete. 2023-11-24T06:10:19.504 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:19.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:19.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.623+0000 7efe1fa9f700 1 Processor -- start 2023-11-24T06:10:19.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.624+0000 7efe1fa9f700 1 -- start start 2023-11-24T06:10:19.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.624+0000 7efe1fa9f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180746a0 0x7efe180b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.624+0000 7efe1fa9f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efe180afc90 0x7efe1806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.624+0000 7efe1fa9f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180b3c60 con 0x7efe1815e160 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.624+0000 7efe1fa9f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe180b4500 con 0x7efe180afc90 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.624+0000 7efe1fa9f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe180b58c0 con 0x7efe180746a0 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1d83b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efe1815e160 0x7efe18073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40140/0 (socket says 172.21.15.97:40140) 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1d83b700 1 -- 172.21.15.97:0/3527582084 learned_addr learned my addr 172.21.15.97:0/3527582084 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1e03c700 1 --2- 172.21.15.97:0/3527582084 >> v2:172.21.15.183:3300/0 conn(0x7efe180afc90 0x7efe1806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1d03a700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180746a0 0x7efe180b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146428931 0 0) 0x7efe180b3c60 con 0x7efe1815e160 2023-11-24T06:10:19.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe040035b0 con 0x7efe1815e160 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3629732447 0 0) 0x7efe040035b0 con 0x7efe1815e160 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180746a0 msgr2=0x7efe180b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180746a0 0x7efe180b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 >> v2:172.21.15.183:3300/0 conn(0x7efe180afc90 msgr2=0x7efe1806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 --2- 172.21.15.97:0/3527582084 >> v2:172.21.15.183:3300/0 conn(0x7efe180afc90 0x7efe1806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180b0140 con 0x7efe1815e160 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7efe14001670 con 0x7efe1815e160 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 >> v1:172.21.15.97:6789/0 conn(0x7efe1815e160 legacy=0x7efe18073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180746a0 0x7efe04003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe04003f80 0x7efe04004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe04004d90 0x7efe04005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180b4500 con 0x7efe04004d90 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180b58c0 con 0x7efe04003f80 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0400ba10 con 0x7efe180746a0 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1e03c700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe04004d90 0x7efe04005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1d03a700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe04003f80 0x7efe04004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.625+0000 7efe1d83b700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180746a0 0x7efe04003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1e03c700 1 -- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180746a0 msgr2=0x7efe04003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1e03c700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180746a0 0x7efe04003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1e03c700 1 -- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe04003f80 msgr2=0x7efe04004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1e03c700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe04003f80 0x7efe04004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1e03c700 1 -- 172.21.15.97:0/3527582084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040035b0 con 0x7efe04004d90 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1d03a700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe04003f80 0x7efe04004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1e03c700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe04004d90 0x7efe04005210 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efe10009fc0 tx=0x7efe04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacbe66d75de04a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000ebd0 con 0x7efe04004d90 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1000f040 con 0x7efe04004d90 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1c839700 1 -- 172.21.15.97:0/3527582084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10014480 con 0x7efe04004d90 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1fa9f700 1 -- 172.21.15.97:0/3527582084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe04004d90 msgr2=0x7efe04005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1fa9f700 1 --2- 172.21.15.97:0/3527582084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe04004d90 0x7efe04005210 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efe10009fc0 tx=0x7efe04005880 comp rx=0 tx=0).stop 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.626+0000 7efe1e03c700 1 -- 172.21.15.97:0/3527582084 reap_dead start 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.627+0000 7efe1fa9f700 1 -- 172.21.15.97:0/3527582084 shutdown_connections 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.627+0000 7efe1fa9f700 1 -- 172.21.15.97:0/3527582084 >> 172.21.15.97:0/3527582084 conn(0x7efe180d0bf0 msgr2=0x7efe180d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:19.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.627+0000 7efe1fa9f700 1 -- 172.21.15.97:0/3527582084 shutdown_connections 2023-11-24T06:10:19.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.627+0000 7efe1fa9f700 1 -- 172.21.15.97:0/3527582084 wait complete. 2023-11-24T06:10:19.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.628+0000 7efe1fa9f700 1 Processor -- start 2023-11-24T06:10:19.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.628+0000 7efe1fa9f700 1 -- start start 2023-11-24T06:10:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1fa9f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe181588f0 0x7efe180c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1fa9f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe181f4700 0x7efe180c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1fa9f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180c7070 0x7efe180b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1fa9f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe181f4ea0 con 0x7efe181588f0 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1fa9f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c81a0 con 0x7efe181f4700 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1fa9f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c66a0 con 0x7efe180c7070 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe181588f0 0x7efe180c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe181588f0 0x7efe180c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47700/0 (socket says 172.21.15.97:47700) 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 -- 172.21.15.97:0/2531027111 learned_addr learned my addr 172.21.15.97:0/2531027111 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1e03c700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180c7070 0x7efe180b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d03a700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe181f4700 0x7efe180c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 -- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180c7070 msgr2=0x7efe180b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180c7070 0x7efe180b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 -- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe181f4700 msgr2=0x7efe180c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe181f4700 0x7efe180c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 -- 172.21.15.97:0/2531027111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180c5d70 con 0x7efe181588f0 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1e03c700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180c7070 0x7efe180b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d03a700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe181f4700 0x7efe180c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1d83b700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe181588f0 0x7efe180c7c60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efe14001a30 tx=0x7efe14007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe0e7fc700 1 -- 172.21.15.97:0/2531027111 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1400f840 con 0x7efe181588f0 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe0e7fc700 1 -- 172.21.15.97:0/2531027111 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1400f9a0 con 0x7efe181588f0 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe180c6050 con 0x7efe181588f0 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe0e7fc700 1 -- 172.21.15.97:0/2531027111 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1400fd50 con 0x7efe181588f0 2023-11-24T06:10:19.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.629+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe181f4b80 con 0x7efe181588f0 2023-11-24T06:10:19.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.630+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe18068fb0 con 0x7efe181588f0 2023-11-24T06:10:19.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.631+0000 7efe0e7fc700 1 -- 172.21.15.97:0/2531027111 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7efe1401d460 con 0x7efe181588f0 2023-11-24T06:10:19.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.631+0000 7efe0e7fc700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efdfc06af60 0x7efdfc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.631+0000 7efe0e7fc700 1 -- 172.21.15.97:0/2531027111 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efe14089490 con 0x7efe181588f0 2023-11-24T06:10:19.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.631+0000 7efe1d03a700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efdfc06af60 0x7efdfc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.632+0000 7efe1d03a700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efdfc06af60 0x7efdfc06d420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe08006ba0 tx=0x7efe08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.636+0000 7efe0e7fc700 1 -- 172.21.15.97:0/2531027111 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe140548e0 con 0x7efe181588f0 2023-11-24T06:10:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.825+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe1806c140 con 0x7efe181588f0 2023-11-24T06:10:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.825+0000 7efe0e7fc700 1 -- 172.21.15.97:0/2531027111 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe14057f00 con 0x7efe181588f0 2023-11-24T06:10:19.829 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:19.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efdfc06af60 msgr2=0x7efdfc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efdfc06af60 0x7efdfc06d420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efe08006ba0 tx=0x7efe08008040 comp rx=0 tx=0).stop 2023-11-24T06:10:19.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe181588f0 msgr2=0x7efe180c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe181588f0 0x7efe180c7c60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efe14001a30 tx=0x7efe14007710 comp rx=0 tx=0).stop 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 shutdown_connections 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efdfc06af60 0x7efdfc06d420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe180c7070 0x7efe180b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe181588f0 0x7efe180c7c60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 --2- 172.21.15.97:0/2531027111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe181f4700 0x7efe180c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 >> 172.21.15.97:0/2531027111 conn(0x7efe180d0bf0 msgr2=0x7efe180add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 shutdown_connections 2023-11-24T06:10:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.827+0000 7efe1fa9f700 1 -- 172.21.15.97:0/2531027111 wait complete. 2023-11-24T06:10:19.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T06:10:19.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.960+0000 7f0d8798e700 1 Processor -- start 2023-11-24T06:10:19.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.960+0000 7f0d8798e700 1 -- start start 2023-11-24T06:10:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d8798e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d8006c710 0x7f0d800b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d8798e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0d800af4d0 0x7f0d800afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d8798e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800ae430 con 0x7f0d8015e080 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d8798e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800af0b0 con 0x7f0d8006c710 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d8798e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800b05a0 con 0x7f0d800af4d0 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d84f29700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d8006c710 0x7f0d800b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d84f29700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0d8006c710 0x7f0d800b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47712/0 (socket says 172.21.15.97:47712) 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d84f29700 1 -- 172.21.15.97:0/927308289 learned_addr learned my addr 172.21.15.97:0/927308289 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d84f29700 1 -- 172.21.15.97:0/927308289 >> v2:172.21.15.183:3300/0 conn(0x7f0d800af4d0 msgr2=0x7f0d800afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d84f29700 1 --2- 172.21.15.97:0/927308289 >> v2:172.21.15.183:3300/0 conn(0x7f0d800af4d0 0x7f0d800afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d84f29700 1 -- 172.21.15.97:0/927308289 >> v1:172.21.15.183:6789/0 conn(0x7f0d8015e080 legacy=0x7f0d8006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.961+0000 7f0d84f29700 1 -- 172.21.15.97:0/927308289 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800ae710 con 0x7f0d8006c710 2023-11-24T06:10:19.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d84f29700 1 --2- 172.21.15.97:0/927308289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d800b5050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d8009eea0 tx=0x7f0d7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd48b28f8a890da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d77fff700 1 -- 172.21.15.97:0/927308289 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c004230 con 0x7f0d8006c710 2023-11-24T06:10:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d77fff700 1 -- 172.21.15.97:0/927308289 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c004390 con 0x7f0d8006c710 2023-11-24T06:10:19.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d77fff700 1 -- 172.21.15.97:0/927308289 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0036a0 con 0x7f0d8006c710 2023-11-24T06:10:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d8798e700 1 -- 172.21.15.97:0/927308289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 msgr2=0x7f0d800b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d8798e700 1 --2- 172.21.15.97:0/927308289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d800b5050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d8009eea0 tx=0x7f0d7c009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d8798e700 1 -- 172.21.15.97:0/927308289 shutdown_connections 2023-11-24T06:10:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d8798e700 1 --2- 172.21.15.97:0/927308289 >> v2:172.21.15.183:3300/0 conn(0x7f0d800af4d0 0x7f0d800afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d8798e700 1 --2- 172.21.15.97:0/927308289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d800b5050 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d8798e700 1 -- 172.21.15.97:0/927308289 >> 172.21.15.97:0/927308289 conn(0x7f0d80071b90 msgr2=0x7f0d80071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d8798e700 1 -- 172.21.15.97:0/927308289 shutdown_connections 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.962+0000 7f0d8798e700 1 -- 172.21.15.97:0/927308289 wait complete. 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8798e700 1 Processor -- start 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8798e700 1 -- start start 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8798e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d801574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8798e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d800af4d0 0x7f0d8014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8798e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d8015e080 0x7f0d80160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8798e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8014ac80 con 0x7f0d8006c710 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8798e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d80161350 con 0x7f0d800af4d0 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8798e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d80157b50 con 0x7f0d8015e080 2023-11-24T06:10:19.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8572a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d801574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8572a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d801574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47714/0 (socket says 172.21.15.97:47714) 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.964+0000 7f0d8572a700 1 -- 172.21.15.97:0/1992686087 learned_addr learned my addr 172.21.15.97:0/1992686087 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d8572a700 1 -- 172.21.15.97:0/1992686087 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d8015e080 msgr2=0x7f0d80160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d8572a700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d8015e080 0x7f0d80160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d8572a700 1 -- 172.21.15.97:0/1992686087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d800af4d0 msgr2=0x7f0d8014a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d8572a700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d800af4d0 0x7f0d8014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d8572a700 1 -- 172.21.15.97:0/1992686087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009550 con 0x7f0d8006c710 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d8572a700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d801574e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b470 tx=0x7f0d7000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d75ffb700 1 -- 172.21.15.97:0/1992686087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70011830 con 0x7f0d8006c710 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d75ffb700 1 -- 172.21.15.97:0/1992686087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70011990 con 0x7f0d8006c710 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d80157e30 con 0x7f0d8006c710 2023-11-24T06:10:19.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d75ffb700 1 -- 172.21.15.97:0/1992686087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70011d40 con 0x7f0d8006c710 2023-11-24T06:10:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.965+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d801583f0 con 0x7f0d8006c710 2023-11-24T06:10:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.966+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d800753f0 con 0x7f0d8006c710 2023-11-24T06:10:19.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.967+0000 7f0d75ffb700 1 -- 172.21.15.97:0/1992686087 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0d7001a400 con 0x7f0d8006c710 2023-11-24T06:10:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.967+0000 7f0d75ffb700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0d6406af60 0x7f0d6406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.967+0000 7f0d84f29700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0d6406af60 0x7f0d6406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.967+0000 7f0d75ffb700 1 -- 172.21.15.97:0/1992686087 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0d700892b0 con 0x7f0d8006c710 2023-11-24T06:10:19.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.968+0000 7f0d84f29700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0d6406af60 0x7f0d6406d420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d80161980 tx=0x7f0d7c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:19.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:19.972+0000 7f0d75ffb700 1 -- 172.21.15.97:0/1992686087 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d70054700 con 0x7f0d8006c710 2023-11-24T06:10:20.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:20.245+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f0d8006e590 con 0x7f0d8006c710 2023-11-24T06:10:21.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.152+0000 7f0d75ffb700 1 -- 172.21.15.97:0/1992686087 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v101) v1 ==== 175+0+0 (secure 0 0 0) 0x7f0d70057d20 con 0x7f0d8006c710 2023-11-24T06:10:21.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0d6406af60 msgr2=0x7f0d6406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0d6406af60 0x7f0d6406d420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d80161980 tx=0x7f0d7c010000 comp rx=0 tx=0).stop 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 msgr2=0x7f0d801574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d801574e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b470 tx=0x7f0d7000b830 comp rx=0 tx=0).stop 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 shutdown_connections 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0d6406af60 0x7f0d6406d420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0d8015e080 0x7f0d80160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0d8006c710 0x7f0d801574e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 --2- 172.21.15.97:0/1992686087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d800af4d0 0x7f0d8014a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 >> 172.21.15.97:0/1992686087 conn(0x7f0d80071b90 msgr2=0x7f0d800cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 shutdown_connections 2023-11-24T06:10:21.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.155+0000 7f0d8798e700 1 -- 172.21.15.97:0/1992686087 wait complete. 2023-11-24T06:10:21.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:10:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.299+0000 7f982d923700 1 Processor -- start 2023-11-24T06:10:21.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.299+0000 7f982d923700 1 -- start start 2023-11-24T06:10:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f982d923700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f98280d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f982d923700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f98280d24e0 0x7f9828071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f982d923700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9828071f70 con 0x7f9828071b50 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f982d923700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98280d2a60 con 0x7f98280d24e0 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f982d923700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98280d2d40 con 0x7f982815e090 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f98277fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9828071b50 0x7f98280b5920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40174/0 (socket says 172.21.15.97:40174) 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f98277fe700 1 -- 172.21.15.97:0/1694026604 learned_addr learned my addr 172.21.15.97:0/1694026604 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f98267fc700 1 --2- 172.21.15.97:0/1694026604 >> v2:172.21.15.183:3300/0 conn(0x7f98280d24e0 0x7f9828071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f9826ffd700 1 --2- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f98280d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f9825ffb700 1 -- 172.21.15.97:0/1694026604 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284166539 0 0) 0x7f9828071f70 con 0x7f9828071b50 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f9825ffb700 1 -- 172.21.15.97:0/1694026604 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98100035b0 con 0x7f9828071b50 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f98267fc700 1 -- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 msgr2=0x7f98280d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f98267fc700 1 --2- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f98280d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f98267fc700 1 -- 172.21.15.97:0/1694026604 >> v1:172.21.15.97:6789/0 conn(0x7f9828071b50 legacy=0x7f98280b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f9825ffb700 1 -- 172.21.15.97:0/1694026604 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2204712203 0 0) 0x7f98100035b0 con 0x7f9828071b50 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f98267fc700 1 -- 172.21.15.97:0/1694026604 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98280d3020 con 0x7f98280d24e0 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f9826ffd700 1 --2- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f98280d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.300+0000 7f98267fc700 1 --2- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98280d24e0 0x7f9828071610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f981c009840 tx=0x7f981c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=665db0b60883e7db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f9825ffb700 1 -- 172.21.15.97:0/1694026604 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98200019f0 con 0x7f98280d24e0 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f9825ffb700 1 -- 172.21.15.97:0/1694026604 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981c004030 con 0x7f98280d24e0 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f9825ffb700 1 -- 172.21.15.97:0/1694026604 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0043a0 con 0x7f98280d24e0 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f982d923700 1 -- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98280d24e0 msgr2=0x7f9828071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f982d923700 1 --2- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98280d24e0 0x7f9828071610 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f981c009840 tx=0x7f981c009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f982d923700 1 -- 172.21.15.97:0/1694026604 shutdown_connections 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f982d923700 1 --2- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f98280d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f982d923700 1 --2- 172.21.15.97:0/1694026604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98280d24e0 0x7f9828071610 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f982d923700 1 -- 172.21.15.97:0/1694026604 >> 172.21.15.97:0/1694026604 conn(0x7f982806b410 msgr2=0x7f982806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f982d923700 1 -- 172.21.15.97:0/1694026604 shutdown_connections 2023-11-24T06:10:21.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.301+0000 7f982d923700 1 -- 172.21.15.97:0/1694026604 wait complete. 2023-11-24T06:10:21.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.302+0000 7f982d923700 1 Processor -- start 2023-11-24T06:10:21.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f982d923700 1 -- start start 2023-11-24T06:10:21.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f982d923700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9828071b50 0x7f9828160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f982d923700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98280d24e0 0x7f9828158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f982d923700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f9828158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f982d923700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9828160e70 con 0x7f9828071b50 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f982d923700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98280cb590 con 0x7f98280d24e0 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f982d923700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98280caf80 con 0x7f982815e090 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f9826ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9828071b50 0x7f9828160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f9826ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9828071b50 0x7f9828160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47728/0 (socket says 172.21.15.97:47728) 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.303+0000 7f9826ffd700 1 -- 172.21.15.97:0/1311509975 learned_addr learned my addr 172.21.15.97:0/1311509975 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f9826ffd700 1 -- 172.21.15.97:0/1311509975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 msgr2=0x7f9828158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f98277fe700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f9828158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f9826ffd700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f9828158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f9826ffd700 1 -- 172.21.15.97:0/1311509975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98280d24e0 msgr2=0x7f9828158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f9826ffd700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98280d24e0 0x7f9828158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f9826ffd700 1 -- 172.21.15.97:0/1311509975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180094a0 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f9826ffd700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9828071b50 0x7f9828160930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f981800ec10 tx=0x7f9818009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f980ffff700 1 -- 172.21.15.97:0/1311509975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981800c8e0 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f980ffff700 1 -- 172.21.15.97:0/1311509975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981800ca40 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c0094a0 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f980ffff700 1 -- 172.21.15.97:0/1311509975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981800cdb0 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.304+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9828158950 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.305+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98281610c0 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.305+0000 7f980ffff700 1 -- 172.21.15.97:0/1311509975 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9818013470 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.306+0000 7f980ffff700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f981406af60 0x7f981406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.306+0000 7f980ffff700 1 -- 172.21.15.97:0/1311509975 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f98180894c0 con 0x7f9828071b50 2023-11-24T06:10:21.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.306+0000 7f98267fc700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f981406af60 0x7f981406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.306+0000 7f98267fc700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f981406af60 0x7f981406d420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f982815afc0 tx=0x7f981c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.311+0000 7f980ffff700 1 -- 172.21.15.97:0/1311509975 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9818054800 con 0x7f9828071b50 2023-11-24T06:10:21.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.502+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9828068fb0 con 0x7f9828071b50 2023-11-24T06:10:21.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.505+0000 7f980ffff700 1 -- 172.21.15.97:0/1311509975 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9818057e20 con 0x7f9828071b50 2023-11-24T06:10:21.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f981406af60 msgr2=0x7f981406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f981406af60 0x7f981406d420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f982815afc0 tx=0x7f981c010000 comp rx=0 tx=0).stop 2023-11-24T06:10:21.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9828071b50 msgr2=0x7f9828160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9828071b50 0x7f9828160930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f981800ec10 tx=0x7f9818009e20 comp rx=0 tx=0).stop 2023-11-24T06:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 shutdown_connections 2023-11-24T06:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f981406af60 0x7f981406d420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f982815e090 0x7f9828158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9828071b50 0x7f9828160930 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 --2- 172.21.15.97:0/1311509975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98280d24e0 0x7f9828158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 >> 172.21.15.97:0/1311509975 conn(0x7f982806b410 msgr2=0x7f982815d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 shutdown_connections 2023-11-24T06:10:21.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.507+0000 7f982d923700 1 -- 172.21.15.97:0/1311509975 wait complete. 2023-11-24T06:10:21.523 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:10:21.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:10:21.564 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:10:21.621 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:10:21.621 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:21.621 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:21.638 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:10:21.638 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:21.638 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:21.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:10:21.658 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:21.658 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:21.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:10:21.695 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:21.695 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:21.718 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:10:21.718 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:10:21.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.852+0000 7f6af70e7700 1 Processor -- start 2023-11-24T06:10:21.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.853+0000 7f6af70e7700 1 -- start start 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.853+0000 7f6af70e7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6af00af1e0 0x7f6af00622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.853+0000 7f6af70e7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af015c540 con 0x7f6af00b04b0 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.853+0000 7f6af70e7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af00b2ee0 con 0x7f6af015c160 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.853+0000 7f6af70e7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af00add40 con 0x7f6af00af1e0 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aeffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6af00b04b0 0x7f6af0163c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40190/0 (socket says 172.21.15.97:40190) 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aeffff700 1 -- 172.21.15.97:0/2706985134 learned_addr learned my addr 172.21.15.97:0/2706985134 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6af5684700 1 --2- 172.21.15.97:0/2706985134 >> v2:172.21.15.183:3300/0 conn(0x7f6af00af1e0 0x7f6af00622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949105146 0 0) 0x7f6af015c540 con 0x7f6af00b04b0 2023-11-24T06:10:21.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae00035b0 con 0x7f6af00b04b0 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3945327460 0 0) 0x7f6ae00035b0 con 0x7f6af00b04b0 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 >> v2:172.21.15.183:3300/0 conn(0x7f6af00af1e0 msgr2=0x7f6af00622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 --2- 172.21.15.97:0/2706985134 >> v2:172.21.15.183:3300/0 conn(0x7f6af00af1e0 0x7f6af00622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 >> v1:172.21.15.183:6789/0 conn(0x7f6af015c160 legacy=0x7f6af00afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00ad410 con 0x7f6af00b04b0 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f6adc001670 con 0x7f6af00b04b0 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 >> v1:172.21.15.97:6789/0 conn(0x7f6af00b04b0 legacy=0x7f6af0163c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0003a00 0x7f6ae0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ae0004850 0x7f6ae0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ae0005210 0x7f6ae000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00add40 con 0x7f6ae0004850 2023-11-24T06:10:21.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae000c0b0 con 0x7f6ae0003a00 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae000c270 con 0x7f6ae0005210 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6aeffff700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ae0004850 0x7f6ae0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6af4e83700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0003a00 0x7f6ae0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.854+0000 7f6af5684700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ae0005210 0x7f6ae000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aeffff700 1 -- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ae0005210 msgr2=0x7f6ae000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aeffff700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ae0005210 0x7f6ae000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aeffff700 1 -- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0003a00 msgr2=0x7f6ae0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aeffff700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0003a00 0x7f6ae0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aeffff700 1 -- 172.21.15.97:0/2706985134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae00035b0 con 0x7f6ae0004850 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af5684700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ae0005210 0x7f6ae000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af5684700 1 -- 172.21.15.97:0/2706985134 reap_dead start 2023-11-24T06:10:21.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aeffff700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ae0004850 0x7f6ae0004cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c240 tx=0x7f6ae0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f653a9c884147c43 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc007c10 con 0x7f6ae0004850 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00cde0 con 0x7f6ae0004850 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6aef7fe700 1 -- 172.21.15.97:0/2706985134 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc005ba0 con 0x7f6ae0004850 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af70e7700 1 -- 172.21.15.97:0/2706985134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ae0004850 msgr2=0x7f6ae0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af70e7700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ae0004850 0x7f6ae0004cd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00c240 tx=0x7f6ae0005880 comp rx=0 tx=0).stop 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af70e7700 1 -- 172.21.15.97:0/2706985134 shutdown_connections 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af70e7700 1 --2- 172.21.15.97:0/2706985134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ae0004850 0x7f6ae0004cd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af70e7700 1 -- 172.21.15.97:0/2706985134 >> 172.21.15.97:0/2706985134 conn(0x7f6af00b57b0 msgr2=0x7f6af00b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af70e7700 1 -- 172.21.15.97:0/2706985134 shutdown_connections 2023-11-24T06:10:21.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.855+0000 7f6af70e7700 1 -- 172.21.15.97:0/2706985134 wait complete. 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.857+0000 7f6af70e7700 1 Processor -- start 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.857+0000 7f6af70e7700 1 -- start start 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af70e7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0004850 0x7f6af014a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af70e7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6af01470d0 0x7f6af0148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af70e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6af014ab00 0x7f6af0148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af70e7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01475d0 con 0x7f6af014ab00 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af70e7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01478b0 con 0x7f6ae0004850 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af70e7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af0147b90 con 0x7f6af01470d0 2023-11-24T06:10:21.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af5684700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6af014ab00 0x7f6af0148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af5684700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6af014ab00 0x7f6af0148bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47752/0 (socket says 172.21.15.97:47752) 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af5684700 1 -- 172.21.15.97:0/2284918189 learned_addr learned my addr 172.21.15.97:0/2284918189 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af4e83700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0004850 0x7f6af014a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6aeffff700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6af01470d0 0x7f6af0148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af5684700 1 -- 172.21.15.97:0/2284918189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6af01470d0 msgr2=0x7f6af0148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af5684700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6af01470d0 0x7f6af0148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af5684700 1 -- 172.21.15.97:0/2284918189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0004850 msgr2=0x7f6af014a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.858+0000 7f6af5684700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0004850 0x7f6af014a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.859+0000 7f6af5684700 1 -- 172.21.15.97:0/2284918189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af0147e70 con 0x7f6af014ab00 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.859+0000 7f6af5684700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6af014ab00 0x7f6af0148bb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6af006fce0 tx=0x7f6ae80101c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.859+0000 7f6aed7fa700 1 -- 172.21.15.97:0/2284918189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae801b070 con 0x7f6af014ab00 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.859+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af0148150 con 0x7f6af014ab00 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.859+0000 7f6aed7fa700 1 -- 172.21.15.97:0/2284918189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae800f040 con 0x7f6af014ab00 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.859+0000 7f6aed7fa700 1 -- 172.21.15.97:0/2284918189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800e050 con 0x7f6af014ab00 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.859+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af01463c0 con 0x7f6af014ab00 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.860+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af01484c0 con 0x7f6af014ab00 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.861+0000 7f6aed7fa700 1 -- 172.21.15.97:0/2284918189 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6ae8003c40 con 0x7f6af014ab00 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.861+0000 7f6aed7fa700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ad406af60 0x7f6ad406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.861+0000 7f6aed7fa700 1 -- 172.21.15.97:0/2284918189 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6ae808a500 con 0x7f6af014ab00 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.861+0000 7f6af4e83700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ad406af60 0x7f6ad406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:21.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.861+0000 7f6af4e83700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ad406af60 0x7f6ad406d420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4007fd0 tx=0x7f6ae4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:21.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:21.865+0000 7f6aed7fa700 1 -- 172.21.15.97:0/2284918189 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ae80557b0 con 0x7f6af014ab00 2023-11-24T06:10:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.102+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6af0067f60 con 0x7f6af014ab00 2023-11-24T06:10:22.105 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.103+0000 7f6aed7fa700 1 -- 172.21.15.97:0/2284918189 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f6ae8058dd0 con 0x7f6af014ab00 2023-11-24T06:10:22.106 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:22.106 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4724,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1778230784","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1778230784},{"type":"v1","addr":"172.21.15.97:6836","nonce":1778230784}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2183416217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2183416217},{"type":"v1","addr":"172.21.15.183:6834","nonce":2183416217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4921,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3650123299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3650123299},{"type":"v1","addr":"172.21.15.183:6835","nonce":3650123299}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/1915861460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":1915861460},{"type":"v1","addr":"172.21.15.97:6837","nonce":1915861460}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-11-24T06:10:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ad406af60 msgr2=0x7f6ad406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ad406af60 0x7f6ad406d420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4007fd0 tx=0x7f6ae4009040 comp rx=0 tx=0).stop 2023-11-24T06:10:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6af014ab00 msgr2=0x7f6af0148bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6af014ab00 0x7f6af0148bb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6af006fce0 tx=0x7f6ae80101c0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 shutdown_connections 2023-11-24T06:10:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6af014ab00 0x7f6af0148bb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ae0004850 0x7f6af014a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6ad406af60 0x7f6ad406d420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 --2- 172.21.15.97:0/2284918189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6af01470d0 0x7f6af0148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 >> 172.21.15.97:0/2284918189 conn(0x7f6af00b57b0 msgr2=0x7f6af00b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 shutdown_connections 2023-11-24T06:10:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.105+0000 7f6af70e7700 1 -- 172.21.15.97:0/2284918189 wait complete. 2023-11-24T06:10:22.109 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 20 2023-11-24T06:10:22.120 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:10:22.120 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:10:22.120 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:10:22.120 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:10:22.121 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:10:22.121 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:10:22.121 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:10:22.121 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:10:22.121 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:10:22.121 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:10:22.122 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:10:22.122 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:10:22.122 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:10:22.122 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:10:22.122 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:10:22.122 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:10:22.129 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:10:22.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:10:22.130 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:10:22.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:10:22.133 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:10:22.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:10:22.134 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:10:22.134 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:10:22.172 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:10:22.172 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:22.172 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:10:22.173 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:22.175 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:10:22.175 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:22.176 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:10:22.176 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:22.176 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:10:22.250 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-11-24T06:10:22.251 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-11-24T06:10:22.261 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-11-24T06:10:22.262 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-11-24T06:10:22.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.300+0000 7ff6c69ee700 1 Processor -- start 2023-11-24T06:10:22.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.301+0000 7ff6c69ee700 1 -- start start 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.301+0000 7ff6c69ee700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6c00b1390 0x7ff6c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.301+0000 7ff6c69ee700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 0x7ff6c006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.301+0000 7ff6c69ee700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00b1c60 con 0x7ff6c00b0fb0 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.301+0000 7ff6c69ee700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00d0450 con 0x7ff6c00b1390 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.301+0000 7ff6c69ee700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00d0720 con 0x7ff6c00d2390 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.301+0000 7ff6bf7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6c00b1390 0x7ff6c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.301+0000 7ff6bf7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6c00b1390 0x7ff6c006b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47782/0 (socket says 172.21.15.97:47782) 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6bf7fe700 1 -- 172.21.15.97:0/3471038833 learned_addr learned my addr 172.21.15.97:0/3471038833 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:22.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6c4f8b700 1 --2- 172.21.15.97:0/3471038833 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 0x7ff6c006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6bf7fe700 1 -- 172.21.15.97:0/3471038833 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 msgr2=0x7ff6c006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6bf7fe700 1 --2- 172.21.15.97:0/3471038833 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 0x7ff6c006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6bf7fe700 1 -- 172.21.15.97:0/3471038833 >> v1:172.21.15.183:6789/0 conn(0x7ff6c00b0fb0 legacy=0x7ff6c00cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6bf7fe700 1 -- 172.21.15.97:0/3471038833 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00d0a00 con 0x7ff6c00b1390 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6bf7fe700 1 --2- 172.21.15.97:0/3471038833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c006b750 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40098f0 tx=0x7ff6b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da02e6a080f0f46 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6beffd700 1 -- 172.21.15.97:0/3471038833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4011830 con 0x7ff6c00b1390 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6beffd700 1 -- 172.21.15.97:0/3471038833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b4011990 con 0x7ff6c00b1390 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.302+0000 7ff6beffd700 1 -- 172.21.15.97:0/3471038833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4011d20 con 0x7ff6c00b1390 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.303+0000 7ff6c69ee700 1 -- 172.21.15.97:0/3471038833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 msgr2=0x7ff6c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.303+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/3471038833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c006b750 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff6b40098f0 tx=0x7ff6b4009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.303+0000 7ff6c69ee700 1 -- 172.21.15.97:0/3471038833 shutdown_connections 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.303+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/3471038833 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 0x7ff6c006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.303+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/3471038833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c006b750 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.303+0000 7ff6c69ee700 1 -- 172.21.15.97:0/3471038833 >> 172.21.15.97:0/3471038833 conn(0x7ff6c009aa00 msgr2=0x7ff6c009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.303+0000 7ff6c69ee700 1 -- 172.21.15.97:0/3471038833 shutdown_connections 2023-11-24T06:10:22.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.303+0000 7ff6c69ee700 1 -- 172.21.15.97:0/3471038833 wait complete. 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.304+0000 7ff6c69ee700 1 Processor -- start 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.304+0000 7ff6c69ee700 1 -- start start 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6c69ee700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6c00b0fb0 0x7ff6c00b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6c69ee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c0157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6c69ee700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 0x7ff6c01f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6c69ee700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b5db0 con 0x7ff6c00b1390 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6c69ee700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b6090 con 0x7ff6c00b0fb0 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6c69ee700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b6370 con 0x7ff6c00d2390 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c0157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c0157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47788/0 (socket says 172.21.15.97:47788) 2023-11-24T06:10:22.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 -- 172.21.15.97:0/1440428903 learned_addr learned my addr 172.21.15.97:0/1440428903 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6c4f8b700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 0x7ff6c01f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bffff700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6c00b0fb0 0x7ff6c00b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 -- 172.21.15.97:0/1440428903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 msgr2=0x7ff6c01f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 0x7ff6c01f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 -- 172.21.15.97:0/1440428903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6c00b0fb0 msgr2=0x7ff6c00b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6c00b0fb0 0x7ff6c00b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 -- 172.21.15.97:0/1440428903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b4009550 con 0x7ff6c00b1390 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bf7fe700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c0157d60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4000c00 tx=0x7ff6b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bcff9700 1 -- 172.21.15.97:0/1440428903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004400 con 0x7ff6c00b1390 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bcff9700 1 -- 172.21.15.97:0/1440428903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b401f070 con 0x7ff6c00b1390 2023-11-24T06:10:22.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c00b6650 con 0x7ff6c00b1390 2023-11-24T06:10:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.305+0000 7ff6bcff9700 1 -- 172.21.15.97:0/1440428903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b40038e0 con 0x7ff6c00b1390 2023-11-24T06:10:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.306+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c01f4910 con 0x7ff6c00b1390 2023-11-24T06:10:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.306+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c0068480 con 0x7ff6c00b1390 2023-11-24T06:10:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.307+0000 7ff6bcff9700 1 -- 172.21.15.97:0/1440428903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff6b4021030 con 0x7ff6c00b1390 2023-11-24T06:10:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.307+0000 7ff6bcff9700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff6ac06af60 0x7ff6ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.307+0000 7ff6bcff9700 1 -- 172.21.15.97:0/1440428903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff6b408a690 con 0x7ff6c00b1390 2023-11-24T06:10:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.307+0000 7ff6bffff700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff6ac06af60 0x7ff6ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:22.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.308+0000 7ff6bffff700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff6ac06af60 0x7ff6ac06d420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6c006bf30 tx=0x7ff6b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:22.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.312+0000 7ff6bcff9700 1 -- 172.21.15.97:0/1440428903 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6b4055a60 con 0x7ff6c00b1390 2023-11-24T06:10:22.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:22.517+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff6c00b6b70 con 0x7ff6c00b1390 2023-11-24T06:10:23.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.523+0000 7ff6bcff9700 1 -- 172.21.15.97:0/1440428903 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff6c00b6b70 con 0x7ff6c00b1390 2023-11-24T06:10:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff6ac06af60 msgr2=0x7ff6ac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff6ac06af60 0x7ff6ac06d420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6c006bf30 tx=0x7ff6b0009040 comp rx=0 tx=0).stop 2023-11-24T06:10:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 msgr2=0x7ff6c0157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c0157d60 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4000c00 tx=0x7ff6b4004750 comp rx=0 tx=0).stop 2023-11-24T06:10:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 shutdown_connections 2023-11-24T06:10:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff6ac06af60 0x7ff6ac06d420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff6c00d2390 0x7ff6c01f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff6c00b0fb0 0x7ff6c00b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 --2- 172.21.15.97:0/1440428903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00b1390 0x7ff6c0157d60 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 >> 172.21.15.97:0/1440428903 conn(0x7ff6c009aa00 msgr2=0x7ff6c00cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 shutdown_connections 2023-11-24T06:10:23.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.526+0000 7ff6c69ee700 1 -- 172.21.15.97:0/1440428903 wait complete. 2023-11-24T06:10:23.530 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-11-24T06:10:23.544 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:10:23.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.667+0000 7ff995355700 1 Processor -- start 2023-11-24T06:10:23.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.667+0000 7ff995355700 1 -- start start 2023-11-24T06:10:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff995355700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff99015e1e0 0x7ff9900d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff995355700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9900d20b0 con 0x7ff9900d3a00 2023-11-24T06:10:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff995355700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9900d2390 con 0x7ff9900d0f80 2023-11-24T06:10:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff995355700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9900d3150 con 0x7ff99015e1e0 2023-11-24T06:10:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98e7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff9900d3a00 0x7ff9900d0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40250/0 (socket says 172.21.15.97:40250) 2023-11-24T06:10:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98e7fc700 1 -- 172.21.15.97:0/4234964723 learned_addr learned my addr 172.21.15.97:0/4234964723 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750397331 0 0) 0x7ff9900d20b0 con 0x7ff9900d3a00 2023-11-24T06:10:23.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9780035b0 con 0x7ff9900d3a00 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750826130 0 0) 0x7ff9780035b0 con 0x7ff9900d3a00 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 >> v2:172.21.15.183:3300/0 conn(0x7ff99015e1e0 msgr2=0x7ff9900d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 --2- 172.21.15.97:0/4234964723 >> v2:172.21.15.183:3300/0 conn(0x7ff99015e1e0 0x7ff9900d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 >> v1:172.21.15.183:6789/0 conn(0x7ff9900d0f80 legacy=0x7ff9900b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9900d2670 con 0x7ff9900d3a00 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7ff980001670 con 0x7ff9900d3a00 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 >> v1:172.21.15.97:6789/0 conn(0x7ff9900d3a00 legacy=0x7ff9900d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff978003a00 0x7ff978003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff978004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff978005330 0x7ff97800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900d3150 con 0x7ff978004850 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97800c220 con 0x7ff978005330 2023-11-24T06:10:23.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.668+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97800c3e0 con 0x7ff978003a00 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98e7fc700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff978004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98effd700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff978003a00 0x7ff978003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98f7fe700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff978005330 0x7ff97800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98e7fc700 1 -- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff978003a00 msgr2=0x7ff978003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98e7fc700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff978003a00 0x7ff978003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98e7fc700 1 -- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff978005330 msgr2=0x7ff97800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98e7fc700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff978005330 0x7ff97800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98e7fc700 1 -- 172.21.15.97:0/4234964723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9780035b0 con 0x7ff978004850 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98f7fe700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff978005330 0x7ff97800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98f7fe700 1 -- 172.21.15.97:0/4234964723 reap_dead start 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98e7fc700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff978004cd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff98000c270 tx=0x7ff978005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c5be4ba5a4db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:23.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980007c10 con 0x7ff978004850 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98000cd90 con 0x7ff978004850 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.669+0000 7ff98dffb700 1 -- 172.21.15.97:0/4234964723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980005ba0 con 0x7ff978004850 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.670+0000 7ff995355700 1 -- 172.21.15.97:0/4234964723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 msgr2=0x7ff978004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.670+0000 7ff995355700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff978004cd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff98000c270 tx=0x7ff978005880 comp rx=0 tx=0).stop 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.670+0000 7ff995355700 1 -- 172.21.15.97:0/4234964723 shutdown_connections 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.670+0000 7ff995355700 1 --2- 172.21.15.97:0/4234964723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff978004cd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.670+0000 7ff995355700 1 -- 172.21.15.97:0/4234964723 >> 172.21.15.97:0/4234964723 conn(0x7ff9900b5b50 msgr2=0x7ff9900b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.670+0000 7ff995355700 1 -- 172.21.15.97:0/4234964723 shutdown_connections 2023-11-24T06:10:23.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.670+0000 7ff995355700 1 -- 172.21.15.97:0/4234964723 wait complete. 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.671+0000 7ff995355700 1 Processor -- start 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.671+0000 7ff995355700 1 -- start start 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff995355700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff9901f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff995355700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990157800 0x7ff99015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff995355700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9901f79e0 0x7ff99015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff995355700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9901f75c0 con 0x7ff978004850 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff995355700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900cad50 con 0x7ff990157800 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff995355700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff990157c90 con 0x7ff9901f79e0 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff9901f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff9901f2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47810/0 (socket says 172.21.15.97:47810) 2023-11-24T06:10:23.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 -- 172.21.15.97:0/1445116502 learned_addr learned my addr 172.21.15.97:0/1445116502 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98f7fe700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9901f79e0 0x7ff99015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98e7fc700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990157800 0x7ff99015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 -- 172.21.15.97:0/1445116502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9901f79e0 msgr2=0x7ff99015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9901f79e0 0x7ff99015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 -- 172.21.15.97:0/1445116502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990157800 msgr2=0x7ff99015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990157800 0x7ff99015b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 -- 172.21.15.97:0/1445116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9901f6f00 con 0x7ff978004850 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff98effd700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff9901f2fd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff98400e820 tx=0x7ff98400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff9777fe700 1 -- 172.21.15.97:0/1445116502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9840095b0 con 0x7ff978004850 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff9777fe700 1 -- 172.21.15.97:0/1445116502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff984013680 con 0x7ff978004850 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9901f71e0 con 0x7ff978004850 2023-11-24T06:10:23.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff9777fe700 1 -- 172.21.15.97:0/1445116502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984013a30 con 0x7ff978004850 2023-11-24T06:10:23.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.672+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99014a4c0 con 0x7ff978004850 2023-11-24T06:10:23.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.673+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9901f6d60 con 0x7ff978004850 2023-11-24T06:10:23.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.674+0000 7ff9777fe700 1 -- 172.21.15.97:0/1445116502 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff984013b90 con 0x7ff978004850 2023-11-24T06:10:23.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.674+0000 7ff9777fe700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff97c06af60 0x7ff97c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:23.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.674+0000 7ff98e7fc700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff97c06af60 0x7ff97c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:23.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.674+0000 7ff9777fe700 1 -- 172.21.15.97:0/1445116502 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9840891b0 con 0x7ff978004850 2023-11-24T06:10:23.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.675+0000 7ff98e7fc700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff97c06af60 0x7ff97c06d420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff980001930 tx=0x7ff980007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:23.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.679+0000 7ff9777fe700 1 -- 172.21.15.97:0/1445116502 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9901f6d60 con 0x7ff978004850 2023-11-24T06:10:23.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.943+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff9901f6d60 con 0x7ff978004850 2023-11-24T06:10:23.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.944+0000 7ff9777fe700 1 -- 172.21.15.97:0/1445116502 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff9901f6d60 con 0x7ff978004850 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlijU/IxAAp5/+zEd6ER/JFqum0ptk0Q==", 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:10:23.947 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLftpLxAAH+Ew2GQv73YtKLzZJjSBew==", 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.948 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlGtxpKhAAiQm5ore3YYETyG+sIKzEcw==", 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.949 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBl05iyNhAAHZYZ5pGghTwe55n84iXXQw==", 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDAPWBlwipJFBAAGLCD8wGcYoSy4yM4RXb0/Q==", 2023-11-24T06:10:23.950 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDDPWBlgW7fExAAQuUSGJn+ZfHSMwSi5r1pQg==", 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:10:23.951 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDGPWBl3E8uExAAJKjzbRFyYie6dmnRrP7VVw==", 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.952 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJPWBl5gnRExAAbKbgF6DcdTSv8M84GXyONw==", 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDPPWBlmCfaFRAAsIQWHUnlgDQcRXLndlUZKg==", 2023-11-24T06:10:23.953 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSPWBlPxfqFRAARyMzgivfTfgmFxZRjp54JQ==", 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:10:23.954 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVPWBl8cjTFBAAsNfkNY4gGVTJwh+mMtADgg==", 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.955 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDYPWBljh8EFBAA4nVPoyEXigIDTUB6enXsXA==", 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA==", 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:10:23.956 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlOXAsBhAABOT6CW2RVj7Iv5XMPvQTgQ==", 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:10:23.957 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlJRe4AhAAGQVWmqQr/urD7AefOfmoZQ==", 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.958 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlwX2YCRAAIM9eB39aohf1xUdEvHr/tA==", 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-11-24T06:10:23.959 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlVBcHBhAAbu6IOCKCLUA+5APAYwbRUw==", 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlOezbHxAAlNbpLHDzRHvk2SXFvQoTXQ==", 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.960 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBla7myGxAAHEPHxXHfBzCeqi2nLjhK6g==", 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:23.962 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-11-24T06:10:23.962 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:10:23.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff97c06af60 msgr2=0x7ff97c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff97c06af60 0x7ff97c06d420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff980001930 tx=0x7ff980007000 comp rx=0 tx=0).stop 2023-11-24T06:10:23.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 msgr2=0x7ff9901f2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:23.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff9901f2fd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff98400e820 tx=0x7ff98400ebe0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 shutdown_connections 2023-11-24T06:10:23.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff97c06af60 0x7ff97c06d420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff9901f79e0 0x7ff99015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff978004850 0x7ff9901f2fd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 --2- 172.21.15.97:0/1445116502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990157800 0x7ff99015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:23.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 >> 172.21.15.97:0/1445116502 conn(0x7ff9900b5b50 msgr2=0x7ff99006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:23.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 shutdown_connections 2023-11-24T06:10:23.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:23.946+0000 7ff995355700 1 -- 172.21.15.97:0/1445116502 wait complete. 2023-11-24T06:10:23.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:24.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.082+0000 7fc7cf620700 1 Processor -- start 2023-11-24T06:10:24.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.083+0000 7fc7cf620700 1 -- start start 2023-11-24T06:10:24.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.083+0000 7fc7cf620700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc7c815bd60 0x7fc7c80d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.083+0000 7fc7cf620700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c80c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.083+0000 7fc7cf620700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80cc780 con 0x7fc7c815c140 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.083+0000 7fc7cf620700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80cc9b0 con 0x7fc7c815bd60 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.083+0000 7fc7cf620700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80ccc90 con 0x7fc7c80cf2b0 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cdbbd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c80c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cd3bc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc7c815bd60 0x7fc7c80d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cdbbd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c80c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:34858/0 (socket says 172.21.15.97:34858) 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7ccbbb700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fc7c815c140 0x7fc7c80cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:49592/0 (socket says 172.21.15.97:49592) 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cd3bc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc7c815bd60 0x7fc7c80d1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:41978/0 (socket says 172.21.15.97:41978) 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cdbbd700 1 -- 172.21.15.97:0/2039962383 learned_addr learned my addr 172.21.15.97:0/2039962383 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cdbbd700 1 -- 172.21.15.97:0/2039962383 >> v2:172.21.15.183:3300/0 conn(0x7fc7c815bd60 msgr2=0x7fc7c80d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7bffff700 1 -- 172.21.15.97:0/2039962383 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443447032 0 0) 0x7fc7c80cc780 con 0x7fc7c815c140 2023-11-24T06:10:24.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cdbbd700 1 --2- 172.21.15.97:0/2039962383 >> v2:172.21.15.183:3300/0 conn(0x7fc7c815bd60 0x7fc7c80d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cdbbd700 1 -- 172.21.15.97:0/2039962383 >> v1:172.21.15.183:6789/0 conn(0x7fc7c815c140 legacy=0x7fc7c80cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cdbbd700 1 -- 172.21.15.97:0/2039962383 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80ccf70 con 0x7fc7c80cf2b0 2023-11-24T06:10:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.084+0000 7fc7cdbbd700 1 --2- 172.21.15.97:0/2039962383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c80c57c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009e50 tx=0x7fc7c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e10a61cf5d2a4f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7bffff700 1 -- 172.21.15.97:0/2039962383 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000e070 con 0x7fc7c80cf2b0 2023-11-24T06:10:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7bffff700 1 -- 172.21.15.97:0/2039962383 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c000b910 con 0x7fc7c80cf2b0 2023-11-24T06:10:24.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7bffff700 1 -- 172.21.15.97:0/2039962383 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000bcc0 con 0x7fc7c80cf2b0 2023-11-24T06:10:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7cf620700 1 -- 172.21.15.97:0/2039962383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 msgr2=0x7fc7c80c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2039962383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c80c57c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009e50 tx=0x7fc7c000b330 comp rx=0 tx=0).stop 2023-11-24T06:10:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7cf620700 1 -- 172.21.15.97:0/2039962383 shutdown_connections 2023-11-24T06:10:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2039962383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c80c57c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2039962383 >> v2:172.21.15.183:3300/0 conn(0x7fc7c815bd60 0x7fc7c80d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7cf620700 1 -- 172.21.15.97:0/2039962383 >> 172.21.15.97:0/2039962383 conn(0x7fc7c8073130 msgr2=0x7fc7c809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:24.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7cf620700 1 -- 172.21.15.97:0/2039962383 shutdown_connections 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.085+0000 7fc7cf620700 1 -- 172.21.15.97:0/2039962383 wait complete. 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.086+0000 7fc7cf620700 1 Processor -- start 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.086+0000 7fc7cf620700 1 -- start start 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7cf620700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c8158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7cf620700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7c815bd60 0x7fc7c81f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7cf620700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c815c140 0x7fc7c8155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7cf620700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81f74c0 con 0x7fc7c815bd60 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7cf620700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8159940 con 0x7fc7c815c140 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7cf620700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8159380 con 0x7fc7c80cf2b0 2023-11-24T06:10:24.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7ccbbb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7c815bd60 0x7fc7c81f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7ccbbb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7c815bd60 0x7fc7c81f7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47826/0 (socket says 172.21.15.97:47826) 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7ccbbb700 1 -- 172.21.15.97:0/2397471257 learned_addr learned my addr 172.21.15.97:0/2397471257 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7cd3bc700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c8158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.087+0000 7fc7cdbbd700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c815c140 0x7fc7c8155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7ccbbb700 1 -- 172.21.15.97:0/2397471257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 msgr2=0x7fc7c8158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7ccbbb700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c8158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7ccbbb700 1 -- 172.21.15.97:0/2397471257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c815c140 msgr2=0x7fc7c8155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7ccbbb700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c815c140 0x7fc7c8155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7ccbbb700 1 -- 172.21.15.97:0/2397471257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c0009b30 con 0x7fc7c815bd60 2023-11-24T06:10:24.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7ccbbb700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7c815bd60 0x7fc7c81f7950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c550 tx=0x7fc7c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7bdffb700 1 -- 172.21.15.97:0/2397471257 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4004c80 con 0x7fc7c815bd60 2023-11-24T06:10:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7bdffb700 1 -- 172.21.15.97:0/2397471257 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4004e00 con 0x7fc7c815bd60 2023-11-24T06:10:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c8159660 con 0x7fc7c815bd60 2023-11-24T06:10:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7bdffb700 1 -- 172.21.15.97:0/2397471257 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4007310 con 0x7fc7c815bd60 2023-11-24T06:10:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.088+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81f7e90 con 0x7fc7c815bd60 2023-11-24T06:10:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.089+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c8067f60 con 0x7fc7c815bd60 2023-11-24T06:10:24.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.089+0000 7fc7bdffb700 1 -- 172.21.15.97:0/2397471257 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc7c4007470 con 0x7fc7c815bd60 2023-11-24T06:10:24.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.090+0000 7fc7bdffb700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7ac06af60 0x7fc7ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.090+0000 7fc7cd3bc700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7ac06af60 0x7fc7ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.090+0000 7fc7bdffb700 1 -- 172.21.15.97:0/2397471257 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7c4089160 con 0x7fc7c815bd60 2023-11-24T06:10:24.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.090+0000 7fc7cd3bc700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7ac06af60 0x7fc7ac06d420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc7c806b180 tx=0x7fc7b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.094+0000 7fc7bdffb700 1 -- 172.21.15.97:0/2397471257 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7c4054470 con 0x7fc7c815bd60 2023-11-24T06:10:24.186 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:10:24.188 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:10:24.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.291+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7c81f7600 con 0x7fc7c815bd60 2023-11-24T06:10:24.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.292+0000 7fc7bdffb700 1 -- 172.21.15.97:0/2397471257 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12869 (secure 0 0 0) 0x7fc7c4057a90 con 0x7fc7c815bd60 2023-11-24T06:10:24.295 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:24.295 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":41,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:23.522621+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:24.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7ac06af60 msgr2=0x7fc7ac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7ac06af60 0x7fc7ac06d420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc7c806b180 tx=0x7fc7b8008040 comp rx=0 tx=0).stop 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7c815bd60 msgr2=0x7fc7c81f7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7c815bd60 0x7fc7c81f7950 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c550 tx=0x7fc7c400c910 comp rx=0 tx=0).stop 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 shutdown_connections 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7c815c140 0x7fc7c8155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7ac06af60 0x7fc7ac06d420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7c80cf2b0 0x7fc7c8158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 --2- 172.21.15.97:0/2397471257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7c815bd60 0x7fc7c81f7950 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.294+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 >> 172.21.15.97:0/2397471257 conn(0x7fc7c8073130 msgr2=0x7fc7c815e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.295+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 shutdown_connections 2023-11-24T06:10:24.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.295+0000 7fc7cf620700 1 -- 172.21.15.97:0/2397471257 wait complete. 2023-11-24T06:10:24.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:24.310 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.431+0000 7fb1181fc700 1 Processor -- start 2023-11-24T06:10:24.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.432+0000 7fb1181fc700 1 -- start start 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.432+0000 7fb1181fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1100ceb40 0x7fb11006bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.432+0000 7fb1181fc700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11006ba50 con 0x7fb1100b4310 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.432+0000 7fb1181fc700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11006b120 con 0x7fb11015e080 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.432+0000 7fb1181fc700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1100d1200 con 0x7fb1100ceb40 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.432+0000 7fb116799700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1100ceb40 0x7fb11006bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb115797700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb1100b4310 0x7fb1100ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40284/0 (socket says 172.21.15.97:40284) 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb115797700 1 -- 172.21.15.97:0/2723795012 learned_addr learned my addr 172.21.15.97:0/2723795012 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb116799700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1100ceb40 0x7fb11006bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47836/0 (socket says 172.21.15.97:47836) 2023-11-24T06:10:24.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb116799700 1 -- 172.21.15.97:0/2723795012 >> v1:172.21.15.183:6789/0 conn(0x7fb11015e080 legacy=0x7fb1100d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb116799700 1 -- 172.21.15.97:0/2723795012 >> v1:172.21.15.97:6789/0 conn(0x7fb1100b4310 legacy=0x7fb1100ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb116799700 1 -- 172.21.15.97:0/2723795012 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11006b400 con 0x7fb1100ceb40 2023-11-24T06:10:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb114f96700 1 -- 172.21.15.97:0/2723795012 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299420975 0 0) 0x7fb11006ba50 con 0x7fb1100b4310 2023-11-24T06:10:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb116799700 1 --2- 172.21.15.97:0/2723795012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100ceb40 0x7fb11006bee0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb108009d90 tx=0x7fb10800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d5784a6044a833 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb114f96700 1 -- 172.21.15.97:0/2723795012 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10800e070 con 0x7fb1100ceb40 2023-11-24T06:10:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb114f96700 1 -- 172.21.15.97:0/2723795012 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10800bb90 con 0x7fb1100ceb40 2023-11-24T06:10:24.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.433+0000 7fb114f96700 1 -- 172.21.15.97:0/2723795012 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108003a20 con 0x7fb1100ceb40 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.434+0000 7fb1181fc700 1 -- 172.21.15.97:0/2723795012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100ceb40 msgr2=0x7fb11006bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.434+0000 7fb1181fc700 1 --2- 172.21.15.97:0/2723795012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100ceb40 0x7fb11006bee0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb108009d90 tx=0x7fb10800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.434+0000 7fb1181fc700 1 -- 172.21.15.97:0/2723795012 shutdown_connections 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.434+0000 7fb1181fc700 1 --2- 172.21.15.97:0/2723795012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100ceb40 0x7fb11006bee0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.434+0000 7fb1181fc700 1 -- 172.21.15.97:0/2723795012 >> 172.21.15.97:0/2723795012 conn(0x7fb1100b3780 msgr2=0x7fb1100b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.434+0000 7fb1181fc700 1 -- 172.21.15.97:0/2723795012 shutdown_connections 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.434+0000 7fb1181fc700 1 -- 172.21.15.97:0/2723795012 wait complete. 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.435+0000 7fb1181fc700 1 Processor -- start 2023-11-24T06:10:24.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.435+0000 7fb1181fc700 1 -- start start 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1181fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100b4310 0x7fb11015fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1181fc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1100ceb40 0x7fb1100caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1181fc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11015e080 0x7fb110160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1181fc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1101f72c0 con 0x7fb1100b4310 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1181fc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11014fd20 con 0x7fb11015e080 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1181fc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb110150000 con 0x7fb1100ceb40 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100b4310 0x7fb11015fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100b4310 0x7fb11015fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47838/0 (socket says 172.21.15.97:47838) 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 -- 172.21.15.97:0/3319578600 learned_addr learned my addr 172.21.15.97:0/3319578600 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115797700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1100ceb40 0x7fb1100caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb116799700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11015e080 0x7fb110160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 -- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1100ceb40 msgr2=0x7fb1100caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1100ceb40 0x7fb1100caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 -- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11015e080 msgr2=0x7fb110160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11015e080 0x7fb110160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 -- 172.21.15.97:0/3319578600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb108009a70 con 0x7fb1100b4310 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb116799700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11015e080 0x7fb110160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115797700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1100ceb40 0x7fb1100caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb115f98700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100b4310 0x7fb11015fe10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00b4a0 tx=0x7fb10c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1027fc700 1 -- 172.21.15.97:0/3319578600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c011830 con 0x7fb1100b4310 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1027fc700 1 -- 172.21.15.97:0/3319578600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c011990 con 0x7fb1100b4310 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1101502e0 con 0x7fb1100b4310 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1027fc700 1 -- 172.21.15.97:0/3319578600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c011d40 con 0x7fb1100b4310 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.436+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb110160350 con 0x7fb1100b4310 2023-11-24T06:10:24.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.437+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb110068fb0 con 0x7fb1100b4310 2023-11-24T06:10:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.438+0000 7fb1027fc700 1 -- 172.21.15.97:0/3319578600 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb10c004020 con 0x7fb1100b4310 2023-11-24T06:10:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.438+0000 7fb1027fc700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb0f406af60 0x7fb0f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.438+0000 7fb1027fc700 1 -- 172.21.15.97:0/3319578600 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb10c089480 con 0x7fb1100b4310 2023-11-24T06:10:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.439+0000 7fb115797700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb0f406af60 0x7fb0f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.439+0000 7fb115797700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb0f406af60 0x7fb0f406d420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb104000f60 tx=0x7fb104009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.442+0000 7fb1027fc700 1 -- 172.21.15.97:0/3319578600 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb10c054910 con 0x7fb1100b4310 2023-11-24T06:10:24.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.632+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1100b7ab0 con 0x7fb1100b4310 2023-11-24T06:10:24.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.633+0000 7fb1027fc700 1 -- 172.21.15.97:0/3319578600 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb10c087e60 con 0x7fb1100b4310 2023-11-24T06:10:24.636 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb0f406af60 msgr2=0x7fb0f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb0f406af60 0x7fb0f406d420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb104000f60 tx=0x7fb104009040 comp rx=0 tx=0).stop 2023-11-24T06:10:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100b4310 msgr2=0x7fb11015fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100b4310 0x7fb11015fe10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00b4a0 tx=0x7fb10c00b860 comp rx=0 tx=0).stop 2023-11-24T06:10:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 shutdown_connections 2023-11-24T06:10:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb11015e080 0x7fb110160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1100b4310 0x7fb11015fe10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb0f406af60 0x7fb0f406d420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 --2- 172.21.15.97:0/3319578600 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb1100ceb40 0x7fb1100caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 >> 172.21.15.97:0/3319578600 conn(0x7fb1100b3780 msgr2=0x7fb11006ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:24.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 shutdown_connections 2023-11-24T06:10:24.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.635+0000 7fb1181fc700 1 -- 172.21.15.97:0/3319578600 wait complete. 2023-11-24T06:10:24.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:10:24.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.768+0000 7f3345b7c700 1 Processor -- start 2023-11-24T06:10:24.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.768+0000 7f3345b7c700 1 -- start start 2023-11-24T06:10:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f3345b7c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f33400b3370 0x7f33401604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f3345b7c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400d1c70 0x7f33400d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f3345b7c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33400b22d0 con 0x7f334015bdd0 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f3345b7c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33400b2f50 con 0x7f33400b3370 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f3345b7c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400d2640 con 0x7f33400d1c70 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f334015bdd0 0x7f33400b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40298/0 (socket says 172.21.15.97:40298) 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333f7fe700 1 -- 172.21.15.97:0/1512626671 learned_addr learned my addr 172.21.15.97:0/1512626671 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333ffff700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400d1c70 0x7f33400d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333effd700 1 --2- 172.21.15.97:0/1512626671 >> v2:172.21.15.183:3300/0 conn(0x7f33400b3370 0x7f33401604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231158213 0 0) 0x7f33400b22d0 con 0x7f334015bdd0 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f332c0035b0 con 0x7f334015bdd0 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 423712868 0 0) 0x7f332c0035b0 con 0x7f334015bdd0 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400d1c70 msgr2=0x7f33400d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400d1c70 0x7f33400d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 >> v2:172.21.15.183:3300/0 conn(0x7f33400b3370 msgr2=0x7f33401604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 --2- 172.21.15.97:0/1512626671 >> v2:172.21.15.183:3300/0 conn(0x7f33400b3370 0x7f33401604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400b25b0 con 0x7f334015bdd0 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f3330001670 con 0x7f334015bdd0 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 >> v1:172.21.15.97:6789/0 conn(0x7f334015bdd0 legacy=0x7f33400b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.769+0000 7f333e7fc700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f332c003a00 0x7f332c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333e7fc700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400d1c70 0x7f332c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333e7fc700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c004d90 0x7f332c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400b2f50 con 0x7f332c004d90 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400d2640 con 0x7f332c003a00 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c00ba10 con 0x7f33400d1c70 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333ffff700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c004d90 0x7f332c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.776 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333effd700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400d1c70 0x7f332c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333f7fe700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f332c003a00 0x7f332c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333ffff700 1 -- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400d1c70 msgr2=0x7f332c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333ffff700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400d1c70 0x7f332c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333ffff700 1 -- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f332c003a00 msgr2=0x7f332c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333ffff700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f332c003a00 0x7f332c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333ffff700 1 -- 172.21.15.97:0/1512626671 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0035b0 con 0x7f332c004d90 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333ffff700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c004d90 0x7f332c005210 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3338009fc0 tx=0x7f332c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada2e0e78a2e38b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333800ec30 con 0x7f332c004d90 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333800f040 con 0x7f332c004d90 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.770+0000 7f333e7fc700 1 -- 172.21.15.97:0/1512626671 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338014480 con 0x7f332c004d90 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.771+0000 7f3345b7c700 1 -- 172.21.15.97:0/1512626671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c004d90 msgr2=0x7f332c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.771+0000 7f3345b7c700 1 --2- 172.21.15.97:0/1512626671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f332c004d90 0x7f332c005210 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3338009fc0 tx=0x7f332c005880 comp rx=0 tx=0).stop 2023-11-24T06:10:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.771+0000 7f333ffff700 1 -- 172.21.15.97:0/1512626671 reap_dead start 2023-11-24T06:10:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.771+0000 7f3345b7c700 1 -- 172.21.15.97:0/1512626671 shutdown_connections 2023-11-24T06:10:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.771+0000 7f3345b7c700 1 -- 172.21.15.97:0/1512626671 >> 172.21.15.97:0/1512626671 conn(0x7f33400731a0 msgr2=0x7f334009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.771+0000 7f3345b7c700 1 -- 172.21.15.97:0/1512626671 shutdown_connections 2023-11-24T06:10:24.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.771+0000 7f3345b7c700 1 -- 172.21.15.97:0/1512626671 wait complete. 2023-11-24T06:10:24.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.772+0000 7f3345b7c700 1 Processor -- start 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.772+0000 7f3345b7c700 1 -- start start 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f3345b7c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400cd710 0x7f33400cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f3345b7c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33400c8db0 0x7f33400c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f3345b7c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33400b50e0 0x7f33400b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f3345b7c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400ce030 con 0x7f33400b50e0 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f3345b7c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400c9a50 con 0x7f33400c8db0 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f3345b7c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400ca0a0 con 0x7f33400cd710 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33400b50e0 0x7f33400b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33400b50e0 0x7f33400b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47862/0 (socket says 172.21.15.97:47862) 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 -- 172.21.15.97:0/4187015549 learned_addr learned my addr 172.21.15.97:0/4187015549 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:24.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333f7fe700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400cd710 0x7f33400cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333effd700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33400c8db0 0x7f33400c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 -- 172.21.15.97:0/4187015549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400cd710 msgr2=0x7f33400cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400cd710 0x7f33400cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 -- 172.21.15.97:0/4187015549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33400c8db0 msgr2=0x7f33400c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33400c8db0 0x7f33400c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 -- 172.21.15.97:0/4187015549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400ca380 con 0x7f33400b50e0 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.773+0000 7f333ffff700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33400b50e0 0x7f33400b5560 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3338016000 tx=0x7f3338009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.774+0000 7f3337fff700 1 -- 172.21.15.97:0/4187015549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338007000 con 0x7f33400b50e0 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.774+0000 7f3337fff700 1 -- 172.21.15.97:0/4187015549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3338022070 con 0x7f33400b50e0 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.774+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33400ca660 con 0x7f33400b50e0 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.774+0000 7f3337fff700 1 -- 172.21.15.97:0/4187015549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338029480 con 0x7f33400b50e0 2023-11-24T06:10:24.782 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.774+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33400c9d30 con 0x7f33400b50e0 2023-11-24T06:10:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.775+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33400ca9d0 con 0x7f33400b50e0 2023-11-24T06:10:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.775+0000 7f3337fff700 1 -- 172.21.15.97:0/4187015549 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f333801e430 con 0x7f33400b50e0 2023-11-24T06:10:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.775+0000 7f3337fff700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006af60 0x7f332006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.775+0000 7f3337fff700 1 -- 172.21.15.97:0/4187015549 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3338024070 con 0x7f33400b50e0 2023-11-24T06:10:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.776+0000 7f333f7fe700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006af60 0x7f332006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.776+0000 7f333f7fe700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006af60 0x7f332006d420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3330001cf0 tx=0x7f3330007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:24.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.780+0000 7f3337fff700 1 -- 172.21.15.97:0/4187015549 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33380567b0 con 0x7f33400b50e0 2023-11-24T06:10:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.973+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3340067f60 con 0x7f33400b50e0 2023-11-24T06:10:24.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.973+0000 7f3337fff700 1 -- 172.21.15.97:0/4187015549 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3338059dd0 con 0x7f33400b50e0 2023-11-24T06:10:24.976 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:24.976 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:10:24.976 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:24.977 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-11-24T06:10:24.977 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-11-24T06:10:24.977 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:24.977 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006af60 msgr2=0x7f332006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006af60 0x7f332006d420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3330001cf0 tx=0x7f3330007980 comp rx=0 tx=0).stop 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33400b50e0 msgr2=0x7f33400b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33400b50e0 0x7f33400b5560 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3338016000 tx=0x7f3338009750 comp rx=0 tx=0).stop 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 shutdown_connections 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33400b50e0 0x7f33400b5560 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f332006af60 0x7f332006d420 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33400cd710 0x7f33400cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 --2- 172.21.15.97:0/4187015549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33400c8db0 0x7f33400c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 >> 172.21.15.97:0/4187015549 conn(0x7f33400731a0 msgr2=0x7f334015a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:24.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.975+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 shutdown_connections 2023-11-24T06:10:24.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:24.976+0000 7f3345b7c700 1 -- 172.21.15.97:0/4187015549 wait complete. 2023-11-24T06:10:24.990 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:10:24.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:10:25.114 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.112+0000 7fa299062700 1 Processor -- start 2023-11-24T06:10:25.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.112+0000 7fa299062700 1 -- start start 2023-11-24T06:10:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.112+0000 7fa299062700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa2940b01e0 0x7fa29406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.112+0000 7fa299062700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.112+0000 7fa299062700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2940b05c0 con 0x7fa2940b59c0 2023-11-24T06:10:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.112+0000 7fa299062700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2941470a0 con 0x7fa2940b01e0 2023-11-24T06:10:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.112+0000 7fa299062700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940b5e70 con 0x7fa2940746a0 2023-11-24T06:10:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa29259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa2940b59c0 0x7fa294073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40318/0 (socket says 172.21.15.97:40318) 2023-11-24T06:10:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa29259c700 1 -- 172.21.15.97:0/2274846818 learned_addr learned my addr 172.21.15.97:0/2274846818 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa29359e700 1 --2- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa292d9d700 1 --2- 172.21.15.97:0/2274846818 >> v2:172.21.15.183:3300/0 conn(0x7fa2940b01e0 0x7fa29406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa291d9b700 1 -- 172.21.15.97:0/2274846818 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687065927 0 0) 0x7fa2940b05c0 con 0x7fa2940b59c0 2023-11-24T06:10:25.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa291d9b700 1 -- 172.21.15.97:0/2274846818 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa27c0035b0 con 0x7fa2940b59c0 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa292d9d700 1 -- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 msgr2=0x7fa2940b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa292d9d700 1 --2- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa292d9d700 1 -- 172.21.15.97:0/2274846818 >> v1:172.21.15.97:6789/0 conn(0x7fa2940b59c0 legacy=0x7fa294073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa292d9d700 1 -- 172.21.15.97:0/2274846818 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2940b4250 con 0x7fa2940b01e0 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa29359e700 1 --2- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa292d9d700 1 --2- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 0x7fa29406e9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa284009870 tx=0x7fa284009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae974b8e636882f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.113+0000 7fa291d9b700 1 -- 172.21.15.97:0/2274846818 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284011830 con 0x7fa2940b01e0 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa291d9b700 1 -- 172.21.15.97:0/2274846818 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa284011990 con 0x7fa2940b01e0 2023-11-24T06:10:25.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa291d9b700 1 -- 172.21.15.97:0/2274846818 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284011d00 con 0x7fa2940b01e0 2023-11-24T06:10:25.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa299062700 1 -- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 msgr2=0x7fa29406e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa299062700 1 --2- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 0x7fa29406e9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa284009870 tx=0x7fa284009b80 comp rx=0 tx=0).stop 2023-11-24T06:10:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa299062700 1 -- 172.21.15.97:0/2274846818 shutdown_connections 2023-11-24T06:10:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa299062700 1 --2- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa299062700 1 --2- 172.21.15.97:0/2274846818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 0x7fa29406e9c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa299062700 1 -- 172.21.15.97:0/2274846818 >> 172.21.15.97:0/2274846818 conn(0x7fa2940d0b40 msgr2=0x7fa2940d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa299062700 1 -- 172.21.15.97:0/2274846818 shutdown_connections 2023-11-24T06:10:25.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.114+0000 7fa299062700 1 -- 172.21.15.97:0/2274846818 wait complete. 2023-11-24T06:10:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.115+0000 7fa299062700 1 Processor -- start 2023-11-24T06:10:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa299062700 1 -- start start 2023-11-24T06:10:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa299062700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa299062700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 0x7fa29413f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa299062700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940b59c0 0x7fa29413e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa299062700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2940cfc80 con 0x7fa2940b59c0 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa299062700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa294151c20 con 0x7fa2940b01e0 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa299062700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa294151f00 con 0x7fa2940746a0 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa29359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940b59c0 0x7fa29413e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa29359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940b59c0 0x7fa29413e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47876/0 (socket says 172.21.15.97:47876) 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.116+0000 7fa29359e700 1 -- 172.21.15.97:0/1175913875 learned_addr learned my addr 172.21.15.97:0/1175913875 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa29259c700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 0x7fa29413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa292d9d700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa29359e700 1 -- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 msgr2=0x7fa2940cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa29359e700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa29359e700 1 -- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 msgr2=0x7fa29413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa29359e700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 0x7fa29413f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa29359e700 1 -- 172.21.15.97:0/1175913875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c009a70 con 0x7fa2940b59c0 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa292d9d700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa29259c700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 0x7fa29413f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa29359e700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940b59c0 0x7fa29413e780 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00efc0 tx=0x7fa28c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa27b7fe700 1 -- 172.21.15.97:0/1175913875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c009cd0 con 0x7fa2940b59c0 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa27b7fe700 1 -- 172.21.15.97:0/1175913875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28c00f040 con 0x7fa2940b59c0 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa27b7fe700 1 -- 172.21.15.97:0/1175913875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa28c00cd60 con 0x7fa2940b59c0 2023-11-24T06:10:25.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2840094a0 con 0x7fa2940b59c0 2023-11-24T06:10:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.117+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2941524c0 con 0x7fa2940b59c0 2023-11-24T06:10:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.118+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa294152df0 con 0x7fa2940b59c0 2023-11-24T06:10:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.118+0000 7fa27b7fe700 1 -- 172.21.15.97:0/1175913875 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa28c014480 con 0x7fa2940b59c0 2023-11-24T06:10:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.119+0000 7fa27b7fe700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa28006af60 0x7fa28006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.119+0000 7fa27b7fe700 1 -- 172.21.15.97:0/1175913875 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa28c089be0 con 0x7fa2940b59c0 2023-11-24T06:10:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.119+0000 7fa292d9d700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa28006af60 0x7fa28006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:25.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.119+0000 7fa292d9d700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa28006af60 0x7fa28006d420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa2840090b0 tx=0x7fa284008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:25.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.123+0000 7fa27b7fe700 1 -- 172.21.15.97:0/1175913875 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa28c054de0 con 0x7fa2940b59c0 2023-11-24T06:10:25.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:25.316+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa294068670 con 0x7fa2940b59c0 2023-11-24T06:10:26.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.173+0000 7fa27b7fe700 1 -- 172.21.15.97:0/1175913875 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa28c058400 con 0x7fa2940b59c0 2023-11-24T06:10:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa28006af60 msgr2=0x7fa28006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa28006af60 0x7fa28006d420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa2840090b0 tx=0x7fa284008030 comp rx=0 tx=0).stop 2023-11-24T06:10:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940b59c0 msgr2=0x7fa29413e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940b59c0 0x7fa29413e780 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00efc0 tx=0x7fa28c00c3a0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 shutdown_connections 2023-11-24T06:10:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2940b59c0 0x7fa29413e780 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa28006af60 0x7fa28006d420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa2940746a0 0x7fa2940cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 --2- 172.21.15.97:0/1175913875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2940b01e0 0x7fa29413f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.177+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 >> 172.21.15.97:0/1175913875 conn(0x7fa2940d0b40 msgr2=0x7fa294145040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.178+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 shutdown_connections 2023-11-24T06:10:26.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.178+0000 7fa299062700 1 -- 172.21.15.97:0/1175913875 wait complete. 2023-11-24T06:10:26.182 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-11-24T06:10:26.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:10:26.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.327+0000 7f59068af700 1 Processor -- start 2023-11-24T06:10:26.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.328+0000 7f59068af700 1 -- start start 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.328+0000 7f59068af700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f590015e160 0x7f59000b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.328+0000 7f59068af700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f59000afc90 0x7f5900073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.328+0000 7f59068af700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f59000744d0 0x7f590006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.328+0000 7f59068af700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59000b4500 con 0x7f590015e160 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.328+0000 7f59068af700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5900074960 con 0x7f59000afc90 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.328+0000 7f59068af700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59000b3c60 con 0x7f59000744d0 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f590015e160 0x7f59000b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f590015e160 0x7f59000b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47880/0 (socket says 172.21.15.97:47880) 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 -- 172.21.15.97:0/1785392870 learned_addr learned my addr 172.21.15.97:0/1785392870 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:26.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f5904e4c700 1 --2- 172.21.15.97:0/1785392870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f59000744d0 0x7f590006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 -- 172.21.15.97:0/1785392870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f59000744d0 msgr2=0x7f590006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 --2- 172.21.15.97:0/1785392870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f59000744d0 0x7f590006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58ff7fe700 1 --2- 172.21.15.97:0/1785392870 >> v2:172.21.15.183:3300/0 conn(0x7f59000afc90 0x7f5900073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 -- 172.21.15.97:0/1785392870 >> v2:172.21.15.183:3300/0 conn(0x7f59000afc90 msgr2=0x7f5900073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 --2- 172.21.15.97:0/1785392870 >> v2:172.21.15.183:3300/0 conn(0x7f59000afc90 0x7f5900073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 -- 172.21.15.97:0/1785392870 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000b58c0 con 0x7f590015e160 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58fffff700 1 --2- 172.21.15.97:0/1785392870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f590015e160 0x7f59000b5da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f59000d0880 tx=0x7f58f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5e2d0a4e492a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58feffd700 1 -- 172.21.15.97:0/1785392870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400b870 con 0x7f590015e160 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58feffd700 1 -- 172.21.15.97:0/1785392870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f400b9d0 con 0x7f590015e160 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.329+0000 7f58feffd700 1 -- 172.21.15.97:0/1785392870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f400bd60 con 0x7f590015e160 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 -- 172.21.15.97:0/1785392870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f590015e160 msgr2=0x7f59000b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 --2- 172.21.15.97:0/1785392870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f590015e160 0x7f59000b5da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f59000d0880 tx=0x7f58f4009bd0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 -- 172.21.15.97:0/1785392870 shutdown_connections 2023-11-24T06:10:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 --2- 172.21.15.97:0/1785392870 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f59000744d0 0x7f590006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 --2- 172.21.15.97:0/1785392870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f590015e160 0x7f59000b5da0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 --2- 172.21.15.97:0/1785392870 >> v2:172.21.15.183:3300/0 conn(0x7f59000afc90 0x7f5900073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 -- 172.21.15.97:0/1785392870 >> 172.21.15.97:0/1785392870 conn(0x7f59000d0bf0 msgr2=0x7f59000d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 -- 172.21.15.97:0/1785392870 shutdown_connections 2023-11-24T06:10:26.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.330+0000 7f59068af700 1 -- 172.21.15.97:0/1785392870 wait complete. 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.331+0000 7f59068af700 1 Processor -- start 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.331+0000 7f59068af700 1 -- start start 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f59068af700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59000744d0 0x7f590014c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f59068af700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59000afc90 0x7f590015e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f59068af700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f590015e160 0x7f590015eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f59068af700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900152240 con 0x7f59000afc90 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f59068af700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590014ddd0 con 0x7f59000744d0 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f59068af700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590015f170 con 0x7f590015e160 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59000afc90 0x7f590015e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59000afc90 0x7f590015e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47886/0 (socket says 172.21.15.97:47886) 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 -- 172.21.15.97:0/2678842280 learned_addr learned my addr 172.21.15.97:0/2678842280 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f5904e4c700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f590015e160 0x7f590015eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58fffff700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59000744d0 0x7f590014c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 -- 172.21.15.97:0/2678842280 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f590015e160 msgr2=0x7f590015eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f590015e160 0x7f590015eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 -- 172.21.15.97:0/2678842280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59000744d0 msgr2=0x7f590014c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59000744d0 0x7f590014c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 -- 172.21.15.97:0/2678842280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4009550 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.332+0000 7f58ff7fe700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59000afc90 0x7f590015e5b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c7f0 tx=0x7f58f000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.333+0000 7f58fcff9700 1 -- 172.21.15.97:0/2678842280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0004220 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.333+0000 7f58fcff9700 1 -- 172.21.15.97:0/2678842280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0004380 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.333+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590015f450 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.333+0000 7f58fcff9700 1 -- 172.21.15.97:0/2678842280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f00036a0 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.333+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590015fa10 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.334+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5900068fb0 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.334+0000 7f58fcff9700 1 -- 172.21.15.97:0/2678842280 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f58f0003800 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.335+0000 7f58fcff9700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58ec06af60 0x7f58ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.335+0000 7f58fcff9700 1 -- 172.21.15.97:0/2678842280 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f58f0088fe0 con 0x7f59000afc90 2023-11-24T06:10:26.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.335+0000 7f58fffff700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58ec06af60 0x7f58ec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:26.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.335+0000 7f58fffff700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58ec06af60 0x7f58ec06d420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f58f4014040 tx=0x7f58f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:26.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.339+0000 7f58fcff9700 1 -- 172.21.15.97:0/2678842280 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58f00bbe90 con 0x7f59000afc90 2023-11-24T06:10:26.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:26.529+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f590006c140 con 0x7f59000afc90 2023-11-24T06:10:27.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.177+0000 7f58fcff9700 1 -- 172.21.15.97:0/2678842280 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7f58f001e0d0 con 0x7f59000afc90 2023-11-24T06:10:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.179+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58ec06af60 msgr2=0x7f58ec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.179+0000 7f59068af700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58ec06af60 0x7f58ec06d420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f58f4014040 tx=0x7f58f4008030 comp rx=0 tx=0).stop 2023-11-24T06:10:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.179+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59000afc90 msgr2=0x7f590015e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.179+0000 7f59068af700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59000afc90 0x7f590015e5b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c7f0 tx=0x7f58f000cb00 comp rx=0 tx=0).stop 2023-11-24T06:10:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.180+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 shutdown_connections 2023-11-24T06:10:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.180+0000 7f59068af700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58ec06af60 0x7f58ec06d420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.180+0000 7f59068af700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f590015e160 0x7f590015eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.180+0000 7f59068af700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59000744d0 0x7f590014c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.180+0000 7f59068af700 1 --2- 172.21.15.97:0/2678842280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59000afc90 0x7f590015e5b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.180+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 >> 172.21.15.97:0/2678842280 conn(0x7f59000d0bf0 msgr2=0x7f590006eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:27.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.180+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 shutdown_connections 2023-11-24T06:10:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.180+0000 7f59068af700 1 -- 172.21.15.97:0/2678842280 wait complete. 2023-11-24T06:10:27.185 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-11-24T06:10:27.205 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:10:27.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.326+0000 7f51c8de7700 1 Processor -- start 2023-11-24T06:10:27.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.326+0000 7f51c8de7700 1 -- start start 2023-11-24T06:10:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c8de7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40b5da0 0x7f51c40afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c8de7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40b3c60 con 0x7f51c415e160 2023-11-24T06:10:27.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c8de7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40b58c0 con 0x7f51c40746a0 2023-11-24T06:10:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c8de7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40b4500 con 0x7f51c40b5da0 2023-11-24T06:10:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f51c415e160 0x7f51c4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40346/0 (socket says 172.21.15.97:40346) 2023-11-24T06:10:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c259c700 1 -- 172.21.15.97:0/4286939954 learned_addr learned my addr 172.21.15.97:0/4286939954 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c2d9d700 1 --2- 172.21.15.97:0/4286939954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40b5da0 0x7f51c40afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c159a700 1 -- 172.21.15.97:0/4286939954 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950314826 0 0) 0x7f51c40b3c60 con 0x7f51c415e160 2023-11-24T06:10:27.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.327+0000 7f51c159a700 1 -- 172.21.15.97:0/4286939954 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51ac0035b0 con 0x7f51c415e160 2023-11-24T06:10:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c159a700 1 -- 172.21.15.97:0/4286939954 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970725503 0 0) 0x7f51c40b58c0 con 0x7f51c40746a0 2023-11-24T06:10:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c2d9d700 1 -- 172.21.15.97:0/4286939954 >> v1:172.21.15.183:6789/0 conn(0x7f51c40746a0 legacy=0x7f51c406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c2d9d700 1 -- 172.21.15.97:0/4286939954 >> v1:172.21.15.97:6789/0 conn(0x7f51c415e160 legacy=0x7f51c4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c2d9d700 1 -- 172.21.15.97:0/4286939954 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c406bdc0 con 0x7f51c40b5da0 2023-11-24T06:10:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c2d9d700 1 --2- 172.21.15.97:0/4286939954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40b5da0 0x7f51c40afc90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009dc0 tx=0x7f51bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34cbc72389d13fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:27.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c159a700 1 -- 172.21.15.97:0/4286939954 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00e070 con 0x7f51c40b5da0 2023-11-24T06:10:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c159a700 1 -- 172.21.15.97:0/4286939954 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc003a20 con 0x7f51c40b5da0 2023-11-24T06:10:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c159a700 1 -- 172.21.15.97:0/4286939954 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b8001670 con 0x7f51c40b5da0 2023-11-24T06:10:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c8de7700 1 -- 172.21.15.97:0/4286939954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40b5da0 msgr2=0x7f51c40afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.328+0000 7f51c8de7700 1 --2- 172.21.15.97:0/4286939954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40b5da0 0x7f51c40afc90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009dc0 tx=0x7f51bc00b330 comp rx=0 tx=0).stop 2023-11-24T06:10:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.329+0000 7f51c8de7700 1 -- 172.21.15.97:0/4286939954 shutdown_connections 2023-11-24T06:10:27.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.329+0000 7f51c8de7700 1 --2- 172.21.15.97:0/4286939954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40b5da0 0x7f51c40afc90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.329+0000 7f51c8de7700 1 -- 172.21.15.97:0/4286939954 >> 172.21.15.97:0/4286939954 conn(0x7f51c40d0bf0 msgr2=0x7f51c40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.329+0000 7f51c8de7700 1 -- 172.21.15.97:0/4286939954 shutdown_connections 2023-11-24T06:10:27.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.329+0000 7f51c8de7700 1 -- 172.21.15.97:0/4286939954 wait complete. 2023-11-24T06:10:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.330+0000 7f51c8de7700 1 Processor -- start 2023-11-24T06:10:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.330+0000 7f51c8de7700 1 -- start start 2023-11-24T06:10:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c8de7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40746a0 0x7f51c414f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c8de7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c40b5da0 0x7f51c4160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c8de7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c415e160 0x7f51c415e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c8de7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c414f900 con 0x7f51c415e160 2023-11-24T06:10:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c8de7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40b26f0 con 0x7f51c40b5da0 2023-11-24T06:10:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c8de7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c415f2c0 con 0x7f51c40746a0 2023-11-24T06:10:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c415e160 0x7f51c415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c2d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c415e160 0x7f51c415e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47904/0 (socket says 172.21.15.97:47904) 2023-11-24T06:10:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c2d9d700 1 -- 172.21.15.97:0/3997052004 learned_addr learned my addr 172.21.15.97:0/3997052004 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c259c700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40746a0 0x7f51c414f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.331+0000 7f51c1d9b700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c40b5da0 0x7f51c4160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c2d9d700 1 -- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40746a0 msgr2=0x7f51c414f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c2d9d700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40746a0 0x7f51c414f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c2d9d700 1 -- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c40b5da0 msgr2=0x7f51c4160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:27.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c2d9d700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c40b5da0 0x7f51c4160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:27.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c2d9d700 1 -- 172.21.15.97:0/3997052004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc009a70 con 0x7f51c415e160 2023-11-24T06:10:27.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c1d9b700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c40b5da0 0x7f51c4160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:27.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c259c700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40746a0 0x7f51c414f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c2d9d700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c415e160 0x7f51c415e750 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f51c4072ed0 tx=0x7f51bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51b2ffd700 1 -- 172.21.15.97:0/3997052004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0047a0 con 0x7f51c415e160 2023-11-24T06:10:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51b2ffd700 1 -- 172.21.15.97:0/3997052004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc004940 con 0x7f51c415e160 2023-11-24T06:10:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c415f5a0 con 0x7f51c415e160 2023-11-24T06:10:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51b2ffd700 1 -- 172.21.15.97:0/3997052004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc012670 con 0x7f51c415e160 2023-11-24T06:10:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.332+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c415fb60 con 0x7f51c415e160 2023-11-24T06:10:27.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.333+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c4068fb0 con 0x7f51c415e160 2023-11-24T06:10:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.333+0000 7f51b2ffd700 1 -- 172.21.15.97:0/3997052004 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f51bc0127d0 con 0x7f51c415e160 2023-11-24T06:10:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.334+0000 7f51b2ffd700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51a406af60 0x7f51a406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.334+0000 7f51b2ffd700 1 -- 172.21.15.97:0/3997052004 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f51bc08b130 con 0x7f51c415e160 2023-11-24T06:10:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.334+0000 7f51c259c700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51a406af60 0x7f51a406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:27.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.334+0000 7f51c259c700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51a406af60 0x7f51a406d420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51b80017a0 tx=0x7f51b80016d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:27.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.339+0000 7f51b2ffd700 1 -- 172.21.15.97:0/3997052004 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51bc056280 con 0x7f51c415e160 2023-11-24T06:10:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:27.574+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f51c406c140 con 0x7f51c415e160 2023-11-24T06:10:28.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.499+0000 7f51b2ffd700 1 -- 172.21.15.97:0/3997052004 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f51bc0598a0 con 0x7f51c415e160 2023-11-24T06:10:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51a406af60 msgr2=0x7f51a406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51a406af60 0x7f51a406d420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f51b80017a0 tx=0x7f51b80016d0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c415e160 msgr2=0x7f51c415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c415e160 0x7f51c415e750 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f51c4072ed0 tx=0x7f51bc004ad0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 shutdown_connections 2023-11-24T06:10:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51c415e160 0x7f51c415e750 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f51a406af60 0x7f51a406d420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f51c40746a0 0x7f51c414f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 --2- 172.21.15.97:0/3997052004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51c40b5da0 0x7f51c4160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 >> 172.21.15.97:0/3997052004 conn(0x7f51c40d0bf0 msgr2=0x7f51c4073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 shutdown_connections 2023-11-24T06:10:28.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.503+0000 7f51c8de7700 1 -- 172.21.15.97:0/3997052004 wait complete. 2023-11-24T06:10:28.508 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 6 and data pool 7 2023-11-24T06:10:28.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:28.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.646+0000 7f65167e6700 1 Processor -- start 2023-11-24T06:10:28.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.647+0000 7f65167e6700 1 -- start start 2023-11-24T06:10:28.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.647+0000 7f65167e6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f651015e080 0x7f65100b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.647+0000 7f65167e6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65100af4d0 0x7f65100afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.647+0000 7f65167e6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65100ae430 con 0x7f651006c000 2023-11-24T06:10:28.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.647+0000 7f65167e6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65100af0b0 con 0x7f65100af4d0 2023-11-24T06:10:28.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.647+0000 7f65167e6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65100b05a0 con 0x7f651015e080 2023-11-24T06:10:28.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.647+0000 7f650f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65100af4d0 0x7f65100afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65100af4d0 0x7f65100afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47924/0 (socket says 172.21.15.97:47924) 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650f7fe700 1 -- 172.21.15.97:0/656335663 learned_addr learned my addr 172.21.15.97:0/656335663 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650ffff700 1 --2- 172.21.15.97:0/656335663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f651015e080 0x7f65100b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650f7fe700 1 -- 172.21.15.97:0/656335663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f651015e080 msgr2=0x7f65100b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650f7fe700 1 --2- 172.21.15.97:0/656335663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f651015e080 0x7f65100b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650f7fe700 1 -- 172.21.15.97:0/656335663 >> v1:172.21.15.183:6789/0 conn(0x7f651006c000 legacy=0x7f6510073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650f7fe700 1 -- 172.21.15.97:0/656335663 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65100ae710 con 0x7f65100af4d0 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650f7fe700 1 --2- 172.21.15.97:0/656335663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100af4d0 0x7f65100afb90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6500009920 tx=0x7f6500009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4505df7b75101999 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650effd700 1 -- 172.21.15.97:0/656335663 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500004230 con 0x7f65100af4d0 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650effd700 1 -- 172.21.15.97:0/656335663 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6500004390 con 0x7f65100af4d0 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.648+0000 7f650effd700 1 -- 172.21.15.97:0/656335663 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6500009e70 con 0x7f65100af4d0 2023-11-24T06:10:28.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.649+0000 7f65167e6700 1 -- 172.21.15.97:0/656335663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100af4d0 msgr2=0x7f65100afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.649+0000 7f65167e6700 1 --2- 172.21.15.97:0/656335663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100af4d0 0x7f65100afb90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6500009920 tx=0x7f6500009c30 comp rx=0 tx=0).stop 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.649+0000 7f65167e6700 1 -- 172.21.15.97:0/656335663 shutdown_connections 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.649+0000 7f65167e6700 1 --2- 172.21.15.97:0/656335663 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f651015e080 0x7f65100b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.649+0000 7f65167e6700 1 --2- 172.21.15.97:0/656335663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65100af4d0 0x7f65100afb90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.649+0000 7f65167e6700 1 -- 172.21.15.97:0/656335663 >> 172.21.15.97:0/656335663 conn(0x7f6510071b90 msgr2=0x7f6510071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.649+0000 7f65167e6700 1 -- 172.21.15.97:0/656335663 shutdown_connections 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.649+0000 7f65167e6700 1 -- 172.21.15.97:0/656335663 wait complete. 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.650+0000 7f65167e6700 1 Processor -- start 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.650+0000 7f65167e6700 1 -- start start 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f65167e6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f651006c000 0x7f651014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f65167e6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65100af4d0 0x7f651015f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f65167e6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f651015e080 0x7f6510156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f65167e6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651015fb20 con 0x7f651006c000 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f65167e6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651015fe00 con 0x7f651015e080 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f65167e6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65101600e0 con 0x7f65100af4d0 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f651006c000 0x7f651014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f651006c000 0x7f651014e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47928/0 (socket says 172.21.15.97:47928) 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 -- 172.21.15.97:0/281652966 learned_addr learned my addr 172.21.15.97:0/281652966 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650f7fe700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65100af4d0 0x7f651015f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f6514d83700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f651015e080 0x7f6510156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 -- 172.21.15.97:0/281652966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65100af4d0 msgr2=0x7f651015f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65100af4d0 0x7f651015f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 -- 172.21.15.97:0/281652966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f651015e080 msgr2=0x7f6510156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f651015e080 0x7f6510156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 -- 172.21.15.97:0/281652966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500009550 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.651+0000 7f650ffff700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f651006c000 0x7f651014e3c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f650400c7f0 tx=0x7f650400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.652+0000 7f650cff9700 1 -- 172.21.15.97:0/281652966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504011800 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.652+0000 7f650cff9700 1 -- 172.21.15.97:0/281652966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6504011960 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.652+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65101603c0 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.652+0000 7f650cff9700 1 -- 172.21.15.97:0/281652966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6504011cf0 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.652+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651014e900 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.652+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65100ce6e0 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.653+0000 7f650cff9700 1 -- 172.21.15.97:0/281652966 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6504018360 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.653+0000 7f650cff9700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64fc06af60 0x7f64fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.653+0000 7f650cff9700 1 -- 172.21.15.97:0/281652966 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6504089630 con 0x7f651006c000 2023-11-24T06:10:28.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.653+0000 7f650f7fe700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64fc06af60 0x7f64fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.654+0000 7f650f7fe700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64fc06af60 0x7f64fc06d420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6500009920 tx=0x7f6500008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:28.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.658+0000 7f650cff9700 1 -- 172.21.15.97:0/281652966 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65040545e0 con 0x7f651006c000 2023-11-24T06:10:28.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.850+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65100753f0 con 0x7f651006c000 2023-11-24T06:10:28.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.851+0000 7f650cff9700 1 -- 172.21.15.97:0/281652966 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16538 (secure 0 0 0) 0x7f6504057c00 con 0x7f651006c000 2023-11-24T06:10:28.855 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:28.855 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":44,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:27.575852+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:25.335173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:26.547561+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:28.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64fc06af60 msgr2=0x7f64fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64fc06af60 0x7f64fc06d420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6500009920 tx=0x7f6500008030 comp rx=0 tx=0).stop 2023-11-24T06:10:28.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f651006c000 msgr2=0x7f651014e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f651006c000 0x7f651014e3c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f650400c7f0 tx=0x7f650400cbb0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 shutdown_connections 2023-11-24T06:10:28.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f651015e080 0x7f6510156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f651006c000 0x7f651014e3c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f64fc06af60 0x7f64fc06d420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 --2- 172.21.15.97:0/281652966 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f65100af4d0 0x7f651015f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 >> 172.21.15.97:0/281652966 conn(0x7f6510071b90 msgr2=0x7f651006f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:28.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 shutdown_connections 2023-11-24T06:10:28.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.853+0000 7f65167e6700 1 -- 172.21.15.97:0/281652966 wait complete. 2023-11-24T06:10:28.869 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:28.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.985+0000 7fc9f4d10700 1 Processor -- start 2023-11-24T06:10:28.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.985+0000 7fc9f4d10700 1 -- start start 2023-11-24T06:10:28.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9f4d10700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 0x7fc9f00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9f4d10700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9f006c710 0x7fc9f00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9f4d10700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f00ae430 con 0x7fc9f00af4d0 2023-11-24T06:10:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9f4d10700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f00af0b0 con 0x7fc9f006c710 2023-11-24T06:10:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9f4d10700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f00b05a0 con 0x7fc9f015e080 2023-11-24T06:10:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9eed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9f006c710 0x7fc9f00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9eed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9f006c710 0x7fc9f00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47938/0 (socket says 172.21.15.97:47938) 2023-11-24T06:10:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9eed9d700 1 -- 172.21.15.97:0/1610534337 learned_addr learned my addr 172.21.15.97:0/1610534337 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9ee59c700 1 --2- 172.21.15.97:0/1610534337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 0x7fc9f00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9eed9d700 1 -- 172.21.15.97:0/1610534337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 msgr2=0x7fc9f00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9eed9d700 1 --2- 172.21.15.97:0/1610534337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 0x7fc9f00b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9eed9d700 1 -- 172.21.15.97:0/1610534337 >> v1:172.21.15.183:6789/0 conn(0x7fc9f00af4d0 legacy=0x7fc9f006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:28.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.986+0000 7fc9eed9d700 1 -- 172.21.15.97:0/1610534337 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00ae710 con 0x7fc9f006c710 2023-11-24T06:10:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9eed9d700 1 --2- 172.21.15.97:0/1610534337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f00afb90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009e80 tx=0x7fc9e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf5b8a0d2842d0f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9ed59a700 1 -- 172.21.15.97:0/1610534337 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800e070 con 0x7fc9f006c710 2023-11-24T06:10:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9ed59a700 1 -- 172.21.15.97:0/1610534337 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e8003a20 con 0x7fc9f006c710 2023-11-24T06:10:28.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9ed59a700 1 -- 172.21.15.97:0/1610534337 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8003dd0 con 0x7fc9f006c710 2023-11-24T06:10:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9f4d10700 1 -- 172.21.15.97:0/1610534337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 msgr2=0x7fc9f00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/1610534337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f00afb90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009e80 tx=0x7fc9e800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9f4d10700 1 -- 172.21.15.97:0/1610534337 shutdown_connections 2023-11-24T06:10:28.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/1610534337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f00afb90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/1610534337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 0x7fc9f00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9f4d10700 1 -- 172.21.15.97:0/1610534337 >> 172.21.15.97:0/1610534337 conn(0x7fc9f0071b90 msgr2=0x7fc9f0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9f4d10700 1 -- 172.21.15.97:0/1610534337 shutdown_connections 2023-11-24T06:10:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.987+0000 7fc9f4d10700 1 -- 172.21.15.97:0/1610534337 wait complete. 2023-11-24T06:10:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9f4d10700 1 Processor -- start 2023-11-24T06:10:28.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9f4d10700 1 -- start start 2023-11-24T06:10:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9f4d10700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f0157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9f4d10700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f00af4d0 0x7fc9f01f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9f4d10700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 0x7fc9f0159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9f4d10700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f0157710 con 0x7fc9f006c710 2023-11-24T06:10:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9f4d10700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f0157560 con 0x7fc9f00af4d0 2023-11-24T06:10:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9f4d10700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f0157920 con 0x7fc9f015e080 2023-11-24T06:10:28.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.989+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f0157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f0157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47940/0 (socket says 172.21.15.97:47940) 2023-11-24T06:10:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9ee59c700 1 -- 172.21.15.97:0/174842304 learned_addr learned my addr 172.21.15.97:0/174842304 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9eed9d700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 0x7fc9f0159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9edd9b700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f00af4d0 0x7fc9f01f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:28.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9ee59c700 1 -- 172.21.15.97:0/174842304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 msgr2=0x7fc9f0159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9ee59c700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 0x7fc9f0159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9ee59c700 1 -- 172.21.15.97:0/174842304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f00af4d0 msgr2=0x7fc9f01f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9ee59c700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f00af4d0 0x7fc9f01f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9ee59c700 1 -- 172.21.15.97:0/174842304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8009b30 con 0x7fc9f006c710 2023-11-24T06:10:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9ee59c700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f0157020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400c7f0 tx=0x7fc9e400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9deffd700 1 -- 172.21.15.97:0/174842304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4011800 con 0x7fc9f006c710 2023-11-24T06:10:28.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9deffd700 1 -- 172.21.15.97:0/174842304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e4011960 con 0x7fc9f006c710 2023-11-24T06:10:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f0157c00 con 0x7fc9f006c710 2023-11-24T06:10:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9deffd700 1 -- 172.21.15.97:0/174842304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4011cf0 con 0x7fc9f006c710 2023-11-24T06:10:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.990+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f01581c0 con 0x7fc9f006c710 2023-11-24T06:10:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.991+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f00753f0 con 0x7fc9f006c710 2023-11-24T06:10:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.991+0000 7fc9deffd700 1 -- 172.21.15.97:0/174842304 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc9e4018360 con 0x7fc9f006c710 2023-11-24T06:10:29.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.992+0000 7fc9deffd700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc9d006af60 0x7fc9d006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.992+0000 7fc9deffd700 1 -- 172.21.15.97:0/174842304 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9e4089620 con 0x7fc9f006c710 2023-11-24T06:10:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.992+0000 7fc9edd9b700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc9d006af60 0x7fc9d006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.992+0000 7fc9edd9b700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc9d006af60 0x7fc9d006d420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9f006f3d0 tx=0x7fc9e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:28.996+0000 7fc9deffd700 1 -- 172.21.15.97:0/174842304 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9e40545e0 con 0x7fc9f006c710 2023-11-24T06:10:29.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.189+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9f0067f60 con 0x7fc9f006c710 2023-11-24T06:10:29.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.191+0000 7fc9deffd700 1 -- 172.21.15.97:0/174842304 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16525 (secure 0 0 0) 0x7fc9e4057c00 con 0x7fc9f006c710 2023-11-24T06:10:29.194 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:29.194 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":45,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:29.183019+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:25.335173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:26.547561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:29.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc9d006af60 msgr2=0x7fc9d006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc9d006af60 0x7fc9d006d420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc9f006f3d0 tx=0x7fc9e0009040 comp rx=0 tx=0).stop 2023-11-24T06:10:29.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 msgr2=0x7fc9f0157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f0157020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400c7f0 tx=0x7fc9e400cbb0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 shutdown_connections 2023-11-24T06:10:29.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc9d006af60 0x7fc9d006d420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9f015e080 0x7fc9f0159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9f006c710 0x7fc9f0157020 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 --2- 172.21.15.97:0/174842304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9f00af4d0 0x7fc9f01f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 >> 172.21.15.97:0/174842304 conn(0x7fc9f0071b90 msgr2=0x7fc9f00b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:29.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 shutdown_connections 2023-11-24T06:10:29.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.193+0000 7fc9f4d10700 1 -- 172.21.15.97:0/174842304 wait complete. 2023-11-24T06:10:29.208 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:10:29.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.329+0000 7efdab29e700 1 Processor -- start 2023-11-24T06:10:29.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.329+0000 7efdab29e700 1 -- start start 2023-11-24T06:10:29.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efdab29e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda415be20 0x7efda40af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efdab29e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efda40cdf50 0x7efda40b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efdab29e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda40b4970 con 0x7efda40b4e00 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efdab29e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda40cfed0 con 0x7efda40cdf50 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efdab29e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda40b4360 con 0x7efda415be20 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efda983b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7efda40cdf50 0x7efda40b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efda8839700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7efda40b4e00 0x7efda40cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:49724/0 (socket says 172.21.15.97:49724) 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efda8839700 1 -- 172.21.15.97:0/550224262 learned_addr learned my addr 172.21.15.97:0/550224262 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efda903a700 1 --2- 172.21.15.97:0/550224262 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda415be20 0x7efda40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efda983b700 1 -- 172.21.15.97:0/550224262 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda415be20 msgr2=0x7efda40af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efda983b700 1 --2- 172.21.15.97:0/550224262 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda415be20 0x7efda40af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efda983b700 1 -- 172.21.15.97:0/550224262 >> v1:172.21.15.183:6789/0 conn(0x7efda40b4e00 legacy=0x7efda40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.330+0000 7efda983b700 1 -- 172.21.15.97:0/550224262 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda406f940 con 0x7efda40cdf50 2023-11-24T06:10:29.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efda983b700 1 --2- 172.21.15.97:0/550224262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda40b5280 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009d90 tx=0x7efd9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfca448d918f9f08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efd9bfff700 1 -- 172.21.15.97:0/550224262 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00e070 con 0x7efda40cdf50 2023-11-24T06:10:29.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efd9bfff700 1 -- 172.21.15.97:0/550224262 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c00bbd0 con 0x7efda40cdf50 2023-11-24T06:10:29.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efd9bfff700 1 -- 172.21.15.97:0/550224262 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c012430 con 0x7efda40cdf50 2023-11-24T06:10:29.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efdab29e700 1 -- 172.21.15.97:0/550224262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 msgr2=0x7efda40b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efdab29e700 1 --2- 172.21.15.97:0/550224262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda40b5280 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009d90 tx=0x7efd9c00b330 comp rx=0 tx=0).stop 2023-11-24T06:10:29.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efdab29e700 1 -- 172.21.15.97:0/550224262 shutdown_connections 2023-11-24T06:10:29.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efdab29e700 1 --2- 172.21.15.97:0/550224262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda40b5280 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efdab29e700 1 --2- 172.21.15.97:0/550224262 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda415be20 0x7efda40af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efdab29e700 1 -- 172.21.15.97:0/550224262 >> 172.21.15.97:0/550224262 conn(0x7efda40b2050 msgr2=0x7efda40b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efdab29e700 1 -- 172.21.15.97:0/550224262 shutdown_connections 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.331+0000 7efdab29e700 1 -- 172.21.15.97:0/550224262 wait complete. 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.333+0000 7efdab29e700 1 Processor -- start 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.333+0000 7efdab29e700 1 -- start start 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.333+0000 7efdab29e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda40b4e00 0x7efda415efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.333+0000 7efdab29e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda4141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.333+0000 7efdab29e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda415be20 0x7efda415c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.333+0000 7efdab29e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda415ce20 con 0x7efda40cdf50 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.333+0000 7efdab29e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda415cb40 con 0x7efda415be20 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.333+0000 7efdab29e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda415e500 con 0x7efda40b4e00 2023-11-24T06:10:29.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda4141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda4141870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47956/0 (socket says 172.21.15.97:47956) 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 -- 172.21.15.97:0/4225359796 learned_addr learned my addr 172.21.15.97:0/4225359796 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda903a700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda40b4e00 0x7efda415efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda983b700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda415be20 0x7efda415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 -- 172.21.15.97:0/4225359796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda40b4e00 msgr2=0x7efda415efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda40b4e00 0x7efda415efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 -- 172.21.15.97:0/4225359796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda415be20 msgr2=0x7efda415c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda415be20 0x7efda415c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 -- 172.21.15.97:0/4225359796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c009a70 con 0x7efda40cdf50 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efda8839700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda4141870 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c430 tx=0x7efd9400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efd99ffb700 1 -- 172.21.15.97:0/4225359796 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94005b70 con 0x7efda40cdf50 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efd99ffb700 1 -- 172.21.15.97:0/4225359796 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94005cd0 con 0x7efda40cdf50 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda415e7e0 con 0x7efda40cdf50 2023-11-24T06:10:29.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efd99ffb700 1 -- 172.21.15.97:0/4225359796 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd940072f0 con 0x7efda40cdf50 2023-11-24T06:10:29.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.334+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda415e1f0 con 0x7efda40cdf50 2023-11-24T06:10:29.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.335+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda4068d90 con 0x7efda40cdf50 2023-11-24T06:10:29.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.336+0000 7efd99ffb700 1 -- 172.21.15.97:0/4225359796 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7efd94007450 con 0x7efda40cdf50 2023-11-24T06:10:29.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.336+0000 7efd99ffb700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd8806af60 0x7efd8806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:29.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.336+0000 7efda903a700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd8806af60 0x7efd8806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:29.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.336+0000 7efd99ffb700 1 -- 172.21.15.97:0/4225359796 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd940888f0 con 0x7efda40cdf50 2023-11-24T06:10:29.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.337+0000 7efda903a700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd8806af60 0x7efd8806d420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efda0006b20 tx=0x7efda0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:29.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.341+0000 7efd99ffb700 1 -- 172.21.15.97:0/4225359796 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd940539b0 con 0x7efda40cdf50 2023-11-24T06:10:29.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:29.578+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efda4068fb0 con 0x7efda40cdf50 2023-11-24T06:10:30.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.505+0000 7efd99ffb700 1 -- 172.21.15.97:0/4225359796 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7efd940170b0 con 0x7efda40cdf50 2023-11-24T06:10:30.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd8806af60 msgr2=0x7efd8806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd8806af60 0x7efd8806d420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efda0006b20 tx=0x7efda0008040 comp rx=0 tx=0).stop 2023-11-24T06:10:30.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 msgr2=0x7efda4141870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda4141870 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c430 tx=0x7efd9400c740 comp rx=0 tx=0).stop 2023-11-24T06:10:30.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 shutdown_connections 2023-11-24T06:10:30.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda415be20 0x7efda415c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd8806af60 0x7efd8806d420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda40b4e00 0x7efda415efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 --2- 172.21.15.97:0/4225359796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda40cdf50 0x7efda4141870 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.508+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 >> 172.21.15.97:0/4225359796 conn(0x7efda40b2050 msgr2=0x7efda41588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:30.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.509+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 shutdown_connections 2023-11-24T06:10:30.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.509+0000 7efdab29e700 1 -- 172.21.15.97:0/4225359796 wait complete. 2023-11-24T06:10:30.526 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:10:30.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:10:30.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.651+0000 7f5c98677700 1 Processor -- start 2023-11-24T06:10:30.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.651+0000 7f5c98677700 1 -- start start 2023-11-24T06:10:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.652+0000 7f5c98677700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5c9015f9e0 0x7f5c900add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.652+0000 7f5c98677700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c90073f90 0x7f5c90074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.652+0000 7f5c98677700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9006b080 con 0x7f5c900cf610 2023-11-24T06:10:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.652+0000 7f5c98677700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c90074960 con 0x7f5c9015f9e0 2023-11-24T06:10:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.652+0000 7f5c98677700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9015c290 con 0x7f5c90073f90 2023-11-24T06:10:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c96413700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5c900cf610 0x7f5c900b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40414/0 (socket says 172.21.15.97:40414) 2023-11-24T06:10:30.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c96413700 1 -- 172.21.15.97:0/3304257334 learned_addr learned my addr 172.21.15.97:0/3304257334 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c96c14700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c90073f90 0x7f5c90074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95c12700 1 --2- 172.21.15.97:0/3304257334 >> v2:172.21.15.183:3300/0 conn(0x7f5c9015f9e0 0x7f5c900add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319097217 0 0) 0x7f5c9006b080 con 0x7f5c900cf610 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c7c0035b0 con 0x7f5c900cf610 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 468838229 0 0) 0x7f5c7c0035b0 con 0x7f5c900cf610 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c90073f90 msgr2=0x7f5c90074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c90073f90 0x7f5c90074420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 >> v2:172.21.15.183:3300/0 conn(0x7f5c9015f9e0 msgr2=0x7f5c900add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 --2- 172.21.15.97:0/3304257334 >> v2:172.21.15.183:3300/0 conn(0x7f5c9015f9e0 0x7f5c900add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900cf9f0 con 0x7f5c900cf610 2023-11-24T06:10:30.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f5c80001670 con 0x7f5c900cf610 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 >> v1:172.21.15.97:6789/0 conn(0x7f5c900cf610 legacy=0x7f5c900b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c7c003a00 0x7f5c7c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c004850 0x7f5c7c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c90073f90 0x7f5c7c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c90074960 con 0x7f5c7c003a00 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9015c290 con 0x7f5c7c004850 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c00ba10 con 0x7f5c90073f90 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c96413700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c7c003a00 0x7f5c7c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c95c12700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c004850 0x7f5c7c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c96c14700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c90073f90 0x7f5c7c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c96413700 1 -- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c90073f90 msgr2=0x7f5c7c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:30.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.653+0000 7f5c96413700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c90073f90 0x7f5c7c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c96413700 1 -- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c004850 msgr2=0x7f5c7c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c96413700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c7c004850 0x7f5c7c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c96413700 1 -- 172.21.15.97:0/3304257334 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0035b0 con 0x7f5c7c003a00 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c96413700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c7c003a00 0x7f5c7c003e20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000c320 tx=0x7f5c7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4f5ca7df5e3516 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000ce60 con 0x7f5c7c003a00 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80005960 con 0x7f5c7c003a00 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c95411700 1 -- 172.21.15.97:0/3304257334 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80005d10 con 0x7f5c7c003a00 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c98677700 1 -- 172.21.15.97:0/3304257334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c7c003a00 msgr2=0x7f5c7c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c98677700 1 --2- 172.21.15.97:0/3304257334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c7c003a00 0x7f5c7c003e20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000c320 tx=0x7f5c7c005880 comp rx=0 tx=0).stop 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c96c14700 1 -- 172.21.15.97:0/3304257334 reap_dead start 2023-11-24T06:10:30.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c98677700 1 -- 172.21.15.97:0/3304257334 shutdown_connections 2023-11-24T06:10:30.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c98677700 1 -- 172.21.15.97:0/3304257334 >> 172.21.15.97:0/3304257334 conn(0x7f5c900d0bf0 msgr2=0x7f5c900d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:30.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c98677700 1 -- 172.21.15.97:0/3304257334 shutdown_connections 2023-11-24T06:10:30.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.654+0000 7f5c98677700 1 -- 172.21.15.97:0/3304257334 wait complete. 2023-11-24T06:10:30.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c98677700 1 Processor -- start 2023-11-24T06:10:30.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c98677700 1 -- start start 2023-11-24T06:10:30.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c98677700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c7c003a00 0x7f5c9015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c98677700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c90150fd0 0x7f5c9015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c98677700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9014a650 0x7f5c9014aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c98677700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9015cf80 con 0x7f5c90150fd0 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c98677700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9014b020 con 0x7f5c9014a650 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c98677700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c901514b0 con 0x7f5c7c003a00 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c95c12700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c90150fd0 0x7f5c9015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c95c12700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c90150fd0 0x7f5c9015a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47974/0 (socket says 172.21.15.97:47974) 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.656+0000 7f5c95c12700 1 -- 172.21.15.97:0/2540761494 learned_addr learned my addr 172.21.15.97:0/2540761494 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c95c12700 1 -- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c7c003a00 msgr2=0x7f5c9015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c96413700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c7c003a00 0x7f5c9015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c96c14700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9014a650 0x7f5c9014aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c95c12700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c7c003a00 0x7f5c9015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c95c12700 1 -- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9014a650 msgr2=0x7f5c9014aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c95c12700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9014a650 0x7f5c9014aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c95c12700 1 -- 172.21.15.97:0/2540761494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c90151790 con 0x7f5c90150fd0 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c96c14700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9014a650 0x7f5c9014aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c95c12700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c90150fd0 0x7f5c9015a340 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c011ad0 tx=0x7f5c8c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c86ffd700 1 -- 172.21.15.97:0/2540761494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00fa00 con 0x7f5c90150fd0 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c86ffd700 1 -- 172.21.15.97:0/2540761494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c00fb80 con 0x7f5c90150fd0 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c90151a70 con 0x7f5c90150fd0 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c86ffd700 1 -- 172.21.15.97:0/2540761494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c014440 con 0x7f5c90150fd0 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.657+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9015f1c0 con 0x7f5c90150fd0 2023-11-24T06:10:30.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.658+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c90068fb0 con 0x7f5c90150fd0 2023-11-24T06:10:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.658+0000 7f5c86ffd700 1 -- 172.21.15.97:0/2540761494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5c8c0145a0 con 0x7f5c90150fd0 2023-11-24T06:10:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.659+0000 7f5c86ffd700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5c7406af60 0x7f5c7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.659+0000 7f5c86ffd700 1 -- 172.21.15.97:0/2540761494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c8c089c40 con 0x7f5c90150fd0 2023-11-24T06:10:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.659+0000 7f5c96413700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5c7406af60 0x7f5c7406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.659+0000 7f5c96413700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5c7406af60 0x7f5c7406d420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c80015040 tx=0x7f5c8000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:30.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.663+0000 7f5c86ffd700 1 -- 172.21.15.97:0/2540761494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c8c054e50 con 0x7f5c90150fd0 2023-11-24T06:10:30.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:30.897+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5c900aefd0 con 0x7f5c90150fd0 2023-11-24T06:10:31.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.506+0000 7f5c86ffd700 1 -- 172.21.15.97:0/2540761494 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c8c058470 con 0x7f5c90150fd0 2023-11-24T06:10:31.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5c7406af60 msgr2=0x7f5c7406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5c7406af60 0x7f5c7406d420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5c80015040 tx=0x7f5c8000a000 comp rx=0 tx=0).stop 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c90150fd0 msgr2=0x7f5c9015a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c90150fd0 0x7f5c9015a340 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c011ad0 tx=0x7f5c8c011e90 comp rx=0 tx=0).stop 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 shutdown_connections 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c9014a650 0x7f5c9014aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5c7406af60 0x7f5c7406d420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5c7c003a00 0x7f5c9015ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 --2- 172.21.15.97:0/2540761494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c90150fd0 0x7f5c9015a340 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 >> 172.21.15.97:0/2540761494 conn(0x7f5c900d0bf0 msgr2=0x7f5c900ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 shutdown_connections 2023-11-24T06:10:31.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.509+0000 7f5c98677700 1 -- 172.21.15.97:0/2540761494 wait complete. 2023-11-24T06:10:31.524 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:10:31.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:31.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.642+0000 7fc7d444a700 1 Processor -- start 2023-11-24T06:10:31.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.642+0000 7fc7d444a700 1 -- start start 2023-11-24T06:10:31.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.643+0000 7fc7d444a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc7cc06c710 0x7fc7cc165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d444a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc0af950 con 0x7fc7cc1446a0 2023-11-24T06:10:31.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d444a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7cc0afc30 con 0x7fc7cc0adf00 2023-11-24T06:10:31.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d21e6700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc7cc1446a0 0x7fc7cc0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40434/0 (socket says 172.21.15.97:40434) 2023-11-24T06:10:31.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d444a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc144a80 con 0x7fc7cc06c710 2023-11-24T06:10:31.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d21e6700 1 -- 172.21.15.97:0/4281073746 learned_addr learned my addr 172.21.15.97:0/4281073746 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d29e7700 1 --2- 172.21.15.97:0/4281073746 >> v2:172.21.15.183:3300/0 conn(0x7fc7cc06c710 0x7fc7cc165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d11e4700 1 -- 172.21.15.97:0/4281073746 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563994358 0 0) 0x7fc7cc0af950 con 0x7fc7cc1446a0 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d11e4700 1 -- 172.21.15.97:0/4281073746 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b80035b0 con 0x7fc7cc1446a0 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d29e7700 1 -- 172.21.15.97:0/4281073746 >> v1:172.21.15.183:6789/0 conn(0x7fc7cc0adf00 legacy=0x7fc7cc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d29e7700 1 -- 172.21.15.97:0/4281073746 >> v1:172.21.15.97:6789/0 conn(0x7fc7cc1446a0 legacy=0x7fc7cc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d29e7700 1 -- 172.21.15.97:0/4281073746 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0aff10 con 0x7fc7cc06c710 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d29e7700 1 --2- 172.21.15.97:0/4281073746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc06c710 0x7fc7cc165b70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009d90 tx=0x7fc7c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff766e6ef2c29143 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d11e4700 1 -- 172.21.15.97:0/4281073746 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400e070 con 0x7fc7cc06c710 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d11e4700 1 -- 172.21.15.97:0/4281073746 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c400bb20 con 0x7fc7cc06c710 2023-11-24T06:10:31.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.644+0000 7fc7d11e4700 1 -- 172.21.15.97:0/4281073746 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4012430 con 0x7fc7cc06c710 2023-11-24T06:10:31.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.645+0000 7fc7d444a700 1 -- 172.21.15.97:0/4281073746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc06c710 msgr2=0x7fc7cc165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.645+0000 7fc7d444a700 1 --2- 172.21.15.97:0/4281073746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc06c710 0x7fc7cc165b70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009d90 tx=0x7fc7c400b330 comp rx=0 tx=0).stop 2023-11-24T06:10:31.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.645+0000 7fc7d444a700 1 -- 172.21.15.97:0/4281073746 shutdown_connections 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.645+0000 7fc7d444a700 1 --2- 172.21.15.97:0/4281073746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc06c710 0x7fc7cc165b70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.645+0000 7fc7d444a700 1 -- 172.21.15.97:0/4281073746 >> 172.21.15.97:0/4281073746 conn(0x7fc7cc071b90 msgr2=0x7fc7cc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.645+0000 7fc7d444a700 1 -- 172.21.15.97:0/4281073746 shutdown_connections 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.645+0000 7fc7d444a700 1 -- 172.21.15.97:0/4281073746 wait complete. 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.646+0000 7fc7d444a700 1 Processor -- start 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.646+0000 7fc7d444a700 1 -- start start 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d444a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7cc06c710 0x7fc7cc1509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d444a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7cc0adf00 0x7fc7cc1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d444a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc1446a0 0x7fc7cc150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d444a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc14ed80 con 0x7fc7cc0adf00 2023-11-24T06:10:31.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d444a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc14efd0 con 0x7fc7cc1446a0 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d444a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc14f2b0 con 0x7fc7cc06c710 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7cc0adf00 0x7fc7cc1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7cc0adf00 0x7fc7cc1536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47990/0 (socket says 172.21.15.97:47990) 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 -- 172.21.15.97:0/1831691913 learned_addr learned my addr 172.21.15.97:0/1831691913 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d29e7700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc1446a0 0x7fc7cc150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d21e6700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7cc06c710 0x7fc7cc1509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 -- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7cc06c710 msgr2=0x7fc7cc1509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7cc06c710 0x7fc7cc1509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 -- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc1446a0 msgr2=0x7fc7cc150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc1446a0 0x7fc7cc150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 -- 172.21.15.97:0/1831691913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4009a70 con 0x7fc7cc0adf00 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d21e6700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7cc06c710 0x7fc7cc1509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.647+0000 7fc7d19e5700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7cc0adf00 0x7fc7cc1536f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800c450 tx=0x7fc7c800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.648+0000 7fc7c2ffd700 1 -- 172.21.15.97:0/1831691913 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80052d0 con 0x7fc7cc0adf00 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.648+0000 7fc7c2ffd700 1 -- 172.21.15.97:0/1831691913 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c8005430 con 0x7fc7cc0adf00 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.648+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc14f590 con 0x7fc7cc0adf00 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.648+0000 7fc7c2ffd700 1 -- 172.21.15.97:0/1831691913 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80038c0 con 0x7fc7cc0adf00 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.648+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc14fb50 con 0x7fc7cc0adf00 2023-11-24T06:10:31.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.649+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7cc0753f0 con 0x7fc7cc0adf00 2023-11-24T06:10:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.649+0000 7fc7c2ffd700 1 -- 172.21.15.97:0/1831691913 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc7c8003a20 con 0x7fc7cc0adf00 2023-11-24T06:10:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.649+0000 7fc7c2ffd700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7b006af60 0x7fc7b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.650+0000 7fc7c2ffd700 1 -- 172.21.15.97:0/1831691913 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7c8088990 con 0x7fc7cc0adf00 2023-11-24T06:10:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.650+0000 7fc7d21e6700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7b006af60 0x7fc7b006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:31.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.650+0000 7fc7d21e6700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7b006af60 0x7fc7b006d420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0011e0 tx=0x7fc7bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:31.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.654+0000 7fc7c2ffd700 1 -- 172.21.15.97:0/1831691913 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7c8053a50 con 0x7fc7cc0adf00 2023-11-24T06:10:31.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.886+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7cc14fd40 con 0x7fc7cc0adf00 2023-11-24T06:10:31.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.886+0000 7fc7c2ffd700 1 -- 172.21.15.97:0/1831691913 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fc7c8017090 con 0x7fc7cc0adf00 2023-11-24T06:10:31.889 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:31.889 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/163703518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":163703518},{"type":"v1","addr":"172.21.15.97:6837","nonce":163703518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/1688123009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1688123009},{"type":"v1","addr":"172.21.15.183:6834","nonce":1688123009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/310487911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":310487911},{"type":"v1","addr":"172.21.15.183:6835","nonce":310487911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:27.576219+0000","modified":"2023-11-24T06:10:30.898770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/19331625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":19331625},{"type":"v1","addr":"172.21.15.97:6835","nonce":19331625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:10:31.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7b006af60 msgr2=0x7fc7b006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7b006af60 0x7fc7b006d420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0011e0 tx=0x7fc7bc009040 comp rx=0 tx=0).stop 2023-11-24T06:10:31.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7cc0adf00 msgr2=0x7fc7cc1536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7cc0adf00 0x7fc7cc1536f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800c450 tx=0x7fc7c800c760 comp rx=0 tx=0).stop 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 shutdown_connections 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7cc1446a0 0x7fc7cc150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc7b006af60 0x7fc7b006d420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc7cc06c710 0x7fc7cc1509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 --2- 172.21.15.97:0/1831691913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7cc0adf00 0x7fc7cc1536f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 >> 172.21.15.97:0/1831691913 conn(0x7fc7cc071b90 msgr2=0x7fc7cc06b150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 shutdown_connections 2023-11-24T06:10:31.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:31.888+0000 7fc7d444a700 1 -- 172.21.15.97:0/1831691913 wait complete. 2023-11-24T06:10:31.893 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2023-11-24T06:10:31.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:32.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.020+0000 7f6198bbd700 1 Processor -- start 2023-11-24T06:10:32.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.020+0000 7f6198bbd700 1 -- start start 2023-11-24T06:10:32.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f6198bbd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f61941587e0 0x7f619415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f6198bbd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f61940b2a40 0x7f619406e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f6198bbd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f619415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f6198bbd700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61940b3d30 con 0x7f61941587e0 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f6198bbd700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619406edf0 con 0x7f61940b2a40 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f6198bbd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61940b30b0 con 0x7f619406e9a0 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f619259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f61941587e0 0x7f619415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f619259c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f61941587e0 0x7f619415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47996/0 (socket says 172.21.15.97:47996) 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f619259c700 1 -- 172.21.15.97:0/3874737606 learned_addr learned my addr 172.21.15.97:0/3874737606 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f6191d9b700 1 --2- 172.21.15.97:0/3874737606 >> v2:172.21.15.183:3300/0 conn(0x7f61940b2a40 0x7f619406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.021+0000 7f6192d9d700 1 --2- 172.21.15.97:0/3874737606 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f619415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619259c700 1 -- 172.21.15.97:0/3874737606 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 msgr2=0x7f619415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619259c700 1 --2- 172.21.15.97:0/3874737606 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f619415e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619259c700 1 -- 172.21.15.97:0/3874737606 >> v2:172.21.15.183:3300/0 conn(0x7f61940b2a40 msgr2=0x7f619406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619259c700 1 --2- 172.21.15.97:0/3874737606 >> v2:172.21.15.183:3300/0 conn(0x7f61940b2a40 0x7f619406e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619259c700 1 -- 172.21.15.97:0/3874737606 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61940b3390 con 0x7f61941587e0 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6191d9b700 1 --2- 172.21.15.97:0/3874737606 >> v2:172.21.15.183:3300/0 conn(0x7f61940b2a40 0x7f619406e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6192d9d700 1 --2- 172.21.15.97:0/3874737606 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f619415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619259c700 1 --2- 172.21.15.97:0/3874737606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61941587e0 0x7f619415b0b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6194070a50 tx=0x7f6188009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d6c2200d6bd7cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619159a700 1 -- 172.21.15.97:0/3874737606 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800b870 con 0x7f61941587e0 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619159a700 1 -- 172.21.15.97:0/3874737606 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618800b9d0 con 0x7f61941587e0 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f619159a700 1 -- 172.21.15.97:0/3874737606 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618800bd60 con 0x7f61941587e0 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6198bbd700 1 -- 172.21.15.97:0/3874737606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61941587e0 msgr2=0x7f619415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6198bbd700 1 --2- 172.21.15.97:0/3874737606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61941587e0 0x7f619415b0b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6194070a50 tx=0x7f6188009bd0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6198bbd700 1 -- 172.21.15.97:0/3874737606 shutdown_connections 2023-11-24T06:10:32.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6198bbd700 1 --2- 172.21.15.97:0/3874737606 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f619415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6198bbd700 1 --2- 172.21.15.97:0/3874737606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61941587e0 0x7f619415b0b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6198bbd700 1 --2- 172.21.15.97:0/3874737606 >> v2:172.21.15.183:3300/0 conn(0x7f61940b2a40 0x7f619406e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.022+0000 7f6198bbd700 1 -- 172.21.15.97:0/3874737606 >> 172.21.15.97:0/3874737606 conn(0x7f61940b1740 msgr2=0x7f61940b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.023+0000 7f6198bbd700 1 -- 172.21.15.97:0/3874737606 shutdown_connections 2023-11-24T06:10:32.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.023+0000 7f6198bbd700 1 -- 172.21.15.97:0/3874737606 wait complete. 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.024+0000 7f6198bbd700 1 Processor -- start 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.024+0000 7f6198bbd700 1 -- start start 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6198bbd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f61940c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6198bbd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61940b2a40 0x7f619414e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6198bbd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61941587e0 0x7f61940ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6198bbd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619415e090 con 0x7f61940b2a40 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6198bbd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619415da60 con 0x7f61941587e0 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6198bbd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61941f3a40 con 0x7f619406e9a0 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61940b2a40 0x7f619414e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61940b2a40 0x7f619414e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48004/0 (socket says 172.21.15.97:48004) 2023-11-24T06:10:32.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 -- 172.21.15.97:0/2390485504 learned_addr learned my addr 172.21.15.97:0/2390485504 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f619259c700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f61940c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6192d9d700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61941587e0 0x7f61940ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 -- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 msgr2=0x7f61940c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f61940c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 -- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61941587e0 msgr2=0x7f61940ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61941587e0 0x7f61940ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 -- 172.21.15.97:0/2390485504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6188009550 con 0x7f61940b2a40 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6192d9d700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61941587e0 0x7f61940ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6191d9b700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61940b2a40 0x7f619414e750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f618400e810 tx=0x7f618400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6182ffd700 1 -- 172.21.15.97:0/2390485504 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184004230 con 0x7f61940b2a40 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6182ffd700 1 -- 172.21.15.97:0/2390485504 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61840043b0 con 0x7f61940b2a40 2023-11-24T06:10:32.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6182ffd700 1 -- 172.21.15.97:0/2390485504 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61840038e0 con 0x7f61940b2a40 2023-11-24T06:10:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61940cead0 con 0x7f61940b2a40 2023-11-24T06:10:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.025+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619415dcb0 con 0x7f61940b2a40 2023-11-24T06:10:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.026+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61940751e0 con 0x7f61940b2a40 2023-11-24T06:10:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.027+0000 7f6182ffd700 1 -- 172.21.15.97:0/2390485504 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6184003a40 con 0x7f61940b2a40 2023-11-24T06:10:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.027+0000 7f6182ffd700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f617406af60 0x7f617406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.027+0000 7f6182ffd700 1 -- 172.21.15.97:0/2390485504 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6184089120 con 0x7f61940b2a40 2023-11-24T06:10:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.027+0000 7f619259c700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f617406af60 0x7f617406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.028+0000 7f619259c700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f617406af60 0x7f617406d420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6188014040 tx=0x7f6188008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.032+0000 7f6182ffd700 1 -- 172.21.15.97:0/2390485504 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61840540d0 con 0x7f61940b2a40 2023-11-24T06:10:32.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.222+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f619406ceb0 con 0x7f61940b2a40 2023-11-24T06:10:32.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.224+0000 7f6182ffd700 1 -- 172.21.15.97:0/2390485504 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16525 (secure 0 0 0) 0x7f61840094c0 con 0x7f61940b2a40 2023-11-24T06:10:32.227 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:32.227 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":45,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:29.183019+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:25.335173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:26.547561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.225+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f617406af60 msgr2=0x7f617406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.225+0000 7f6198bbd700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f617406af60 0x7f617406d420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6188014040 tx=0x7f6188008030 comp rx=0 tx=0).stop 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61940b2a40 msgr2=0x7f619414e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61940b2a40 0x7f619414e750 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f618400e810 tx=0x7f618400eb20 comp rx=0 tx=0).stop 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 shutdown_connections 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61941587e0 0x7f61940ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f617406af60 0x7f617406d420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f619406e9a0 0x7f61940c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 --2- 172.21.15.97:0/2390485504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61940b2a40 0x7f619414e750 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 >> 172.21.15.97:0/2390485504 conn(0x7f61940b1740 msgr2=0x7f6194157330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 shutdown_connections 2023-11-24T06:10:32.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.226+0000 7f6198bbd700 1 -- 172.21.15.97:0/2390485504 wait complete. 2023-11-24T06:10:32.241 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:10:32.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:10:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.358+0000 7f6de430d700 1 Processor -- start 2023-11-24T06:10:32.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.358+0000 7f6de430d700 1 -- start start 2023-11-24T06:10:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.359+0000 7f6de430d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 0x7f6ddc165d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.359+0000 7f6de430d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ddc15f800 con 0x7f6ddc0ce140 2023-11-24T06:10:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.359+0000 7f6de430d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ddc1414c0 con 0x7f6ddc15af10 2023-11-24T06:10:32.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.359+0000 7f6de430d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc15c300 con 0x7f6ddc06c650 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.359+0000 7f6de18a8700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6ddc0ce140 0x7f6ddc15f0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40462/0 (socket says 172.21.15.97:40462) 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.359+0000 7f6de18a8700 1 -- 172.21.15.97:0/1162763043 learned_addr learned my addr 172.21.15.97:0/1162763043 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.359+0000 7f6de28aa700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 0x7f6ddc165d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.359+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283736743 0 0) 0x7f6ddc15f800 con 0x7f6ddc0ce140 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dc80035b0 con 0x7f6ddc0ce140 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 444171622 0 0) 0x7f6dc80035b0 con 0x7f6ddc0ce140 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 msgr2=0x7f6ddc165d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 0x7f6ddc165d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 >> v1:172.21.15.183:6789/0 conn(0x7f6ddc15af10 legacy=0x7f6ddc15e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc0ce9f0 con 0x7f6ddc0ce140 2023-11-24T06:10:32.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f6dd8001670 con 0x7f6ddc0ce140 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de28aa700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 0x7f6ddc165d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 >> v1:172.21.15.97:6789/0 conn(0x7f6ddc0ce140 legacy=0x7f6ddc15f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 0x7f6dc8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6dc8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6dc8004d90 0x7f6dc8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc15c300 con 0x7f6dc8003f70 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc800bb50 con 0x7f6dc8004d90 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc800bd10 con 0x7f6ddc06c650 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de18a8700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6dc8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de20a9700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 0x7f6dc8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de28aa700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6dc8004d90 0x7f6dc8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de18a8700 1 -- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 msgr2=0x7f6dc8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de18a8700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc06c650 0x7f6dc8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de18a8700 1 -- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6dc8004d90 msgr2=0x7f6dc8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de18a8700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6dc8004d90 0x7f6dc8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de18a8700 1 -- 172.21.15.97:0/1162763043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd4009a70 con 0x7f6dc8003f70 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de28aa700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6dc8004d90 0x7f6dc8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de18a8700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6dc8004850 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c210 tx=0x7f6dc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d0580e194a4999 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.360+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8007c10 con 0x7f6dc8003f70 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd800cdd0 con 0x7f6dc8003f70 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de10a7700 1 -- 172.21.15.97:0/1162763043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8005ba0 con 0x7f6dc8003f70 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de430d700 1 -- 172.21.15.97:0/1162763043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 msgr2=0x7f6dc8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de430d700 1 --2- 172.21.15.97:0/1162763043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6dc8004850 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c210 tx=0x7f6dc8005880 comp rx=0 tx=0).stop 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de28aa700 1 -- 172.21.15.97:0/1162763043 reap_dead start 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de430d700 1 -- 172.21.15.97:0/1162763043 shutdown_connections 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de430d700 1 -- 172.21.15.97:0/1162763043 >> 172.21.15.97:0/1162763043 conn(0x7f6ddc0cd570 msgr2=0x7f6ddc0cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de430d700 1 -- 172.21.15.97:0/1162763043 shutdown_connections 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.361+0000 7f6de430d700 1 -- 172.21.15.97:0/1162763043 wait complete. 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.362+0000 7f6de430d700 1 Processor -- start 2023-11-24T06:10:32.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.362+0000 7f6de430d700 1 -- start start 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de430d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6ddc159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de430d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc13eac0 0x7f6ddc0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de430d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ddc0ca700 0x7f6ddc0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de430d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0b4ce0 con 0x7f6dc8003f70 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de430d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc15cff0 con 0x7f6ddc0ca700 2023-11-24T06:10:32.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de430d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc15c6c0 con 0x7f6ddc13eac0 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de20a9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6ddc159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de20a9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6ddc159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48022/0 (socket says 172.21.15.97:48022) 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de20a9700 1 -- 172.21.15.97:0/1365535182 learned_addr learned my addr 172.21.15.97:0/1365535182 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de18a8700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc13eac0 0x7f6ddc0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de28aa700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ddc0ca700 0x7f6ddc0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.363+0000 7f6de20a9700 1 -- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc13eac0 msgr2=0x7f6ddc0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6de20a9700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc13eac0 0x7f6ddc0b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6de20a9700 1 -- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ddc0ca700 msgr2=0x7f6ddc0cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6de20a9700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ddc0ca700 0x7f6ddc0cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6de20a9700 1 -- 172.21.15.97:0/1365535182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc15c9a0 con 0x7f6dc8003f70 2023-11-24T06:10:32.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6de28aa700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ddc0ca700 0x7f6ddc0cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6de20a9700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6ddc159950 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00f7f0 tx=0x7f6dcc00fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6dd2ffd700 1 -- 172.21.15.97:0/1365535182 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0052e0 con 0x7f6dc8003f70 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6dd2ffd700 1 -- 172.21.15.97:0/1365535182 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc005440 con 0x7f6dc8003f70 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc15cc80 con 0x7f6dc8003f70 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6dd2ffd700 1 -- 172.21.15.97:0/1365535182 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0038c0 con 0x7f6dc8003f70 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.364+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc13efb0 con 0x7f6dc8003f70 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.365+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc15d2d0 con 0x7f6dc8003f70 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.365+0000 7f6dd2ffd700 1 -- 172.21.15.97:0/1365535182 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6dcc003a20 con 0x7f6dc8003f70 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.365+0000 7f6dd2ffd700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6dc006af60 0x7f6dc006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.366+0000 7f6dd2ffd700 1 -- 172.21.15.97:0/1365535182 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6dcc089370 con 0x7f6dc8003f70 2023-11-24T06:10:32.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.366+0000 7f6de18a8700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6dc006af60 0x7f6dc006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.366+0000 7f6de18a8700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6dc006af60 0x7f6dc006d420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8000c00 tx=0x7f6dd8007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.370+0000 7f6dd2ffd700 1 -- 172.21.15.97:0/1365535182 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6dcc054430 con 0x7f6dc8003f70 2023-11-24T06:10:32.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.556+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ddc0cefb0 con 0x7f6dc006af60 2023-11-24T06:10:32.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.564+0000 7f6dd2ffd700 1 -- 172.21.15.97:0/1365535182 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289119 (secure 0 0 0) 0x7f6dd8022070 con 0x7f6dc006af60 2023-11-24T06:10:32.570 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.567+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6dc006af60 msgr2=0x7f6dc006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.567+0000 7f6de430d700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6dc006af60 0x7f6dc006d420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8000c00 tx=0x7f6dd8007000 comp rx=0 tx=0).stop 2023-11-24T06:10:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.567+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 msgr2=0x7f6ddc159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.567+0000 7f6de430d700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6ddc159950 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00f7f0 tx=0x7f6dcc00fb00 comp rx=0 tx=0).stop 2023-11-24T06:10:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.568+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 shutdown_connections 2023-11-24T06:10:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.568+0000 7f6de430d700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ddc0ca700 0x7f6ddc0cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.568+0000 7f6de430d700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dc8003f70 0x7f6ddc159950 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.568+0000 7f6de430d700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6dc006af60 0x7f6dc006d420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.568+0000 7f6de430d700 1 --2- 172.21.15.97:0/1365535182 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6ddc13eac0 0x7f6ddc0b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.568+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 >> 172.21.15.97:0/1365535182 conn(0x7f6ddc0cd570 msgr2=0x7f6ddc06b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.568+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 shutdown_connections 2023-11-24T06:10:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.568+0000 7f6de430d700 1 -- 172.21.15.97:0/1365535182 wait complete. 2023-11-24T06:10:32.574 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:10:32.582 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-11-24T06:10:32.343960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221408,"kb_used_data":5244,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753312,"statfs":{"total":773094113280,"available":772867391488,"internally_reserved":0,"allocated":5369856,"data_stored":2854336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001353"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.334024+0000","last_change":"2023-11-24T06:10:28.193269+0000","last_active":"2023-11-24T06:10:29.334024+0000","last_peered":"2023-11-24T06:10:29.334024+0000","last_clean":"2023-11-24T06:10:29.334024+0000","last_became_active":"2023-11-24T06:10:28.192978+0000","last_became_peered":"2023-11-24T06:10:28.192978+0000","last_unstale":"2023-11-24T06:10:29.334024+0000","last_undegraded":"2023-11-24T06:10:29.334024+0000","last_fullsized":"2023-11-24T06:10:29.334024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:28:17.045045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194702+0000","last_change":"2023-11-24T06:10:28.206982+0000","last_active":"2023-11-24T06:10:29.194702+0000","last_peered":"2023-11-24T06:10:29.194702+0000","last_clean":"2023-11-24T06:10:29.194702+0000","last_became_active":"2023-11-24T06:10:28.206538+0000","last_became_peered":"2023-11-24T06:10:28.206538+0000","last_unstale":"2023-11-24T06:10:29.194702+0000","last_undegraded":"2023-11-24T06:10:29.194702+0000","last_fullsized":"2023-11-24T06:10:29.194702+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:05.224113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.204814+0000","last_change":"2023-11-24T06:10:28.197956+0000","last_active":"2023-11-24T06:10:28.204814+0000","last_peered":"2023-11-24T06:10:28.204814+0000","last_clean":"2023-11-24T06:10:28.204814+0000","last_became_active":"2023-11-24T06:10:28.197675+0000","last_became_peered":"2023-11-24T06:10:28.197675+0000","last_unstale":"2023-11-24T06:10:28.204814+0000","last_undegraded":"2023-11-24T06:10:28.204814+0000","last_fullsized":"2023-11-24T06:10:28.204814+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:45.912268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211204+0000","last_change":"2023-11-24T06:10:28.210321+0000","last_active":"2023-11-24T06:10:28.211204+0000","last_peered":"2023-11-24T06:10:28.211204+0000","last_clean":"2023-11-24T06:10:28.211204+0000","last_became_active":"2023-11-24T06:10:28.210047+0000","last_became_peered":"2023-11-24T06:10:28.210047+0000","last_unstale":"2023-11-24T06:10:28.211204+0000","last_undegraded":"2023-11-24T06:10:28.211204+0000","last_fullsized":"2023-11-24T06:10:28.211204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:34.656648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194637+0000","last_change":"2023-11-24T06:10:28.206019+0000","last_active":"2023-11-24T06:10:29.194637+0000","last_peered":"2023-11-24T06:10:29.194637+0000","last_clean":"2023-11-24T06:10:29.194637+0000","last_became_active":"2023-11-24T06:10:28.205684+0000","last_became_peered":"2023-11-24T06:10:28.205684+0000","last_unstale":"2023-11-24T06:10:29.194637+0000","last_undegraded":"2023-11-24T06:10:29.194637+0000","last_fullsized":"2023-11-24T06:10:29.194637+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:35.779994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207946+0000","last_change":"2023-11-24T06:10:28.203038+0000","last_active":"2023-11-24T06:10:28.207946+0000","last_peered":"2023-11-24T06:10:28.207946+0000","last_clean":"2023-11-24T06:10:28.207946+0000","last_became_active":"2023-11-24T06:10:28.202725+0000","last_became_peered":"2023-11-24T06:10:28.202725+0000","last_unstale":"2023-11-24T06:10:28.207946+0000","last_undegraded":"2023-11-24T06:10:28.207946+0000","last_fullsized":"2023-11-24T06:10:28.207946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:35:42.437608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333747+0000","last_change":"2023-11-24T06:10:28.195333+0000","last_active":"2023-11-24T06:10:29.333747+0000","last_peered":"2023-11-24T06:10:29.333747+0000","last_clean":"2023-11-24T06:10:29.333747+0000","last_became_active":"2023-11-24T06:10:28.195029+0000","last_became_peered":"2023-11-24T06:10:28.195029+0000","last_unstale":"2023-11-24T06:10:29.333747+0000","last_undegraded":"2023-11-24T06:10:29.333747+0000","last_fullsized":"2023-11-24T06:10:29.333747+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:52:18.285812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211818+0000","last_change":"2023-11-24T06:10:28.210971+0000","last_active":"2023-11-24T06:10:28.211818+0000","last_peered":"2023-11-24T06:10:28.211818+0000","last_clean":"2023-11-24T06:10:28.211818+0000","last_became_active":"2023-11-24T06:10:28.210658+0000","last_became_peered":"2023-11-24T06:10:28.210658+0000","last_unstale":"2023-11-24T06:10:28.211818+0000","last_undegraded":"2023-11-24T06:10:28.211818+0000","last_fullsized":"2023-11-24T06:10:28.211818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:45.912268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209490+0000","last_change":"2023-11-24T06:10:28.207114+0000","last_active":"2023-11-24T06:10:28.209490+0000","last_peered":"2023-11-24T06:10:28.209490+0000","last_clean":"2023-11-24T06:10:28.209490+0000","last_became_active":"2023-11-24T06:10:28.206833+0000","last_became_peered":"2023-11-24T06:10:28.206833+0000","last_unstale":"2023-11-24T06:10:28.209490+0000","last_undegraded":"2023-11-24T06:10:28.209490+0000","last_fullsized":"2023-11-24T06:10:28.209490+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:23.018551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211491+0000","last_change":"2023-11-24T06:10:28.210737+0000","last_active":"2023-11-24T06:10:28.211491+0000","last_peered":"2023-11-24T06:10:28.211491+0000","last_clean":"2023-11-24T06:10:28.211491+0000","last_became_active":"2023-11-24T06:10:28.209841+0000","last_became_peered":"2023-11-24T06:10:28.209841+0000","last_unstale":"2023-11-24T06:10:28.211491+0000","last_undegraded":"2023-11-24T06:10:28.211491+0000","last_fullsized":"2023-11-24T06:10:28.211491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:05.905455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209449+0000","last_change":"2023-11-24T06:10:28.206774+0000","last_active":"2023-11-24T06:10:28.209449+0000","last_peered":"2023-11-24T06:10:28.209449+0000","last_clean":"2023-11-24T06:10:28.209449+0000","last_became_active":"2023-11-24T06:10:28.206490+0000","last_became_peered":"2023-11-24T06:10:28.206490+0000","last_unstale":"2023-11-24T06:10:28.209449+0000","last_undegraded":"2023-11-24T06:10:28.209449+0000","last_fullsized":"2023-11-24T06:10:28.209449+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:21.501785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.195231+0000","last_change":"2023-11-24T06:10:28.205517+0000","last_active":"2023-11-24T06:10:29.195231+0000","last_peered":"2023-11-24T06:10:29.195231+0000","last_clean":"2023-11-24T06:10:29.195231+0000","last_became_active":"2023-11-24T06:10:28.204466+0000","last_became_peered":"2023-11-24T06:10:28.204466+0000","last_unstale":"2023-11-24T06:10:29.195231+0000","last_undegraded":"2023-11-24T06:10:29.195231+0000","last_fullsized":"2023-11-24T06:10:29.195231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:24.055216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190459+0000","last_change":"2023-11-24T06:10:28.201928+0000","last_active":"2023-11-24T06:10:29.190459+0000","last_peered":"2023-11-24T06:10:29.190459+0000","last_clean":"2023-11-24T06:10:29.190459+0000","last_became_active":"2023-11-24T06:10:28.201654+0000","last_became_peered":"2023-11-24T06:10:28.201654+0000","last_unstale":"2023-11-24T06:10:29.190459+0000","last_undegraded":"2023-11-24T06:10:29.190459+0000","last_fullsized":"2023-11-24T06:10:29.190459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209518+0000","last_change":"2023-11-24T06:10:28.207309+0000","last_active":"2023-11-24T06:10:28.209518+0000","last_peered":"2023-11-24T06:10:28.209518+0000","last_clean":"2023-11-24T06:10:28.209518+0000","last_became_active":"2023-11-24T06:10:28.207049+0000","last_became_peered":"2023-11-24T06:10:28.207049+0000","last_unstale":"2023-11-24T06:10:28.209518+0000","last_undegraded":"2023-11-24T06:10:28.209518+0000","last_fullsized":"2023-11-24T06:10:28.209518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.484739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208562+0000","last_change":"2023-11-24T06:10:28.204319+0000","last_active":"2023-11-24T06:10:28.208562+0000","last_peered":"2023-11-24T06:10:28.208562+0000","last_clean":"2023-11-24T06:10:28.208562+0000","last_became_active":"2023-11-24T06:10:28.203844+0000","last_became_peered":"2023-11-24T06:10:28.203844+0000","last_unstale":"2023-11-24T06:10:28.208562+0000","last_undegraded":"2023-11-24T06:10:28.208562+0000","last_fullsized":"2023-11-24T06:10:28.208562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:32.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194147+0000","last_change":"2023-11-24T06:10:28.207422+0000","last_active":"2023-11-24T06:10:29.194147+0000","last_peered":"2023-11-24T06:10:29.194147+0000","last_clean":"2023-11-24T06:10:29.194147+0000","last_became_active":"2023-11-24T06:10:28.206984+0000","last_became_peered":"2023-11-24T06:10:28.206984+0000","last_unstale":"2023-11-24T06:10:29.194147+0000","last_undegraded":"2023-11-24T06:10:29.194147+0000","last_fullsized":"2023-11-24T06:10:29.194147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:21.501785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.205016+0000","last_change":"2023-11-24T06:10:28.200334+0000","last_active":"2023-11-24T06:10:28.205016+0000","last_peered":"2023-11-24T06:10:28.205016+0000","last_clean":"2023-11-24T06:10:28.205016+0000","last_became_active":"2023-11-24T06:10:28.200053+0000","last_became_peered":"2023-11-24T06:10:28.200053+0000","last_unstale":"2023-11-24T06:10:28.205016+0000","last_undegraded":"2023-11-24T06:10:28.205016+0000","last_fullsized":"2023-11-24T06:10:28.205016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:21.501785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.206559+0000","last_change":"2023-11-24T06:10:28.203264+0000","last_active":"2023-11-24T06:10:28.206559+0000","last_peered":"2023-11-24T06:10:28.206559+0000","last_clean":"2023-11-24T06:10:28.206559+0000","last_became_active":"2023-11-24T06:10:28.202861+0000","last_became_peered":"2023-11-24T06:10:28.202861+0000","last_unstale":"2023-11-24T06:10:28.206559+0000","last_undegraded":"2023-11-24T06:10:28.206559+0000","last_fullsized":"2023-11-24T06:10:28.206559+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:34.656648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194894+0000","last_change":"2023-11-24T06:10:28.207425+0000","last_active":"2023-11-24T06:10:29.194894+0000","last_peered":"2023-11-24T06:10:29.194894+0000","last_clean":"2023-11-24T06:10:29.194894+0000","last_became_active":"2023-11-24T06:10:28.206960+0000","last_became_peered":"2023-11-24T06:10:28.206960+0000","last_unstale":"2023-11-24T06:10:29.194894+0000","last_undegraded":"2023-11-24T06:10:29.194894+0000","last_fullsized":"2023-11-24T06:10:29.194894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:33.582325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190321+0000","last_change":"2023-11-24T06:10:28.203199+0000","last_active":"2023-11-24T06:10:29.190321+0000","last_peered":"2023-11-24T06:10:29.190321+0000","last_clean":"2023-11-24T06:10:29.190321+0000","last_became_active":"2023-11-24T06:10:28.202796+0000","last_became_peered":"2023-11-24T06:10:28.202796+0000","last_unstale":"2023-11-24T06:10:29.190321+0000","last_undegraded":"2023-11-24T06:10:29.190321+0000","last_fullsized":"2023-11-24T06:10:29.190321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:51:47.343086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209081+0000","last_change":"2023-11-24T06:10:28.207940+0000","last_active":"2023-11-24T06:10:28.209081+0000","last_peered":"2023-11-24T06:10:28.209081+0000","last_clean":"2023-11-24T06:10:28.209081+0000","last_became_active":"2023-11-24T06:10:28.207513+0000","last_became_peered":"2023-11-24T06:10:28.207513+0000","last_unstale":"2023-11-24T06:10:28.209081+0000","last_undegraded":"2023-11-24T06:10:28.209081+0000","last_fullsized":"2023-11-24T06:10:28.209081+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:56.876021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211831+0000","last_change":"2023-11-24T06:10:28.210968+0000","last_active":"2023-11-24T06:10:28.211831+0000","last_peered":"2023-11-24T06:10:28.211831+0000","last_clean":"2023-11-24T06:10:28.211831+0000","last_became_active":"2023-11-24T06:10:28.210603+0000","last_became_peered":"2023-11-24T06:10:28.210603+0000","last_unstale":"2023-11-24T06:10:28.211831+0000","last_undegraded":"2023-11-24T06:10:28.211831+0000","last_fullsized":"2023-11-24T06:10:28.211831+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:48.458766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194239+0000","last_change":"2023-11-24T06:10:28.207716+0000","last_active":"2023-11-24T06:10:29.194239+0000","last_peered":"2023-11-24T06:10:29.194239+0000","last_clean":"2023-11-24T06:10:29.194239+0000","last_became_active":"2023-11-24T06:10:28.207313+0000","last_became_peered":"2023-11-24T06:10:28.207313+0000","last_unstale":"2023-11-24T06:10:29.194239+0000","last_undegraded":"2023-11-24T06:10:29.194239+0000","last_fullsized":"2023-11-24T06:10:29.194239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:34.656648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194782+0000","last_change":"2023-11-24T06:10:28.207778+0000","last_active":"2023-11-24T06:10:29.194782+0000","last_peered":"2023-11-24T06:10:29.194782+0000","last_clean":"2023-11-24T06:10:29.194782+0000","last_became_active":"2023-11-24T06:10:28.207431+0000","last_became_peered":"2023-11-24T06:10:28.207431+0000","last_unstale":"2023-11-24T06:10:29.194782+0000","last_undegraded":"2023-11-24T06:10:29.194782+0000","last_fullsized":"2023-11-24T06:10:29.194782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:23.018551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194777+0000","last_change":"2023-11-24T06:10:28.203748+0000","last_active":"2023-11-24T06:10:29.194777+0000","last_peered":"2023-11-24T06:10:29.194777+0000","last_clean":"2023-11-24T06:10:29.194777+0000","last_became_active":"2023-11-24T06:10:28.202890+0000","last_became_peered":"2023-11-24T06:10:28.202890+0000","last_unstale":"2023-11-24T06:10:29.194777+0000","last_undegraded":"2023-11-24T06:10:29.194777+0000","last_fullsized":"2023-11-24T06:10:29.194777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:01.465488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190158+0000","last_change":"2023-11-24T06:10:28.206268+0000","last_active":"2023-11-24T06:10:29.190158+0000","last_peered":"2023-11-24T06:10:29.190158+0000","last_clean":"2023-11-24T06:10:29.190158+0000","last_became_active":"2023-11-24T06:10:28.205999+0000","last_became_peered":"2023-11-24T06:10:28.205999+0000","last_unstale":"2023-11-24T06:10:29.190158+0000","last_undegraded":"2023-11-24T06:10:29.190158+0000","last_fullsized":"2023-11-24T06:10:29.190158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:33.863577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.204835+0000","last_change":"2023-11-24T06:10:28.198939+0000","last_active":"2023-11-24T06:10:28.204835+0000","last_peered":"2023-11-24T06:10:28.204835+0000","last_clean":"2023-11-24T06:10:28.204835+0000","last_became_active":"2023-11-24T06:10:28.198631+0000","last_became_peered":"2023-11-24T06:10:28.198631+0000","last_unstale":"2023-11-24T06:10:28.204835+0000","last_undegraded":"2023-11-24T06:10:28.204835+0000","last_fullsized":"2023-11-24T06:10:28.204835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.484739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194928+0000","last_change":"2023-11-24T06:10:28.205792+0000","last_active":"2023-11-24T06:10:29.194928+0000","last_peered":"2023-11-24T06:10:29.194928+0000","last_clean":"2023-11-24T06:10:29.194928+0000","last_became_active":"2023-11-24T06:10:28.204872+0000","last_became_peered":"2023-11-24T06:10:28.204872+0000","last_unstale":"2023-11-24T06:10:29.194928+0000","last_undegraded":"2023-11-24T06:10:29.194928+0000","last_fullsized":"2023-11-24T06:10:29.194928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:06.200967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194410+0000","last_change":"2023-11-24T06:10:28.202864+0000","last_active":"2023-11-24T06:10:29.194410+0000","last_peered":"2023-11-24T06:10:29.194410+0000","last_clean":"2023-11-24T06:10:29.194410+0000","last_became_active":"2023-11-24T06:10:28.202559+0000","last_became_peered":"2023-11-24T06:10:28.202559+0000","last_unstale":"2023-11-24T06:10:29.194410+0000","last_undegraded":"2023-11-24T06:10:29.194410+0000","last_fullsized":"2023-11-24T06:10:29.194410+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:30.721899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.202100+0000","last_change":"2023-11-24T06:10:28.195813+0000","last_active":"2023-11-24T06:10:28.202100+0000","last_peered":"2023-11-24T06:10:28.202100+0000","last_clean":"2023-11-24T06:10:28.202100+0000","last_became_active":"2023-11-24T06:10:28.195554+0000","last_became_peered":"2023-11-24T06:10:28.195554+0000","last_unstale":"2023-11-24T06:10:28.202100+0000","last_undegraded":"2023-11-24T06:10:28.202100+0000","last_fullsized":"2023-11-24T06:10:28.202100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:05.162020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208802+0000","last_change":"2023-11-24T06:10:28.204704+0000","last_active":"2023-11-24T06:10:28.208802+0000","last_peered":"2023-11-24T06:10:28.208802+0000","last_clean":"2023-11-24T06:10:28.208802+0000","last_became_active":"2023-11-24T06:10:28.204348+0000","last_became_peered":"2023-11-24T06:10:28.204348+0000","last_unstale":"2023-11-24T06:10:28.208802+0000","last_undegraded":"2023-11-24T06:10:28.208802+0000","last_fullsized":"2023-11-24T06:10:28.208802+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:28:17.045045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190124+0000","last_change":"2023-11-24T06:10:28.198505+0000","last_active":"2023-11-24T06:10:29.190124+0000","last_peered":"2023-11-24T06:10:29.190124+0000","last_clean":"2023-11-24T06:10:29.190124+0000","last_became_active":"2023-11-24T06:10:28.198240+0000","last_became_peered":"2023-11-24T06:10:28.198240+0000","last_unstale":"2023-11-24T06:10:29.190124+0000","last_undegraded":"2023-11-24T06:10:29.190124+0000","last_fullsized":"2023-11-24T06:10:29.190124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:33:39.258906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.206894+0000","last_change":"2023-11-24T06:10:28.202058+0000","last_active":"2023-11-24T06:10:28.206894+0000","last_peered":"2023-11-24T06:10:28.206894+0000","last_clean":"2023-11-24T06:10:28.206894+0000","last_became_active":"2023-11-24T06:10:28.201727+0000","last_became_peered":"2023-11-24T06:10:28.201727+0000","last_unstale":"2023-11-24T06:10:28.206894+0000","last_undegraded":"2023-11-24T06:10:28.206894+0000","last_fullsized":"2023-11-24T06:10:28.206894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:28.409249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511596+0000","last_change":"2023-11-24T06:10:27.191631+0000","last_active":"2023-11-24T06:10:28.511596+0000","last_peered":"2023-11-24T06:10:28.511596+0000","last_clean":"2023-11-24T06:10:28.511596+0000","last_became_active":"2023-11-24T06:10:27.191257+0000","last_became_peered":"2023-11-24T06:10:27.191257+0000","last_unstale":"2023-11-24T06:10:28.511596+0000","last_undegraded":"2023-11-24T06:10:28.511596+0000","last_fullsized":"2023-11-24T06:10:28.511596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:00.577272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194390+0000","last_change":"2023-11-24T06:10:28.203523+0000","last_active":"2023-11-24T06:10:29.194390+0000","last_peered":"2023-11-24T06:10:29.194390+0000","last_clean":"2023-11-24T06:10:29.194390+0000","last_became_active":"2023-11-24T06:10:28.203257+0000","last_became_peered":"2023-11-24T06:10:28.203257+0000","last_unstale":"2023-11-24T06:10:29.194390+0000","last_undegraded":"2023-11-24T06:10:29.194390+0000","last_fullsized":"2023-11-24T06:10:29.194390+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:05.162020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511499+0000","last_change":"2023-11-24T06:10:27.193130+0000","last_active":"2023-11-24T06:10:28.511499+0000","last_peered":"2023-11-24T06:10:28.511499+0000","last_clean":"2023-11-24T06:10:28.511499+0000","last_became_active":"2023-11-24T06:10:27.192856+0000","last_became_peered":"2023-11-24T06:10:27.192856+0000","last_unstale":"2023-11-24T06:10:28.511499+0000","last_undegraded":"2023-11-24T06:10:28.511499+0000","last_fullsized":"2023-11-24T06:10:28.511499+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:00.460189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.195060+0000","last_change":"2023-11-24T06:10:28.206057+0000","last_active":"2023-11-24T06:10:29.195060+0000","last_peered":"2023-11-24T06:10:29.195060+0000","last_clean":"2023-11-24T06:10:29.195060+0000","last_became_active":"2023-11-24T06:10:28.205746+0000","last_became_peered":"2023-11-24T06:10:28.205746+0000","last_unstale":"2023-11-24T06:10:29.195060+0000","last_undegraded":"2023-11-24T06:10:29.195060+0000","last_fullsized":"2023-11-24T06:10:29.195060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:35:42.437608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.512122+0000","last_change":"2023-11-24T06:10:27.200130+0000","last_active":"2023-11-24T06:10:28.512122+0000","last_peered":"2023-11-24T06:10:28.512122+0000","last_clean":"2023-11-24T06:10:28.512122+0000","last_became_active":"2023-11-24T06:10:27.199881+0000","last_became_peered":"2023-11-24T06:10:27.199881+0000","last_unstale":"2023-11-24T06:10:28.512122+0000","last_undegraded":"2023-11-24T06:10:28.512122+0000","last_fullsized":"2023-11-24T06:10:28.512122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:22.013253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209683+0000","last_change":"2023-11-24T06:10:28.208662+0000","last_active":"2023-11-24T06:10:28.209683+0000","last_peered":"2023-11-24T06:10:28.209683+0000","last_clean":"2023-11-24T06:10:28.209683+0000","last_became_active":"2023-11-24T06:10:28.208386+0000","last_became_peered":"2023-11-24T06:10:28.208386+0000","last_unstale":"2023-11-24T06:10:28.209683+0000","last_undegraded":"2023-11-24T06:10:28.209683+0000","last_fullsized":"2023-11-24T06:10:28.209683+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:05.162020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333799+0000","last_change":"2023-11-24T06:10:27.197154+0000","last_active":"2023-11-24T06:10:29.333799+0000","last_peered":"2023-11-24T06:10:29.333799+0000","last_clean":"2023-11-24T06:10:29.333799+0000","last_became_active":"2023-11-24T06:10:27.196910+0000","last_became_peered":"2023-11-24T06:10:27.196910+0000","last_unstale":"2023-11-24T06:10:29.333799+0000","last_undegraded":"2023-11-24T06:10:29.333799+0000","last_fullsized":"2023-11-24T06:10:29.333799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:30.476022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.204899+0000","last_change":"2023-11-24T06:10:28.199626+0000","last_active":"2023-11-24T06:10:28.204899+0000","last_peered":"2023-11-24T06:10:28.204899+0000","last_clean":"2023-11-24T06:10:28.204899+0000","last_became_active":"2023-11-24T06:10:28.199351+0000","last_became_peered":"2023-11-24T06:10:28.199351+0000","last_unstale":"2023-11-24T06:10:28.204899+0000","last_undegraded":"2023-11-24T06:10:28.204899+0000","last_fullsized":"2023-11-24T06:10:28.204899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:28.409249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511312+0000","last_change":"2023-11-24T06:10:27.196884+0000","last_active":"2023-11-24T06:10:28.511312+0000","last_peered":"2023-11-24T06:10:28.511312+0000","last_clean":"2023-11-24T06:10:28.511312+0000","last_became_active":"2023-11-24T06:10:27.196623+0000","last_became_peered":"2023-11-24T06:10:27.196623+0000","last_unstale":"2023-11-24T06:10:28.511312+0000","last_undegraded":"2023-11-24T06:10:28.511312+0000","last_fullsized":"2023-11-24T06:10:28.511312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:33:37.025817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.205697+0000","last_change":"2023-11-24T06:10:28.204060+0000","last_active":"2023-11-24T06:10:28.205697+0000","last_peered":"2023-11-24T06:10:28.205697+0000","last_clean":"2023-11-24T06:10:28.205697+0000","last_became_active":"2023-11-24T06:10:28.203777+0000","last_became_peered":"2023-11-24T06:10:28.203777+0000","last_unstale":"2023-11-24T06:10:28.205697+0000","last_undegraded":"2023-11-24T06:10:28.205697+0000","last_fullsized":"2023-11-24T06:10:28.205697+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:51:47.343086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.512067+0000","last_change":"2023-11-24T06:10:27.189950+0000","last_active":"2023-11-24T06:10:28.512067+0000","last_peered":"2023-11-24T06:10:28.512067+0000","last_clean":"2023-11-24T06:10:28.512067+0000","last_became_active":"2023-11-24T06:10:27.189668+0000","last_became_peered":"2023-11-24T06:10:27.189668+0000","last_unstale":"2023-11-24T06:10:28.512067+0000","last_undegraded":"2023-11-24T06:10:28.512067+0000","last_fullsized":"2023-11-24T06:10:28.512067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:00.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211440+0000","last_change":"2023-11-24T06:10:28.210641+0000","last_active":"2023-11-24T06:10:28.211440+0000","last_peered":"2023-11-24T06:10:28.211440+0000","last_clean":"2023-11-24T06:10:28.211440+0000","last_became_active":"2023-11-24T06:10:28.210094+0000","last_became_peered":"2023-11-24T06:10:28.210094+0000","last_unstale":"2023-11-24T06:10:28.211440+0000","last_undegraded":"2023-11-24T06:10:28.211440+0000","last_fullsized":"2023-11-24T06:10:28.211440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:33.863577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194367+0000","last_change":"2023-11-24T06:10:27.197066+0000","last_active":"2023-11-24T06:10:29.194367+0000","last_peered":"2023-11-24T06:10:29.194367+0000","last_clean":"2023-11-24T06:10:29.194367+0000","last_became_active":"2023-11-24T06:10:27.196732+0000","last_became_peered":"2023-11-24T06:10:27.196732+0000","last_unstale":"2023-11-24T06:10:29.194367+0000","last_undegraded":"2023-11-24T06:10:29.194367+0000","last_fullsized":"2023-11-24T06:10:29.194367+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:32.577027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.195344+0000","last_change":"2023-11-24T06:10:28.203495+0000","last_active":"2023-11-24T06:10:29.195344+0000","last_peered":"2023-11-24T06:10:29.195344+0000","last_clean":"2023-11-24T06:10:29.195344+0000","last_became_active":"2023-11-24T06:10:28.203203+0000","last_became_peered":"2023-11-24T06:10:28.203203+0000","last_unstale":"2023-11-24T06:10:29.195344+0000","last_undegraded":"2023-11-24T06:10:29.195344+0000","last_fullsized":"2023-11-24T06:10:29.195344+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:05.905455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.196525+0000","last_change":"2023-11-24T06:10:27.203319+0000","last_active":"2023-11-24T06:10:28.196525+0000","last_peered":"2023-11-24T06:10:28.196525+0000","last_clean":"2023-11-24T06:10:28.196525+0000","last_became_active":"2023-11-24T06:10:27.203031+0000","last_became_peered":"2023-11-24T06:10:27.203031+0000","last_unstale":"2023-11-24T06:10:28.196525+0000","last_undegraded":"2023-11-24T06:10:28.196525+0000","last_fullsized":"2023-11-24T06:10:28.196525+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:32.577027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.206535+0000","last_change":"2023-11-24T06:10:28.202974+0000","last_active":"2023-11-24T06:10:28.206535+0000","last_peered":"2023-11-24T06:10:28.206535+0000","last_clean":"2023-11-24T06:10:28.206535+0000","last_became_active":"2023-11-24T06:10:28.202707+0000","last_became_peered":"2023-11-24T06:10:28.202707+0000","last_unstale":"2023-11-24T06:10:28.206535+0000","last_undegraded":"2023-11-24T06:10:28.206535+0000","last_fullsized":"2023-11-24T06:10:28.206535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:48.458766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.188865+0000","last_change":"2023-11-24T06:10:27.188549+0000","last_active":"2023-11-24T06:10:28.188865+0000","last_peered":"2023-11-24T06:10:28.188865+0000","last_clean":"2023-11-24T06:10:28.188865+0000","last_became_active":"2023-11-24T06:10:27.188101+0000","last_became_peered":"2023-11-24T06:10:27.188101+0000","last_unstale":"2023-11-24T06:10:28.188865+0000","last_undegraded":"2023-11-24T06:10:28.188865+0000","last_fullsized":"2023-11-24T06:10:28.188865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:29.716601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.334063+0000","last_change":"2023-11-24T06:10:28.200336+0000","last_active":"2023-11-24T06:10:29.334063+0000","last_peered":"2023-11-24T06:10:29.334063+0000","last_clean":"2023-11-24T06:10:29.334063+0000","last_became_active":"2023-11-24T06:10:28.200050+0000","last_became_peered":"2023-11-24T06:10:28.200050+0000","last_unstale":"2023-11-24T06:10:29.334063+0000","last_undegraded":"2023-11-24T06:10:29.334063+0000","last_fullsized":"2023-11-24T06:10:29.334063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:43.703372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.202693+0000","last_change":"2023-11-24T06:10:27.201868+0000","last_active":"2023-11-24T06:10:28.202693+0000","last_peered":"2023-11-24T06:10:28.202693+0000","last_clean":"2023-11-24T06:10:28.202693+0000","last_became_active":"2023-11-24T06:10:27.201425+0000","last_became_peered":"2023-11-24T06:10:27.201425+0000","last_unstale":"2023-11-24T06:10:28.202693+0000","last_undegraded":"2023-11-24T06:10:28.202693+0000","last_fullsized":"2023-11-24T06:10:28.202693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:03.189139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207863+0000","last_change":"2023-11-24T06:10:28.202383+0000","last_active":"2023-11-24T06:10:28.207863+0000","last_peered":"2023-11-24T06:10:28.207863+0000","last_clean":"2023-11-24T06:10:28.207863+0000","last_became_active":"2023-11-24T06:10:28.202108+0000","last_became_peered":"2023-11-24T06:10:28.202108+0000","last_unstale":"2023-11-24T06:10:28.207863+0000","last_undegraded":"2023-11-24T06:10:28.207863+0000","last_fullsized":"2023-11-24T06:10:28.207863+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:52:18.285812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333976+0000","last_change":"2023-11-24T06:10:27.200059+0000","last_active":"2023-11-24T06:10:29.333976+0000","last_peered":"2023-11-24T06:10:29.333976+0000","last_clean":"2023-11-24T06:10:29.333976+0000","last_became_active":"2023-11-24T06:10:27.199806+0000","last_became_peered":"2023-11-24T06:10:27.199806+0000","last_unstale":"2023-11-24T06:10:29.333976+0000","last_undegraded":"2023-11-24T06:10:29.333976+0000","last_fullsized":"2023-11-24T06:10:29.333976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:04.218814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333866+0000","last_change":"2023-11-24T06:10:28.196009+0000","last_active":"2023-11-24T06:10:29.333866+0000","last_peered":"2023-11-24T06:10:29.333866+0000","last_clean":"2023-11-24T06:10:29.333866+0000","last_became_active":"2023-11-24T06:10:28.195670+0000","last_became_peered":"2023-11-24T06:10:28.195670+0000","last_unstale":"2023-11-24T06:10:29.333866+0000","last_undegraded":"2023-11-24T06:10:29.333866+0000","last_fullsized":"2023-11-24T06:10:29.333866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:56.876021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190334+0000","last_change":"2023-11-24T06:10:27.190251+0000","last_active":"2023-11-24T06:10:29.190334+0000","last_peered":"2023-11-24T06:10:29.190334+0000","last_clean":"2023-11-24T06:10:29.190334+0000","last_became_active":"2023-11-24T06:10:27.190065+0000","last_became_peered":"2023-11-24T06:10:27.190065+0000","last_unstale":"2023-11-24T06:10:29.190334+0000","last_undegraded":"2023-11-24T06:10:29.190334+0000","last_fullsized":"2023-11-24T06:10:29.190334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:01.071112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194285+0000","last_change":"2023-11-24T06:10:28.204100+0000","last_active":"2023-11-24T06:10:29.194285+0000","last_peered":"2023-11-24T06:10:29.194285+0000","last_clean":"2023-11-24T06:10:29.194285+0000","last_became_active":"2023-11-24T06:10:28.203867+0000","last_became_peered":"2023-11-24T06:10:28.203867+0000","last_unstale":"2023-11-24T06:10:29.194285+0000","last_undegraded":"2023-11-24T06:10:29.194285+0000","last_fullsized":"2023-11-24T06:10:29.194285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:45.912268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511098+0000","last_change":"2023-11-24T06:10:27.194385+0000","last_active":"2023-11-24T06:10:28.511098+0000","last_peered":"2023-11-24T06:10:28.511098+0000","last_clean":"2023-11-24T06:10:28.511098+0000","last_became_active":"2023-11-24T06:10:27.194099+0000","last_became_peered":"2023-11-24T06:10:27.194099+0000","last_unstale":"2023-11-24T06:10:28.511098+0000","last_undegraded":"2023-11-24T06:10:28.511098+0000","last_fullsized":"2023-11-24T06:10:28.511098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:05.088649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.205717+0000","last_change":"2023-11-24T06:10:28.204380+0000","last_active":"2023-11-24T06:10:28.205717+0000","last_peered":"2023-11-24T06:10:28.205717+0000","last_clean":"2023-11-24T06:10:28.205717+0000","last_became_active":"2023-11-24T06:10:28.204113+0000","last_became_peered":"2023-11-24T06:10:28.204113+0000","last_unstale":"2023-11-24T06:10:28.205717+0000","last_undegraded":"2023-11-24T06:10:28.205717+0000","last_fullsized":"2023-11-24T06:10:28.205717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333780+0000","last_change":"2023-11-24T06:10:27.199964+0000","last_active":"2023-11-24T06:10:29.333780+0000","last_peered":"2023-11-24T06:10:29.333780+0000","last_clean":"2023-11-24T06:10:29.333780+0000","last_became_active":"2023-11-24T06:10:27.199684+0000","last_became_peered":"2023-11-24T06:10:27.199684+0000","last_unstale":"2023-11-24T06:10:29.333780+0000","last_undegraded":"2023-11-24T06:10:29.333780+0000","last_fullsized":"2023-11-24T06:10:29.333780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:32.577027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207896+0000","last_change":"2023-11-24T06:10:28.202239+0000","last_active":"2023-11-24T06:10:28.207896+0000","last_peered":"2023-11-24T06:10:28.207896+0000","last_clean":"2023-11-24T06:10:28.207896+0000","last_became_active":"2023-11-24T06:10:28.201859+0000","last_became_peered":"2023-11-24T06:10:28.201859+0000","last_unstale":"2023-11-24T06:10:28.207896+0000","last_undegraded":"2023-11-24T06:10:28.207896+0000","last_fullsized":"2023-11-24T06:10:28.207896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:41.131179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.197377+0000","last_change":"2023-11-24T06:10:27.194124+0000","last_active":"2023-11-24T06:10:28.197377+0000","last_peered":"2023-11-24T06:10:28.197377+0000","last_clean":"2023-11-24T06:10:28.197377+0000","last_became_active":"2023-11-24T06:10:27.193669+0000","last_became_peered":"2023-11-24T06:10:27.193669+0000","last_unstale":"2023-11-24T06:10:28.197377+0000","last_undegraded":"2023-11-24T06:10:28.197377+0000","last_fullsized":"2023-11-24T06:10:28.197377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:52:46.350444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333860+0000","last_change":"2023-11-24T06:10:28.208826+0000","last_active":"2023-11-24T06:10:29.333860+0000","last_peered":"2023-11-24T06:10:29.333860+0000","last_clean":"2023-11-24T06:10:29.333860+0000","last_became_active":"2023-11-24T06:10:28.208404+0000","last_became_peered":"2023-11-24T06:10:28.208404+0000","last_unstale":"2023-11-24T06:10:29.333860+0000","last_undegraded":"2023-11-24T06:10:29.333860+0000","last_fullsized":"2023-11-24T06:10:29.333860+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:53:30.744751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.512865+0000","last_change":"2023-11-24T06:10:27.198528+0000","last_active":"2023-11-24T06:10:28.512865+0000","last_peered":"2023-11-24T06:10:28.512865+0000","last_clean":"2023-11-24T06:10:28.512865+0000","last_became_active":"2023-11-24T06:10:27.198282+0000","last_became_peered":"2023-11-24T06:10:27.198282+0000","last_unstale":"2023-11-24T06:10:28.512865+0000","last_undegraded":"2023-11-24T06:10:28.512865+0000","last_fullsized":"2023-11-24T06:10:28.512865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:29.716601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190264+0000","last_change":"2023-11-24T06:10:28.200216+0000","last_active":"2023-11-24T06:10:29.190264+0000","last_peered":"2023-11-24T06:10:29.190264+0000","last_clean":"2023-11-24T06:10:29.190264+0000","last_became_active":"2023-11-24T06:10:28.199923+0000","last_became_peered":"2023-11-24T06:10:28.199923+0000","last_unstale":"2023-11-24T06:10:29.190264+0000","last_undegraded":"2023-11-24T06:10:29.190264+0000","last_fullsized":"2023-11-24T06:10:29.190264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:41.703490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190331+0000","last_change":"2023-11-24T06:10:27.199825+0000","last_active":"2023-11-24T06:10:29.190331+0000","last_peered":"2023-11-24T06:10:29.190331+0000","last_clean":"2023-11-24T06:10:29.190331+0000","last_became_active":"2023-11-24T06:10:27.199551+0000","last_became_peered":"2023-11-24T06:10:27.199551+0000","last_unstale":"2023-11-24T06:10:29.190331+0000","last_undegraded":"2023-11-24T06:10:29.190331+0000","last_fullsized":"2023-11-24T06:10:29.190331+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:03:03.382239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207327+0000","last_change":"2023-11-24T06:10:28.205842+0000","last_active":"2023-11-24T06:10:28.207327+0000","last_peered":"2023-11-24T06:10:28.207327+0000","last_clean":"2023-11-24T06:10:28.207327+0000","last_became_active":"2023-11-24T06:10:28.205532+0000","last_became_peered":"2023-11-24T06:10:28.205532+0000","last_unstale":"2023-11-24T06:10:28.207327+0000","last_undegraded":"2023-11-24T06:10:28.207327+0000","last_fullsized":"2023-11-24T06:10:28.207327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:41.703490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.510986+0000","last_change":"2023-11-24T06:10:27.198614+0000","last_active":"2023-11-24T06:10:28.510986+0000","last_peered":"2023-11-24T06:10:28.510986+0000","last_clean":"2023-11-24T06:10:28.510986+0000","last_became_active":"2023-11-24T06:10:27.198305+0000","last_became_peered":"2023-11-24T06:10:27.198305+0000","last_unstale":"2023-11-24T06:10:28.510986+0000","last_undegraded":"2023-11-24T06:10:28.510986+0000","last_fullsized":"2023-11-24T06:10:28.510986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:22.013253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194576+0000","last_change":"2023-11-24T06:10:28.207906+0000","last_active":"2023-11-24T06:10:29.194576+0000","last_peered":"2023-11-24T06:10:29.194576+0000","last_clean":"2023-11-24T06:10:29.194576+0000","last_became_active":"2023-11-24T06:10:28.206941+0000","last_became_peered":"2023-11-24T06:10:28.206941+0000","last_unstale":"2023-11-24T06:10:29.194576+0000","last_undegraded":"2023-11-24T06:10:29.194576+0000","last_fullsized":"2023-11-24T06:10:29.194576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:01.452285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194955+0000","last_change":"2023-11-24T06:10:27.199302+0000","last_active":"2023-11-24T06:10:29.194955+0000","last_peered":"2023-11-24T06:10:29.194955+0000","last_clean":"2023-11-24T06:10:29.194955+0000","last_became_active":"2023-11-24T06:10:27.198968+0000","last_became_peered":"2023-11-24T06:10:27.198968+0000","last_unstale":"2023-11-24T06:10:29.194955+0000","last_undegraded":"2023-11-24T06:10:29.194955+0000","last_fullsized":"2023-11-24T06:10:29.194955+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:38.091507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194973+0000","last_change":"2023-11-24T06:10:28.208328+0000","last_active":"2023-11-24T06:10:29.194973+0000","last_peered":"2023-11-24T06:10:29.194973+0000","last_clean":"2023-11-24T06:10:29.194973+0000","last_became_active":"2023-11-24T06:10:28.208000+0000","last_became_peered":"2023-11-24T06:10:28.208000+0000","last_unstale":"2023-11-24T06:10:29.194973+0000","last_undegraded":"2023-11-24T06:10:29.194973+0000","last_fullsized":"2023-11-24T06:10:29.194973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:27.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.197344+0000","last_change":"2023-11-24T06:10:27.194472+0000","last_active":"2023-11-24T06:10:28.197344+0000","last_peered":"2023-11-24T06:10:28.197344+0000","last_clean":"2023-11-24T06:10:28.197344+0000","last_became_active":"2023-11-24T06:10:27.194169+0000","last_became_peered":"2023-11-24T06:10:27.194169+0000","last_unstale":"2023-11-24T06:10:28.197344+0000","last_undegraded":"2023-11-24T06:10:28.197344+0000","last_fullsized":"2023-11-24T06:10:28.197344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:53.094666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208705+0000","last_change":"2023-11-24T06:10:28.204667+0000","last_active":"2023-11-24T06:10:28.208705+0000","last_peered":"2023-11-24T06:10:28.208705+0000","last_clean":"2023-11-24T06:10:28.208705+0000","last_became_active":"2023-11-24T06:10:28.204193+0000","last_became_peered":"2023-11-24T06:10:28.204193+0000","last_unstale":"2023-11-24T06:10:28.208705+0000","last_undegraded":"2023-11-24T06:10:28.208705+0000","last_fullsized":"2023-11-24T06:10:28.208705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:53:30.744751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511419+0000","last_change":"2023-11-24T06:10:27.198819+0000","last_active":"2023-11-24T06:10:28.511419+0000","last_peered":"2023-11-24T06:10:28.511419+0000","last_clean":"2023-11-24T06:10:28.511419+0000","last_became_active":"2023-11-24T06:10:27.198522+0000","last_became_peered":"2023-11-24T06:10:27.198522+0000","last_unstale":"2023-11-24T06:10:28.511419+0000","last_undegraded":"2023-11-24T06:10:28.511419+0000","last_fullsized":"2023-11-24T06:10:28.511419+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:15.013775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207165+0000","last_change":"2023-11-24T06:10:28.199999+0000","last_active":"2023-11-24T06:10:28.207165+0000","last_peered":"2023-11-24T06:10:28.207165+0000","last_clean":"2023-11-24T06:10:28.207165+0000","last_became_active":"2023-11-24T06:10:28.199645+0000","last_became_peered":"2023-11-24T06:10:28.199645+0000","last_unstale":"2023-11-24T06:10:28.207165+0000","last_undegraded":"2023-11-24T06:10:28.207165+0000","last_fullsized":"2023-11-24T06:10:28.207165+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:30.721899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194842+0000","last_change":"2023-11-24T06:10:27.190895+0000","last_active":"2023-11-24T06:10:29.194842+0000","last_peered":"2023-11-24T06:10:29.194842+0000","last_clean":"2023-11-24T06:10:29.194842+0000","last_became_active":"2023-11-24T06:10:27.190630+0000","last_became_peered":"2023-11-24T06:10:27.190630+0000","last_unstale":"2023-11-24T06:10:29.194842+0000","last_undegraded":"2023-11-24T06:10:29.194842+0000","last_fullsized":"2023-11-24T06:10:29.194842+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:26.210074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190129+0000","last_change":"2023-11-24T06:10:28.192165+0000","last_active":"2023-11-24T06:10:29.190129+0000","last_peered":"2023-11-24T06:10:29.190129+0000","last_clean":"2023-11-24T06:10:29.190129+0000","last_became_active":"2023-11-24T06:10:28.191917+0000","last_became_peered":"2023-11-24T06:10:28.191917+0000","last_unstale":"2023-11-24T06:10:29.190129+0000","last_undegraded":"2023-11-24T06:10:29.190129+0000","last_fullsized":"2023-11-24T06:10:29.190129+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:41.612455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.196307+0000","last_change":"2023-11-24T06:10:27.202260+0000","last_active":"2023-11-24T06:10:28.196307+0000","last_peered":"2023-11-24T06:10:28.196307+0000","last_clean":"2023-11-24T06:10:28.196307+0000","last_became_active":"2023-11-24T06:10:27.202068+0000","last_became_peered":"2023-11-24T06:10:27.202068+0000","last_unstale":"2023-11-24T06:10:28.196307+0000","last_undegraded":"2023-11-24T06:10:28.196307+0000","last_fullsized":"2023-11-24T06:10:28.196307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:27.403951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208951+0000","last_change":"2023-11-24T06:10:28.206494+0000","last_active":"2023-11-24T06:10:28.208951+0000","last_peered":"2023-11-24T06:10:28.208951+0000","last_clean":"2023-11-24T06:10:28.208951+0000","last_became_active":"2023-11-24T06:10:28.206185+0000","last_became_peered":"2023-11-24T06:10:28.206185+0000","last_unstale":"2023-11-24T06:10:28.208951+0000","last_undegraded":"2023-11-24T06:10:28.208951+0000","last_fullsized":"2023-11-24T06:10:28.208951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:05.162020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333624+0000","last_change":"2023-11-24T06:10:27.200397+0000","last_active":"2023-11-24T06:10:29.333624+0000","last_peered":"2023-11-24T06:10:29.333624+0000","last_clean":"2023-11-24T06:10:29.333624+0000","last_became_active":"2023-11-24T06:10:27.200128+0000","last_became_peered":"2023-11-24T06:10:27.200128+0000","last_unstale":"2023-11-24T06:10:29.333624+0000","last_undegraded":"2023-11-24T06:10:29.333624+0000","last_fullsized":"2023-11-24T06:10:29.333624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:21.696001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194534+0000","last_change":"2023-11-24T06:10:28.202948+0000","last_active":"2023-11-24T06:10:29.194534+0000","last_peered":"2023-11-24T06:10:29.194534+0000","last_clean":"2023-11-24T06:10:29.194534+0000","last_became_active":"2023-11-24T06:10:28.202681+0000","last_became_peered":"2023-11-24T06:10:28.202681+0000","last_unstale":"2023-11-24T06:10:29.194534+0000","last_undegraded":"2023-11-24T06:10:29.194534+0000","last_fullsized":"2023-11-24T06:10:29.194534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:28.409249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.197408+0000","last_change":"2023-11-24T06:10:27.194281+0000","last_active":"2023-11-24T06:10:28.197408+0000","last_peered":"2023-11-24T06:10:28.197408+0000","last_clean":"2023-11-24T06:10:28.197408+0000","last_became_active":"2023-11-24T06:10:27.193593+0000","last_became_peered":"2023-11-24T06:10:27.193593+0000","last_unstale":"2023-11-24T06:10:28.197408+0000","last_undegraded":"2023-11-24T06:10:28.197408+0000","last_fullsized":"2023-11-24T06:10:28.197408+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:42.698073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207292+0000","last_change":"2023-11-24T06:10:28.205820+0000","last_active":"2023-11-24T06:10:28.207292+0000","last_peered":"2023-11-24T06:10:28.207292+0000","last_clean":"2023-11-24T06:10:28.207292+0000","last_became_active":"2023-11-24T06:10:28.205438+0000","last_became_peered":"2023-11-24T06:10:28.205438+0000","last_unstale":"2023-11-24T06:10:28.207292+0000","last_undegraded":"2023-11-24T06:10:28.207292+0000","last_fullsized":"2023-11-24T06:10:28.207292+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:01.465488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.334066+0000","last_change":"2023-11-24T06:10:27.200126+0000","last_active":"2023-11-24T06:10:29.334066+0000","last_peered":"2023-11-24T06:10:29.334066+0000","last_clean":"2023-11-24T06:10:29.334066+0000","last_became_active":"2023-11-24T06:10:27.199846+0000","last_became_peered":"2023-11-24T06:10:27.199846+0000","last_unstale":"2023-11-24T06:10:29.334066+0000","last_undegraded":"2023-11-24T06:10:29.334066+0000","last_fullsized":"2023-11-24T06:10:29.334066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:05.195669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333990+0000","last_change":"2023-11-24T06:10:28.199429+0000","last_active":"2023-11-24T06:10:29.333990+0000","last_peered":"2023-11-24T06:10:29.333990+0000","last_clean":"2023-11-24T06:10:29.333990+0000","last_became_active":"2023-11-24T06:10:28.199149+0000","last_became_peered":"2023-11-24T06:10:28.199149+0000","last_unstale":"2023-11-24T06:10:29.333990+0000","last_undegraded":"2023-11-24T06:10:29.333990+0000","last_fullsized":"2023-11-24T06:10:29.333990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:32.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.192043+0000","last_change":"2023-11-24T06:10:27.199260+0000","last_active":"2023-11-24T06:10:28.192043+0000","last_peered":"2023-11-24T06:10:28.192043+0000","last_clean":"2023-11-24T06:10:28.192043+0000","last_became_active":"2023-11-24T06:10:27.199002+0000","last_became_peered":"2023-11-24T06:10:27.199002+0000","last_unstale":"2023-11-24T06:10:28.192043+0000","last_undegraded":"2023-11-24T06:10:28.192043+0000","last_fullsized":"2023-11-24T06:10:28.192043+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:00.460189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208682+0000","last_change":"2023-11-24T06:10:28.204499+0000","last_active":"2023-11-24T06:10:28.208682+0000","last_peered":"2023-11-24T06:10:28.208682+0000","last_clean":"2023-11-24T06:10:28.208682+0000","last_became_active":"2023-11-24T06:10:28.203878+0000","last_became_peered":"2023-11-24T06:10:28.203878+0000","last_unstale":"2023-11-24T06:10:28.208682+0000","last_undegraded":"2023-11-24T06:10:28.208682+0000","last_fullsized":"2023-11-24T06:10:28.208682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:24.055216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333924+0000","last_change":"2023-11-24T06:10:27.198660+0000","last_active":"2023-11-24T06:10:29.333924+0000","last_peered":"2023-11-24T06:10:29.333924+0000","last_clean":"2023-11-24T06:10:29.333924+0000","last_became_active":"2023-11-24T06:10:27.198406+0000","last_became_peered":"2023-11-24T06:10:27.198406+0000","last_unstale":"2023-11-24T06:10:29.333924+0000","last_undegraded":"2023-11-24T06:10:29.333924+0000","last_fullsized":"2023-11-24T06:10:29.333924+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:38.091507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208177+0000","last_change":"2023-11-24T06:10:28.203247+0000","last_active":"2023-11-24T06:10:28.208177+0000","last_peered":"2023-11-24T06:10:28.208177+0000","last_clean":"2023-11-24T06:10:28.208177+0000","last_became_active":"2023-11-24T06:10:28.202976+0000","last_became_peered":"2023-11-24T06:10:28.202976+0000","last_unstale":"2023-11-24T06:10:28.208177+0000","last_undegraded":"2023-11-24T06:10:28.208177+0000","last_fullsized":"2023-11-24T06:10:28.208177+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:06.200967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.190810+0000","last_change":"2023-11-24T06:10:27.194184+0000","last_active":"2023-11-24T06:10:28.190810+0000","last_peered":"2023-11-24T06:10:28.190810+0000","last_clean":"2023-11-24T06:10:28.190810+0000","last_became_active":"2023-11-24T06:10:27.193859+0000","last_became_peered":"2023-11-24T06:10:27.193859+0000","last_unstale":"2023-11-24T06:10:28.190810+0000","last_undegraded":"2023-11-24T06:10:28.190810+0000","last_fullsized":"2023-11-24T06:10:28.190810+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:32.577027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194886+0000","last_change":"2023-11-24T06:10:28.199326+0000","last_active":"2023-11-24T06:10:29.194886+0000","last_peered":"2023-11-24T06:10:29.194886+0000","last_clean":"2023-11-24T06:10:29.194886+0000","last_became_active":"2023-11-24T06:10:28.199006+0000","last_became_peered":"2023-11-24T06:10:28.199006+0000","last_unstale":"2023-11-24T06:10:29.194886+0000","last_undegraded":"2023-11-24T06:10:29.194886+0000","last_fullsized":"2023-11-24T06:10:29.194886+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:41.131179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.191996+0000","last_change":"2023-11-24T06:10:27.198421+0000","last_active":"2023-11-24T06:10:28.191996+0000","last_peered":"2023-11-24T06:10:28.191996+0000","last_clean":"2023-11-24T06:10:28.191996+0000","last_became_active":"2023-11-24T06:10:27.198163+0000","last_became_peered":"2023-11-24T06:10:27.198163+0000","last_unstale":"2023-11-24T06:10:28.191996+0000","last_undegraded":"2023-11-24T06:10:28.191996+0000","last_fullsized":"2023-11-24T06:10:28.191996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:00.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.210897+0000","last_change":"2023-11-24T06:10:28.206091+0000","last_active":"2023-11-24T06:10:28.210897+0000","last_peered":"2023-11-24T06:10:28.210897+0000","last_clean":"2023-11-24T06:10:28.210897+0000","last_became_active":"2023-11-24T06:10:28.205411+0000","last_became_peered":"2023-11-24T06:10:28.205411+0000","last_unstale":"2023-11-24T06:10:28.210897+0000","last_undegraded":"2023-11-24T06:10:28.210897+0000","last_fullsized":"2023-11-24T06:10:28.210897+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.196387+0000","last_change":"2023-11-24T06:10:27.200706+0000","last_active":"2023-11-24T06:10:28.196387+0000","last_peered":"2023-11-24T06:10:28.196387+0000","last_clean":"2023-11-24T06:10:28.196387+0000","last_became_active":"2023-11-24T06:10:27.200456+0000","last_became_peered":"2023-11-24T06:10:27.200456+0000","last_unstale":"2023-11-24T06:10:28.196387+0000","last_undegraded":"2023-11-24T06:10:28.196387+0000","last_fullsized":"2023-11-24T06:10:28.196387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:00.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211453+0000","last_change":"2023-11-24T06:10:28.210256+0000","last_active":"2023-11-24T06:10:28.211453+0000","last_peered":"2023-11-24T06:10:28.211453+0000","last_clean":"2023-11-24T06:10:28.211453+0000","last_became_active":"2023-11-24T06:10:28.209816+0000","last_became_peered":"2023-11-24T06:10:28.209816+0000","last_unstale":"2023-11-24T06:10:28.211453+0000","last_undegraded":"2023-11-24T06:10:28.211453+0000","last_fullsized":"2023-11-24T06:10:28.211453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:27.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194824+0000","last_change":"2023-11-24T06:10:27.188912+0000","last_active":"2023-11-24T06:10:29.194824+0000","last_peered":"2023-11-24T06:10:29.194824+0000","last_clean":"2023-11-24T06:10:29.194824+0000","last_became_active":"2023-11-24T06:10:27.188618+0000","last_became_peered":"2023-11-24T06:10:27.188618+0000","last_unstale":"2023-11-24T06:10:29.194824+0000","last_undegraded":"2023-11-24T06:10:29.194824+0000","last_fullsized":"2023-11-24T06:10:29.194824+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:21.696001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.188583+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:10:28.188583+0000","last_peered":"2023-11-24T06:10:28.188583+0000","last_clean":"2023-11-24T06:10:28.188583+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:10:28.188583+0000","last_undegraded":"2023-11-24T06:10:28.188583+0000","last_fullsized":"2023-11-24T06:10:28.188583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":799257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.91500000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.747}]},{"osd":2,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.622},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.876}]},{"osd":5,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.97899999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.96699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.502}]},{"osd":2,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":8.4849999999999994,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.75}]},{"osd":5,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":208869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26300000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.372}]},{"osd":4,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":799201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":12.378,"5min":12.378,"15min":12.378},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.5199999999999996,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.502}]},{"osd":4,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":15.319000000000001,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.7590000000000003,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":15.356,"5min":15.356,"15min":15.356},"last":0.877}]},{"osd":6,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.755},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.726,"5min":4.726,"15min":4.726},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":4.7869999999999999,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":15.388,"5min":15.388,"15min":15.388},"last":0.78500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":210433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.33400000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34999999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.374}]},{"osd":4,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.40899999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.443}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":209437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.254},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.30299999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":1.304}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":1.161},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":1.1120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":208799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60999999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.622}]},{"osd":2,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.44},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":208869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.335,"5min":3.335,"15min":3.335},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":4.4489999999999998,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.5069999999999997,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.60999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:10:32.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:10:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.740+0000 7f9015da7700 1 Processor -- start 2023-11-24T06:10:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.740+0000 7f9015da7700 1 -- start start 2023-11-24T06:10:32.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f9015da7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f901015e080 0x7f90100b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f9015da7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f901006c710 0x7f90100afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f9015da7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90100ae430 con 0x7f90100af4d0 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f9015da7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90100af0b0 con 0x7f901006c710 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f9015da7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90100b05a0 con 0x7f901015e080 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f901006c710 0x7f90100afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f901006c710 0x7f90100afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48034/0 (socket says 172.21.15.97:48034) 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900ffff700 1 -- 172.21.15.97:0/2827986246 learned_addr learned my addr 172.21.15.97:0/2827986246 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900ffff700 1 -- 172.21.15.97:0/2827986246 >> v2:172.21.15.183:3300/0 conn(0x7f901015e080 msgr2=0x7f90100b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900f7fe700 1 --2- 172.21.15.97:0/2827986246 >> v2:172.21.15.183:3300/0 conn(0x7f901015e080 0x7f90100b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900ffff700 1 --2- 172.21.15.97:0/2827986246 >> v2:172.21.15.183:3300/0 conn(0x7f901015e080 0x7f90100b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900ffff700 1 -- 172.21.15.97:0/2827986246 >> v1:172.21.15.183:6789/0 conn(0x7f90100af4d0 legacy=0x7f901006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900ffff700 1 -- 172.21.15.97:0/2827986246 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90100ae710 con 0x7f901006c710 2023-11-24T06:10:32.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.741+0000 7f900ffff700 1 --2- 172.21.15.97:0/2827986246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f90100afb90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9010072a90 tx=0x7f900800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed5674f3d92594 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f900e7fc700 1 -- 172.21.15.97:0/2827986246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900800e070 con 0x7f901006c710 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f900e7fc700 1 -- 172.21.15.97:0/2827986246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9008003a20 con 0x7f901006c710 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f900e7fc700 1 -- 172.21.15.97:0/2827986246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008003dd0 con 0x7f901006c710 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f9015da7700 1 -- 172.21.15.97:0/2827986246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 msgr2=0x7f90100afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f9015da7700 1 --2- 172.21.15.97:0/2827986246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f90100afb90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9010072a90 tx=0x7f900800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f9015da7700 1 -- 172.21.15.97:0/2827986246 shutdown_connections 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f9015da7700 1 --2- 172.21.15.97:0/2827986246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f90100afb90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f9015da7700 1 --2- 172.21.15.97:0/2827986246 >> v2:172.21.15.183:3300/0 conn(0x7f901015e080 0x7f90100b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f9015da7700 1 -- 172.21.15.97:0/2827986246 >> 172.21.15.97:0/2827986246 conn(0x7f9010071b90 msgr2=0x7f9010071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f9015da7700 1 -- 172.21.15.97:0/2827986246 shutdown_connections 2023-11-24T06:10:32.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.742+0000 7f9015da7700 1 -- 172.21.15.97:0/2827986246 wait complete. 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.743+0000 7f9015da7700 1 Processor -- start 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f9015da7700 1 -- start start 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f9015da7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f901014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f9015da7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90100af4d0 0x7f90101f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f9015da7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f901015e080 0x7f90100cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f9015da7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90100ce6d0 con 0x7f901006c710 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f9015da7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90100ce9b0 con 0x7f90100af4d0 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f9015da7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90100cec90 con 0x7f901015e080 2023-11-24T06:10:32.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f900f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f901014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f900f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f901014c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48036/0 (socket says 172.21.15.97:48036) 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.744+0000 7f900f7fe700 1 -- 172.21.15.97:0/2802759362 learned_addr learned my addr 172.21.15.97:0/2802759362 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900ffff700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f901015e080 0x7f90100cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900effd700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90100af4d0 0x7f90101f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900f7fe700 1 -- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f901015e080 msgr2=0x7f90100cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900f7fe700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f901015e080 0x7f90100cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900f7fe700 1 -- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90100af4d0 msgr2=0x7f90101f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900f7fe700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90100af4d0 0x7f90101f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900f7fe700 1 -- 172.21.15.97:0/2802759362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9008009b30 con 0x7f901006c710 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900effd700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90100af4d0 0x7f90101f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f900f7fe700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f901014c5d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c7f0 tx=0x7f8ff800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f9007fff700 1 -- 172.21.15.97:0/2802759362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8011800 con 0x7f901006c710 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f9007fff700 1 -- 172.21.15.97:0/2802759362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff8011960 con 0x7f901006c710 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f9007fff700 1 -- 172.21.15.97:0/2802759362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8011cd0 con 0x7f901006c710 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90100cef70 con 0x7f901006c710 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.745+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90101f3b30 con 0x7f901006c710 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.746+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90100753f0 con 0x7f901006c710 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.746+0000 7f9007fff700 1 -- 172.21.15.97:0/2802759362 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8ff8010350 con 0x7f901006c710 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.747+0000 7f9007fff700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ff006af60 0x7f8ff006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.747+0000 7f900effd700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ff006af60 0x7f8ff006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.747+0000 7f9007fff700 1 -- 172.21.15.97:0/2802759362 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ff8089230 con 0x7f901006c710 2023-11-24T06:10:32.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.747+0000 7f900effd700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ff006af60 0x7f8ff006d420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f90100c96a0 tx=0x7f9000009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:32.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.751+0000 7f9007fff700 1 -- 172.21.15.97:0/2802759362 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ff80541f0 con 0x7f901006c710 2023-11-24T06:10:32.941 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.939+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90101f3d50 con 0x7f8ff006af60 2023-11-24T06:10:32.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.946+0000 7f9007fff700 1 -- 172.21.15.97:0/2802759362 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289119 (secure 0 0 0) 0x7f90101f3d50 con 0x7f8ff006af60 2023-11-24T06:10:32.952 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.949+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ff006af60 msgr2=0x7f8ff006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.949+0000 7f9015da7700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ff006af60 0x7f8ff006d420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f90100c96a0 tx=0x7f9000009040 comp rx=0 tx=0).stop 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.949+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 msgr2=0x7f901014c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.949+0000 7f9015da7700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f901014c5d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c7f0 tx=0x7f8ff800cb00 comp rx=0 tx=0).stop 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.950+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 shutdown_connections 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.950+0000 7f9015da7700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8ff006af60 0x7f8ff006d420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.950+0000 7f9015da7700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f901015e080 0x7f90100cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.950+0000 7f9015da7700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f901006c710 0x7f901014c5d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.950+0000 7f9015da7700 1 --2- 172.21.15.97:0/2802759362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90100af4d0 0x7f90101f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:32.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.950+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 >> 172.21.15.97:0/2802759362 conn(0x7f9010071b90 msgr2=0x7f90100b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:32.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.950+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 shutdown_connections 2023-11-24T06:10:32.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:32.950+0000 7f9015da7700 1 -- 172.21.15.97:0/2802759362 wait complete. 2023-11-24T06:10:32.956 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:10:32.964 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-11-24T06:10:32.343960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221408,"kb_used_data":5244,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753312,"statfs":{"total":773094113280,"available":772867391488,"internally_reserved":0,"allocated":5369856,"data_stored":2854336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001353"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.334024+0000","last_change":"2023-11-24T06:10:28.193269+0000","last_active":"2023-11-24T06:10:29.334024+0000","last_peered":"2023-11-24T06:10:29.334024+0000","last_clean":"2023-11-24T06:10:29.334024+0000","last_became_active":"2023-11-24T06:10:28.192978+0000","last_became_peered":"2023-11-24T06:10:28.192978+0000","last_unstale":"2023-11-24T06:10:29.334024+0000","last_undegraded":"2023-11-24T06:10:29.334024+0000","last_fullsized":"2023-11-24T06:10:29.334024+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:28:17.045045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194702+0000","last_change":"2023-11-24T06:10:28.206982+0000","last_active":"2023-11-24T06:10:29.194702+0000","last_peered":"2023-11-24T06:10:29.194702+0000","last_clean":"2023-11-24T06:10:29.194702+0000","last_became_active":"2023-11-24T06:10:28.206538+0000","last_became_peered":"2023-11-24T06:10:28.206538+0000","last_unstale":"2023-11-24T06:10:29.194702+0000","last_undegraded":"2023-11-24T06:10:29.194702+0000","last_fullsized":"2023-11-24T06:10:29.194702+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:05.224113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.204814+0000","last_change":"2023-11-24T06:10:28.197956+0000","last_active":"2023-11-24T06:10:28.204814+0000","last_peered":"2023-11-24T06:10:28.204814+0000","last_clean":"2023-11-24T06:10:28.204814+0000","last_became_active":"2023-11-24T06:10:28.197675+0000","last_became_peered":"2023-11-24T06:10:28.197675+0000","last_unstale":"2023-11-24T06:10:28.204814+0000","last_undegraded":"2023-11-24T06:10:28.204814+0000","last_fullsized":"2023-11-24T06:10:28.204814+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:45.912268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211204+0000","last_change":"2023-11-24T06:10:28.210321+0000","last_active":"2023-11-24T06:10:28.211204+0000","last_peered":"2023-11-24T06:10:28.211204+0000","last_clean":"2023-11-24T06:10:28.211204+0000","last_became_active":"2023-11-24T06:10:28.210047+0000","last_became_peered":"2023-11-24T06:10:28.210047+0000","last_unstale":"2023-11-24T06:10:28.211204+0000","last_undegraded":"2023-11-24T06:10:28.211204+0000","last_fullsized":"2023-11-24T06:10:28.211204+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:34.656648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194637+0000","last_change":"2023-11-24T06:10:28.206019+0000","last_active":"2023-11-24T06:10:29.194637+0000","last_peered":"2023-11-24T06:10:29.194637+0000","last_clean":"2023-11-24T06:10:29.194637+0000","last_became_active":"2023-11-24T06:10:28.205684+0000","last_became_peered":"2023-11-24T06:10:28.205684+0000","last_unstale":"2023-11-24T06:10:29.194637+0000","last_undegraded":"2023-11-24T06:10:29.194637+0000","last_fullsized":"2023-11-24T06:10:29.194637+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:35.779994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207946+0000","last_change":"2023-11-24T06:10:28.203038+0000","last_active":"2023-11-24T06:10:28.207946+0000","last_peered":"2023-11-24T06:10:28.207946+0000","last_clean":"2023-11-24T06:10:28.207946+0000","last_became_active":"2023-11-24T06:10:28.202725+0000","last_became_peered":"2023-11-24T06:10:28.202725+0000","last_unstale":"2023-11-24T06:10:28.207946+0000","last_undegraded":"2023-11-24T06:10:28.207946+0000","last_fullsized":"2023-11-24T06:10:28.207946+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:35:42.437608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333747+0000","last_change":"2023-11-24T06:10:28.195333+0000","last_active":"2023-11-24T06:10:29.333747+0000","last_peered":"2023-11-24T06:10:29.333747+0000","last_clean":"2023-11-24T06:10:29.333747+0000","last_became_active":"2023-11-24T06:10:28.195029+0000","last_became_peered":"2023-11-24T06:10:28.195029+0000","last_unstale":"2023-11-24T06:10:29.333747+0000","last_undegraded":"2023-11-24T06:10:29.333747+0000","last_fullsized":"2023-11-24T06:10:29.333747+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:52:18.285812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211818+0000","last_change":"2023-11-24T06:10:28.210971+0000","last_active":"2023-11-24T06:10:28.211818+0000","last_peered":"2023-11-24T06:10:28.211818+0000","last_clean":"2023-11-24T06:10:28.211818+0000","last_became_active":"2023-11-24T06:10:28.210658+0000","last_became_peered":"2023-11-24T06:10:28.210658+0000","last_unstale":"2023-11-24T06:10:28.211818+0000","last_undegraded":"2023-11-24T06:10:28.211818+0000","last_fullsized":"2023-11-24T06:10:28.211818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:45.912268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209490+0000","last_change":"2023-11-24T06:10:28.207114+0000","last_active":"2023-11-24T06:10:28.209490+0000","last_peered":"2023-11-24T06:10:28.209490+0000","last_clean":"2023-11-24T06:10:28.209490+0000","last_became_active":"2023-11-24T06:10:28.206833+0000","last_became_peered":"2023-11-24T06:10:28.206833+0000","last_unstale":"2023-11-24T06:10:28.209490+0000","last_undegraded":"2023-11-24T06:10:28.209490+0000","last_fullsized":"2023-11-24T06:10:28.209490+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:23.018551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211491+0000","last_change":"2023-11-24T06:10:28.210737+0000","last_active":"2023-11-24T06:10:28.211491+0000","last_peered":"2023-11-24T06:10:28.211491+0000","last_clean":"2023-11-24T06:10:28.211491+0000","last_became_active":"2023-11-24T06:10:28.209841+0000","last_became_peered":"2023-11-24T06:10:28.209841+0000","last_unstale":"2023-11-24T06:10:28.211491+0000","last_undegraded":"2023-11-24T06:10:28.211491+0000","last_fullsized":"2023-11-24T06:10:28.211491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:05.905455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209449+0000","last_change":"2023-11-24T06:10:28.206774+0000","last_active":"2023-11-24T06:10:28.209449+0000","last_peered":"2023-11-24T06:10:28.209449+0000","last_clean":"2023-11-24T06:10:28.209449+0000","last_became_active":"2023-11-24T06:10:28.206490+0000","last_became_peered":"2023-11-24T06:10:28.206490+0000","last_unstale":"2023-11-24T06:10:28.209449+0000","last_undegraded":"2023-11-24T06:10:28.209449+0000","last_fullsized":"2023-11-24T06:10:28.209449+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:21.501785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.195231+0000","last_change":"2023-11-24T06:10:28.205517+0000","last_active":"2023-11-24T06:10:29.195231+0000","last_peered":"2023-11-24T06:10:29.195231+0000","last_clean":"2023-11-24T06:10:29.195231+0000","last_became_active":"2023-11-24T06:10:28.204466+0000","last_became_peered":"2023-11-24T06:10:28.204466+0000","last_unstale":"2023-11-24T06:10:29.195231+0000","last_undegraded":"2023-11-24T06:10:29.195231+0000","last_fullsized":"2023-11-24T06:10:29.195231+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:24.055216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190459+0000","last_change":"2023-11-24T06:10:28.201928+0000","last_active":"2023-11-24T06:10:29.190459+0000","last_peered":"2023-11-24T06:10:29.190459+0000","last_clean":"2023-11-24T06:10:29.190459+0000","last_became_active":"2023-11-24T06:10:28.201654+0000","last_became_peered":"2023-11-24T06:10:28.201654+0000","last_unstale":"2023-11-24T06:10:29.190459+0000","last_undegraded":"2023-11-24T06:10:29.190459+0000","last_fullsized":"2023-11-24T06:10:29.190459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209518+0000","last_change":"2023-11-24T06:10:28.207309+0000","last_active":"2023-11-24T06:10:28.209518+0000","last_peered":"2023-11-24T06:10:28.209518+0000","last_clean":"2023-11-24T06:10:28.209518+0000","last_became_active":"2023-11-24T06:10:28.207049+0000","last_became_peered":"2023-11-24T06:10:28.207049+0000","last_unstale":"2023-11-24T06:10:28.209518+0000","last_undegraded":"2023-11-24T06:10:28.209518+0000","last_fullsized":"2023-11-24T06:10:28.209518+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.484739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208562+0000","last_change":"2023-11-24T06:10:28.204319+0000","last_active":"2023-11-24T06:10:28.208562+0000","last_peered":"2023-11-24T06:10:28.208562+0000","last_clean":"2023-11-24T06:10:28.208562+0000","last_became_active":"2023-11-24T06:10:28.203844+0000","last_became_peered":"2023-11-24T06:10:28.203844+0000","last_unstale":"2023-11-24T06:10:28.208562+0000","last_undegraded":"2023-11-24T06:10:28.208562+0000","last_fullsized":"2023-11-24T06:10:28.208562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:32.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194147+0000","last_change":"2023-11-24T06:10:28.207422+0000","last_active":"2023-11-24T06:10:29.194147+0000","last_peered":"2023-11-24T06:10:29.194147+0000","last_clean":"2023-11-24T06:10:29.194147+0000","last_became_active":"2023-11-24T06:10:28.206984+0000","last_became_peered":"2023-11-24T06:10:28.206984+0000","last_unstale":"2023-11-24T06:10:29.194147+0000","last_undegraded":"2023-11-24T06:10:29.194147+0000","last_fullsized":"2023-11-24T06:10:29.194147+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:21.501785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.205016+0000","last_change":"2023-11-24T06:10:28.200334+0000","last_active":"2023-11-24T06:10:28.205016+0000","last_peered":"2023-11-24T06:10:28.205016+0000","last_clean":"2023-11-24T06:10:28.205016+0000","last_became_active":"2023-11-24T06:10:28.200053+0000","last_became_peered":"2023-11-24T06:10:28.200053+0000","last_unstale":"2023-11-24T06:10:28.205016+0000","last_undegraded":"2023-11-24T06:10:28.205016+0000","last_fullsized":"2023-11-24T06:10:28.205016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:21.501785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.206559+0000","last_change":"2023-11-24T06:10:28.203264+0000","last_active":"2023-11-24T06:10:28.206559+0000","last_peered":"2023-11-24T06:10:28.206559+0000","last_clean":"2023-11-24T06:10:28.206559+0000","last_became_active":"2023-11-24T06:10:28.202861+0000","last_became_peered":"2023-11-24T06:10:28.202861+0000","last_unstale":"2023-11-24T06:10:28.206559+0000","last_undegraded":"2023-11-24T06:10:28.206559+0000","last_fullsized":"2023-11-24T06:10:28.206559+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:34.656648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194894+0000","last_change":"2023-11-24T06:10:28.207425+0000","last_active":"2023-11-24T06:10:29.194894+0000","last_peered":"2023-11-24T06:10:29.194894+0000","last_clean":"2023-11-24T06:10:29.194894+0000","last_became_active":"2023-11-24T06:10:28.206960+0000","last_became_peered":"2023-11-24T06:10:28.206960+0000","last_unstale":"2023-11-24T06:10:29.194894+0000","last_undegraded":"2023-11-24T06:10:29.194894+0000","last_fullsized":"2023-11-24T06:10:29.194894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:33.582325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190321+0000","last_change":"2023-11-24T06:10:28.203199+0000","last_active":"2023-11-24T06:10:29.190321+0000","last_peered":"2023-11-24T06:10:29.190321+0000","last_clean":"2023-11-24T06:10:29.190321+0000","last_became_active":"2023-11-24T06:10:28.202796+0000","last_became_peered":"2023-11-24T06:10:28.202796+0000","last_unstale":"2023-11-24T06:10:29.190321+0000","last_undegraded":"2023-11-24T06:10:29.190321+0000","last_fullsized":"2023-11-24T06:10:29.190321+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:51:47.343086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209081+0000","last_change":"2023-11-24T06:10:28.207940+0000","last_active":"2023-11-24T06:10:28.209081+0000","last_peered":"2023-11-24T06:10:28.209081+0000","last_clean":"2023-11-24T06:10:28.209081+0000","last_became_active":"2023-11-24T06:10:28.207513+0000","last_became_peered":"2023-11-24T06:10:28.207513+0000","last_unstale":"2023-11-24T06:10:28.209081+0000","last_undegraded":"2023-11-24T06:10:28.209081+0000","last_fullsized":"2023-11-24T06:10:28.209081+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:56.876021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211831+0000","last_change":"2023-11-24T06:10:28.210968+0000","last_active":"2023-11-24T06:10:28.211831+0000","last_peered":"2023-11-24T06:10:28.211831+0000","last_clean":"2023-11-24T06:10:28.211831+0000","last_became_active":"2023-11-24T06:10:28.210603+0000","last_became_peered":"2023-11-24T06:10:28.210603+0000","last_unstale":"2023-11-24T06:10:28.211831+0000","last_undegraded":"2023-11-24T06:10:28.211831+0000","last_fullsized":"2023-11-24T06:10:28.211831+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:48.458766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194239+0000","last_change":"2023-11-24T06:10:28.207716+0000","last_active":"2023-11-24T06:10:29.194239+0000","last_peered":"2023-11-24T06:10:29.194239+0000","last_clean":"2023-11-24T06:10:29.194239+0000","last_became_active":"2023-11-24T06:10:28.207313+0000","last_became_peered":"2023-11-24T06:10:28.207313+0000","last_unstale":"2023-11-24T06:10:29.194239+0000","last_undegraded":"2023-11-24T06:10:29.194239+0000","last_fullsized":"2023-11-24T06:10:29.194239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:34.656648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194782+0000","last_change":"2023-11-24T06:10:28.207778+0000","last_active":"2023-11-24T06:10:29.194782+0000","last_peered":"2023-11-24T06:10:29.194782+0000","last_clean":"2023-11-24T06:10:29.194782+0000","last_became_active":"2023-11-24T06:10:28.207431+0000","last_became_peered":"2023-11-24T06:10:28.207431+0000","last_unstale":"2023-11-24T06:10:29.194782+0000","last_undegraded":"2023-11-24T06:10:29.194782+0000","last_fullsized":"2023-11-24T06:10:29.194782+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:23.018551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194777+0000","last_change":"2023-11-24T06:10:28.203748+0000","last_active":"2023-11-24T06:10:29.194777+0000","last_peered":"2023-11-24T06:10:29.194777+0000","last_clean":"2023-11-24T06:10:29.194777+0000","last_became_active":"2023-11-24T06:10:28.202890+0000","last_became_peered":"2023-11-24T06:10:28.202890+0000","last_unstale":"2023-11-24T06:10:29.194777+0000","last_undegraded":"2023-11-24T06:10:29.194777+0000","last_fullsized":"2023-11-24T06:10:29.194777+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:01.465488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190158+0000","last_change":"2023-11-24T06:10:28.206268+0000","last_active":"2023-11-24T06:10:29.190158+0000","last_peered":"2023-11-24T06:10:29.190158+0000","last_clean":"2023-11-24T06:10:29.190158+0000","last_became_active":"2023-11-24T06:10:28.205999+0000","last_became_peered":"2023-11-24T06:10:28.205999+0000","last_unstale":"2023-11-24T06:10:29.190158+0000","last_undegraded":"2023-11-24T06:10:29.190158+0000","last_fullsized":"2023-11-24T06:10:29.190158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:33.863577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.204835+0000","last_change":"2023-11-24T06:10:28.198939+0000","last_active":"2023-11-24T06:10:28.204835+0000","last_peered":"2023-11-24T06:10:28.204835+0000","last_clean":"2023-11-24T06:10:28.204835+0000","last_became_active":"2023-11-24T06:10:28.198631+0000","last_became_peered":"2023-11-24T06:10:28.198631+0000","last_unstale":"2023-11-24T06:10:28.204835+0000","last_undegraded":"2023-11-24T06:10:28.204835+0000","last_fullsized":"2023-11-24T06:10:28.204835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:34:57.484739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194928+0000","last_change":"2023-11-24T06:10:28.205792+0000","last_active":"2023-11-24T06:10:29.194928+0000","last_peered":"2023-11-24T06:10:29.194928+0000","last_clean":"2023-11-24T06:10:29.194928+0000","last_became_active":"2023-11-24T06:10:28.204872+0000","last_became_peered":"2023-11-24T06:10:28.204872+0000","last_unstale":"2023-11-24T06:10:29.194928+0000","last_undegraded":"2023-11-24T06:10:29.194928+0000","last_fullsized":"2023-11-24T06:10:29.194928+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:06.200967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194410+0000","last_change":"2023-11-24T06:10:28.202864+0000","last_active":"2023-11-24T06:10:29.194410+0000","last_peered":"2023-11-24T06:10:29.194410+0000","last_clean":"2023-11-24T06:10:29.194410+0000","last_became_active":"2023-11-24T06:10:28.202559+0000","last_became_peered":"2023-11-24T06:10:28.202559+0000","last_unstale":"2023-11-24T06:10:29.194410+0000","last_undegraded":"2023-11-24T06:10:29.194410+0000","last_fullsized":"2023-11-24T06:10:29.194410+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:30.721899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.202100+0000","last_change":"2023-11-24T06:10:28.195813+0000","last_active":"2023-11-24T06:10:28.202100+0000","last_peered":"2023-11-24T06:10:28.202100+0000","last_clean":"2023-11-24T06:10:28.202100+0000","last_became_active":"2023-11-24T06:10:28.195554+0000","last_became_peered":"2023-11-24T06:10:28.195554+0000","last_unstale":"2023-11-24T06:10:28.202100+0000","last_undegraded":"2023-11-24T06:10:28.202100+0000","last_fullsized":"2023-11-24T06:10:28.202100+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:05.162020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208802+0000","last_change":"2023-11-24T06:10:28.204704+0000","last_active":"2023-11-24T06:10:28.208802+0000","last_peered":"2023-11-24T06:10:28.208802+0000","last_clean":"2023-11-24T06:10:28.208802+0000","last_became_active":"2023-11-24T06:10:28.204348+0000","last_became_peered":"2023-11-24T06:10:28.204348+0000","last_unstale":"2023-11-24T06:10:28.208802+0000","last_undegraded":"2023-11-24T06:10:28.208802+0000","last_fullsized":"2023-11-24T06:10:28.208802+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:28:17.045045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190124+0000","last_change":"2023-11-24T06:10:28.198505+0000","last_active":"2023-11-24T06:10:29.190124+0000","last_peered":"2023-11-24T06:10:29.190124+0000","last_clean":"2023-11-24T06:10:29.190124+0000","last_became_active":"2023-11-24T06:10:28.198240+0000","last_became_peered":"2023-11-24T06:10:28.198240+0000","last_unstale":"2023-11-24T06:10:29.190124+0000","last_undegraded":"2023-11-24T06:10:29.190124+0000","last_fullsized":"2023-11-24T06:10:29.190124+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:33:39.258906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.206894+0000","last_change":"2023-11-24T06:10:28.202058+0000","last_active":"2023-11-24T06:10:28.206894+0000","last_peered":"2023-11-24T06:10:28.206894+0000","last_clean":"2023-11-24T06:10:28.206894+0000","last_became_active":"2023-11-24T06:10:28.201727+0000","last_became_peered":"2023-11-24T06:10:28.201727+0000","last_unstale":"2023-11-24T06:10:28.206894+0000","last_undegraded":"2023-11-24T06:10:28.206894+0000","last_fullsized":"2023-11-24T06:10:28.206894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:28.409249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511596+0000","last_change":"2023-11-24T06:10:27.191631+0000","last_active":"2023-11-24T06:10:28.511596+0000","last_peered":"2023-11-24T06:10:28.511596+0000","last_clean":"2023-11-24T06:10:28.511596+0000","last_became_active":"2023-11-24T06:10:27.191257+0000","last_became_peered":"2023-11-24T06:10:27.191257+0000","last_unstale":"2023-11-24T06:10:28.511596+0000","last_undegraded":"2023-11-24T06:10:28.511596+0000","last_fullsized":"2023-11-24T06:10:28.511596+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:00.577272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194390+0000","last_change":"2023-11-24T06:10:28.203523+0000","last_active":"2023-11-24T06:10:29.194390+0000","last_peered":"2023-11-24T06:10:29.194390+0000","last_clean":"2023-11-24T06:10:29.194390+0000","last_became_active":"2023-11-24T06:10:28.203257+0000","last_became_peered":"2023-11-24T06:10:28.203257+0000","last_unstale":"2023-11-24T06:10:29.194390+0000","last_undegraded":"2023-11-24T06:10:29.194390+0000","last_fullsized":"2023-11-24T06:10:29.194390+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:05.162020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511499+0000","last_change":"2023-11-24T06:10:27.193130+0000","last_active":"2023-11-24T06:10:28.511499+0000","last_peered":"2023-11-24T06:10:28.511499+0000","last_clean":"2023-11-24T06:10:28.511499+0000","last_became_active":"2023-11-24T06:10:27.192856+0000","last_became_peered":"2023-11-24T06:10:27.192856+0000","last_unstale":"2023-11-24T06:10:28.511499+0000","last_undegraded":"2023-11-24T06:10:28.511499+0000","last_fullsized":"2023-11-24T06:10:28.511499+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:00.460189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.195060+0000","last_change":"2023-11-24T06:10:28.206057+0000","last_active":"2023-11-24T06:10:29.195060+0000","last_peered":"2023-11-24T06:10:29.195060+0000","last_clean":"2023-11-24T06:10:29.195060+0000","last_became_active":"2023-11-24T06:10:28.205746+0000","last_became_peered":"2023-11-24T06:10:28.205746+0000","last_unstale":"2023-11-24T06:10:29.195060+0000","last_undegraded":"2023-11-24T06:10:29.195060+0000","last_fullsized":"2023-11-24T06:10:29.195060+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:35:42.437608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.512122+0000","last_change":"2023-11-24T06:10:27.200130+0000","last_active":"2023-11-24T06:10:28.512122+0000","last_peered":"2023-11-24T06:10:28.512122+0000","last_clean":"2023-11-24T06:10:28.512122+0000","last_became_active":"2023-11-24T06:10:27.199881+0000","last_became_peered":"2023-11-24T06:10:27.199881+0000","last_unstale":"2023-11-24T06:10:28.512122+0000","last_undegraded":"2023-11-24T06:10:28.512122+0000","last_fullsized":"2023-11-24T06:10:28.512122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:22.013253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.209683+0000","last_change":"2023-11-24T06:10:28.208662+0000","last_active":"2023-11-24T06:10:28.209683+0000","last_peered":"2023-11-24T06:10:28.209683+0000","last_clean":"2023-11-24T06:10:28.209683+0000","last_became_active":"2023-11-24T06:10:28.208386+0000","last_became_peered":"2023-11-24T06:10:28.208386+0000","last_unstale":"2023-11-24T06:10:28.209683+0000","last_undegraded":"2023-11-24T06:10:28.209683+0000","last_fullsized":"2023-11-24T06:10:28.209683+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:05.162020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333799+0000","last_change":"2023-11-24T06:10:27.197154+0000","last_active":"2023-11-24T06:10:29.333799+0000","last_peered":"2023-11-24T06:10:29.333799+0000","last_clean":"2023-11-24T06:10:29.333799+0000","last_became_active":"2023-11-24T06:10:27.196910+0000","last_became_peered":"2023-11-24T06:10:27.196910+0000","last_unstale":"2023-11-24T06:10:29.333799+0000","last_undegraded":"2023-11-24T06:10:29.333799+0000","last_fullsized":"2023-11-24T06:10:29.333799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:30.476022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.204899+0000","last_change":"2023-11-24T06:10:28.199626+0000","last_active":"2023-11-24T06:10:28.204899+0000","last_peered":"2023-11-24T06:10:28.204899+0000","last_clean":"2023-11-24T06:10:28.204899+0000","last_became_active":"2023-11-24T06:10:28.199351+0000","last_became_peered":"2023-11-24T06:10:28.199351+0000","last_unstale":"2023-11-24T06:10:28.204899+0000","last_undegraded":"2023-11-24T06:10:28.204899+0000","last_fullsized":"2023-11-24T06:10:28.204899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:28.409249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511312+0000","last_change":"2023-11-24T06:10:27.196884+0000","last_active":"2023-11-24T06:10:28.511312+0000","last_peered":"2023-11-24T06:10:28.511312+0000","last_clean":"2023-11-24T06:10:28.511312+0000","last_became_active":"2023-11-24T06:10:27.196623+0000","last_became_peered":"2023-11-24T06:10:27.196623+0000","last_unstale":"2023-11-24T06:10:28.511312+0000","last_undegraded":"2023-11-24T06:10:28.511312+0000","last_fullsized":"2023-11-24T06:10:28.511312+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:33:37.025817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.205697+0000","last_change":"2023-11-24T06:10:28.204060+0000","last_active":"2023-11-24T06:10:28.205697+0000","last_peered":"2023-11-24T06:10:28.205697+0000","last_clean":"2023-11-24T06:10:28.205697+0000","last_became_active":"2023-11-24T06:10:28.203777+0000","last_became_peered":"2023-11-24T06:10:28.203777+0000","last_unstale":"2023-11-24T06:10:28.205697+0000","last_undegraded":"2023-11-24T06:10:28.205697+0000","last_fullsized":"2023-11-24T06:10:28.205697+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:51:47.343086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.512067+0000","last_change":"2023-11-24T06:10:27.189950+0000","last_active":"2023-11-24T06:10:28.512067+0000","last_peered":"2023-11-24T06:10:28.512067+0000","last_clean":"2023-11-24T06:10:28.512067+0000","last_became_active":"2023-11-24T06:10:27.189668+0000","last_became_peered":"2023-11-24T06:10:27.189668+0000","last_unstale":"2023-11-24T06:10:28.512067+0000","last_undegraded":"2023-11-24T06:10:28.512067+0000","last_fullsized":"2023-11-24T06:10:28.512067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:00.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211440+0000","last_change":"2023-11-24T06:10:28.210641+0000","last_active":"2023-11-24T06:10:28.211440+0000","last_peered":"2023-11-24T06:10:28.211440+0000","last_clean":"2023-11-24T06:10:28.211440+0000","last_became_active":"2023-11-24T06:10:28.210094+0000","last_became_peered":"2023-11-24T06:10:28.210094+0000","last_unstale":"2023-11-24T06:10:28.211440+0000","last_undegraded":"2023-11-24T06:10:28.211440+0000","last_fullsized":"2023-11-24T06:10:28.211440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:33.863577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194367+0000","last_change":"2023-11-24T06:10:27.197066+0000","last_active":"2023-11-24T06:10:29.194367+0000","last_peered":"2023-11-24T06:10:29.194367+0000","last_clean":"2023-11-24T06:10:29.194367+0000","last_became_active":"2023-11-24T06:10:27.196732+0000","last_became_peered":"2023-11-24T06:10:27.196732+0000","last_unstale":"2023-11-24T06:10:29.194367+0000","last_undegraded":"2023-11-24T06:10:29.194367+0000","last_fullsized":"2023-11-24T06:10:29.194367+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:32.577027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.195344+0000","last_change":"2023-11-24T06:10:28.203495+0000","last_active":"2023-11-24T06:10:29.195344+0000","last_peered":"2023-11-24T06:10:29.195344+0000","last_clean":"2023-11-24T06:10:29.195344+0000","last_became_active":"2023-11-24T06:10:28.203203+0000","last_became_peered":"2023-11-24T06:10:28.203203+0000","last_unstale":"2023-11-24T06:10:29.195344+0000","last_undegraded":"2023-11-24T06:10:29.195344+0000","last_fullsized":"2023-11-24T06:10:29.195344+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:05.905455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.196525+0000","last_change":"2023-11-24T06:10:27.203319+0000","last_active":"2023-11-24T06:10:28.196525+0000","last_peered":"2023-11-24T06:10:28.196525+0000","last_clean":"2023-11-24T06:10:28.196525+0000","last_became_active":"2023-11-24T06:10:27.203031+0000","last_became_peered":"2023-11-24T06:10:27.203031+0000","last_unstale":"2023-11-24T06:10:28.196525+0000","last_undegraded":"2023-11-24T06:10:28.196525+0000","last_fullsized":"2023-11-24T06:10:28.196525+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:32.577027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.206535+0000","last_change":"2023-11-24T06:10:28.202974+0000","last_active":"2023-11-24T06:10:28.206535+0000","last_peered":"2023-11-24T06:10:28.206535+0000","last_clean":"2023-11-24T06:10:28.206535+0000","last_became_active":"2023-11-24T06:10:28.202707+0000","last_became_peered":"2023-11-24T06:10:28.202707+0000","last_unstale":"2023-11-24T06:10:28.206535+0000","last_undegraded":"2023-11-24T06:10:28.206535+0000","last_fullsized":"2023-11-24T06:10:28.206535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:48.458766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.188865+0000","last_change":"2023-11-24T06:10:27.188549+0000","last_active":"2023-11-24T06:10:28.188865+0000","last_peered":"2023-11-24T06:10:28.188865+0000","last_clean":"2023-11-24T06:10:28.188865+0000","last_became_active":"2023-11-24T06:10:27.188101+0000","last_became_peered":"2023-11-24T06:10:27.188101+0000","last_unstale":"2023-11-24T06:10:28.188865+0000","last_undegraded":"2023-11-24T06:10:28.188865+0000","last_fullsized":"2023-11-24T06:10:28.188865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:29.716601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.334063+0000","last_change":"2023-11-24T06:10:28.200336+0000","last_active":"2023-11-24T06:10:29.334063+0000","last_peered":"2023-11-24T06:10:29.334063+0000","last_clean":"2023-11-24T06:10:29.334063+0000","last_became_active":"2023-11-24T06:10:28.200050+0000","last_became_peered":"2023-11-24T06:10:28.200050+0000","last_unstale":"2023-11-24T06:10:29.334063+0000","last_undegraded":"2023-11-24T06:10:29.334063+0000","last_fullsized":"2023-11-24T06:10:29.334063+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:43.703372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.202693+0000","last_change":"2023-11-24T06:10:27.201868+0000","last_active":"2023-11-24T06:10:28.202693+0000","last_peered":"2023-11-24T06:10:28.202693+0000","last_clean":"2023-11-24T06:10:28.202693+0000","last_became_active":"2023-11-24T06:10:27.201425+0000","last_became_peered":"2023-11-24T06:10:27.201425+0000","last_unstale":"2023-11-24T06:10:28.202693+0000","last_undegraded":"2023-11-24T06:10:28.202693+0000","last_fullsized":"2023-11-24T06:10:28.202693+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:45:03.189139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207863+0000","last_change":"2023-11-24T06:10:28.202383+0000","last_active":"2023-11-24T06:10:28.207863+0000","last_peered":"2023-11-24T06:10:28.207863+0000","last_clean":"2023-11-24T06:10:28.207863+0000","last_became_active":"2023-11-24T06:10:28.202108+0000","last_became_peered":"2023-11-24T06:10:28.202108+0000","last_unstale":"2023-11-24T06:10:28.207863+0000","last_undegraded":"2023-11-24T06:10:28.207863+0000","last_fullsized":"2023-11-24T06:10:28.207863+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:52:18.285812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333976+0000","last_change":"2023-11-24T06:10:27.200059+0000","last_active":"2023-11-24T06:10:29.333976+0000","last_peered":"2023-11-24T06:10:29.333976+0000","last_clean":"2023-11-24T06:10:29.333976+0000","last_became_active":"2023-11-24T06:10:27.199806+0000","last_became_peered":"2023-11-24T06:10:27.199806+0000","last_unstale":"2023-11-24T06:10:29.333976+0000","last_undegraded":"2023-11-24T06:10:29.333976+0000","last_fullsized":"2023-11-24T06:10:29.333976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:04.218814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333866+0000","last_change":"2023-11-24T06:10:28.196009+0000","last_active":"2023-11-24T06:10:29.333866+0000","last_peered":"2023-11-24T06:10:29.333866+0000","last_clean":"2023-11-24T06:10:29.333866+0000","last_became_active":"2023-11-24T06:10:28.195670+0000","last_became_peered":"2023-11-24T06:10:28.195670+0000","last_unstale":"2023-11-24T06:10:29.333866+0000","last_undegraded":"2023-11-24T06:10:29.333866+0000","last_fullsized":"2023-11-24T06:10:29.333866+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:56.876021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190334+0000","last_change":"2023-11-24T06:10:27.190251+0000","last_active":"2023-11-24T06:10:29.190334+0000","last_peered":"2023-11-24T06:10:29.190334+0000","last_clean":"2023-11-24T06:10:29.190334+0000","last_became_active":"2023-11-24T06:10:27.190065+0000","last_became_peered":"2023-11-24T06:10:27.190065+0000","last_unstale":"2023-11-24T06:10:29.190334+0000","last_undegraded":"2023-11-24T06:10:29.190334+0000","last_fullsized":"2023-11-24T06:10:29.190334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:01.071112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194285+0000","last_change":"2023-11-24T06:10:28.204100+0000","last_active":"2023-11-24T06:10:29.194285+0000","last_peered":"2023-11-24T06:10:29.194285+0000","last_clean":"2023-11-24T06:10:29.194285+0000","last_became_active":"2023-11-24T06:10:28.203867+0000","last_became_peered":"2023-11-24T06:10:28.203867+0000","last_unstale":"2023-11-24T06:10:29.194285+0000","last_undegraded":"2023-11-24T06:10:29.194285+0000","last_fullsized":"2023-11-24T06:10:29.194285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:45.912268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511098+0000","last_change":"2023-11-24T06:10:27.194385+0000","last_active":"2023-11-24T06:10:28.511098+0000","last_peered":"2023-11-24T06:10:28.511098+0000","last_clean":"2023-11-24T06:10:28.511098+0000","last_became_active":"2023-11-24T06:10:27.194099+0000","last_became_peered":"2023-11-24T06:10:27.194099+0000","last_unstale":"2023-11-24T06:10:28.511098+0000","last_undegraded":"2023-11-24T06:10:28.511098+0000","last_fullsized":"2023-11-24T06:10:28.511098+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:05.088649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.205717+0000","last_change":"2023-11-24T06:10:28.204380+0000","last_active":"2023-11-24T06:10:28.205717+0000","last_peered":"2023-11-24T06:10:28.205717+0000","last_clean":"2023-11-24T06:10:28.205717+0000","last_became_active":"2023-11-24T06:10:28.204113+0000","last_became_peered":"2023-11-24T06:10:28.204113+0000","last_unstale":"2023-11-24T06:10:28.205717+0000","last_undegraded":"2023-11-24T06:10:28.205717+0000","last_fullsized":"2023-11-24T06:10:28.205717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333780+0000","last_change":"2023-11-24T06:10:27.199964+0000","last_active":"2023-11-24T06:10:29.333780+0000","last_peered":"2023-11-24T06:10:29.333780+0000","last_clean":"2023-11-24T06:10:29.333780+0000","last_became_active":"2023-11-24T06:10:27.199684+0000","last_became_peered":"2023-11-24T06:10:27.199684+0000","last_unstale":"2023-11-24T06:10:29.333780+0000","last_undegraded":"2023-11-24T06:10:29.333780+0000","last_fullsized":"2023-11-24T06:10:29.333780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:32.577027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207896+0000","last_change":"2023-11-24T06:10:28.202239+0000","last_active":"2023-11-24T06:10:28.207896+0000","last_peered":"2023-11-24T06:10:28.207896+0000","last_clean":"2023-11-24T06:10:28.207896+0000","last_became_active":"2023-11-24T06:10:28.201859+0000","last_became_peered":"2023-11-24T06:10:28.201859+0000","last_unstale":"2023-11-24T06:10:28.207896+0000","last_undegraded":"2023-11-24T06:10:28.207896+0000","last_fullsized":"2023-11-24T06:10:28.207896+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:41.131179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.197377+0000","last_change":"2023-11-24T06:10:27.194124+0000","last_active":"2023-11-24T06:10:28.197377+0000","last_peered":"2023-11-24T06:10:28.197377+0000","last_clean":"2023-11-24T06:10:28.197377+0000","last_became_active":"2023-11-24T06:10:27.193669+0000","last_became_peered":"2023-11-24T06:10:27.193669+0000","last_unstale":"2023-11-24T06:10:28.197377+0000","last_undegraded":"2023-11-24T06:10:28.197377+0000","last_fullsized":"2023-11-24T06:10:28.197377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:52:46.350444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333860+0000","last_change":"2023-11-24T06:10:28.208826+0000","last_active":"2023-11-24T06:10:29.333860+0000","last_peered":"2023-11-24T06:10:29.333860+0000","last_clean":"2023-11-24T06:10:29.333860+0000","last_became_active":"2023-11-24T06:10:28.208404+0000","last_became_peered":"2023-11-24T06:10:28.208404+0000","last_unstale":"2023-11-24T06:10:29.333860+0000","last_undegraded":"2023-11-24T06:10:29.333860+0000","last_fullsized":"2023-11-24T06:10:29.333860+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:53:30.744751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.512865+0000","last_change":"2023-11-24T06:10:27.198528+0000","last_active":"2023-11-24T06:10:28.512865+0000","last_peered":"2023-11-24T06:10:28.512865+0000","last_clean":"2023-11-24T06:10:28.512865+0000","last_became_active":"2023-11-24T06:10:27.198282+0000","last_became_peered":"2023-11-24T06:10:27.198282+0000","last_unstale":"2023-11-24T06:10:28.512865+0000","last_undegraded":"2023-11-24T06:10:28.512865+0000","last_fullsized":"2023-11-24T06:10:28.512865+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:29.716601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190264+0000","last_change":"2023-11-24T06:10:28.200216+0000","last_active":"2023-11-24T06:10:29.190264+0000","last_peered":"2023-11-24T06:10:29.190264+0000","last_clean":"2023-11-24T06:10:29.190264+0000","last_became_active":"2023-11-24T06:10:28.199923+0000","last_became_peered":"2023-11-24T06:10:28.199923+0000","last_unstale":"2023-11-24T06:10:29.190264+0000","last_undegraded":"2023-11-24T06:10:29.190264+0000","last_fullsized":"2023-11-24T06:10:29.190264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:41.703490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190331+0000","last_change":"2023-11-24T06:10:27.199825+0000","last_active":"2023-11-24T06:10:29.190331+0000","last_peered":"2023-11-24T06:10:29.190331+0000","last_clean":"2023-11-24T06:10:29.190331+0000","last_became_active":"2023-11-24T06:10:27.199551+0000","last_became_peered":"2023-11-24T06:10:27.199551+0000","last_unstale":"2023-11-24T06:10:29.190331+0000","last_undegraded":"2023-11-24T06:10:29.190331+0000","last_fullsized":"2023-11-24T06:10:29.190331+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:03:03.382239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207327+0000","last_change":"2023-11-24T06:10:28.205842+0000","last_active":"2023-11-24T06:10:28.207327+0000","last_peered":"2023-11-24T06:10:28.207327+0000","last_clean":"2023-11-24T06:10:28.207327+0000","last_became_active":"2023-11-24T06:10:28.205532+0000","last_became_peered":"2023-11-24T06:10:28.205532+0000","last_unstale":"2023-11-24T06:10:28.207327+0000","last_undegraded":"2023-11-24T06:10:28.207327+0000","last_fullsized":"2023-11-24T06:10:28.207327+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:41.703490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.510986+0000","last_change":"2023-11-24T06:10:27.198614+0000","last_active":"2023-11-24T06:10:28.510986+0000","last_peered":"2023-11-24T06:10:28.510986+0000","last_clean":"2023-11-24T06:10:28.510986+0000","last_became_active":"2023-11-24T06:10:27.198305+0000","last_became_peered":"2023-11-24T06:10:27.198305+0000","last_unstale":"2023-11-24T06:10:28.510986+0000","last_undegraded":"2023-11-24T06:10:28.510986+0000","last_fullsized":"2023-11-24T06:10:28.510986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:22.013253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194576+0000","last_change":"2023-11-24T06:10:28.207906+0000","last_active":"2023-11-24T06:10:29.194576+0000","last_peered":"2023-11-24T06:10:29.194576+0000","last_clean":"2023-11-24T06:10:29.194576+0000","last_became_active":"2023-11-24T06:10:28.206941+0000","last_became_peered":"2023-11-24T06:10:28.206941+0000","last_unstale":"2023-11-24T06:10:29.194576+0000","last_undegraded":"2023-11-24T06:10:29.194576+0000","last_fullsized":"2023-11-24T06:10:29.194576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:01.452285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194955+0000","last_change":"2023-11-24T06:10:27.199302+0000","last_active":"2023-11-24T06:10:29.194955+0000","last_peered":"2023-11-24T06:10:29.194955+0000","last_clean":"2023-11-24T06:10:29.194955+0000","last_became_active":"2023-11-24T06:10:27.198968+0000","last_became_peered":"2023-11-24T06:10:27.198968+0000","last_unstale":"2023-11-24T06:10:29.194955+0000","last_undegraded":"2023-11-24T06:10:29.194955+0000","last_fullsized":"2023-11-24T06:10:29.194955+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:38.091507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194973+0000","last_change":"2023-11-24T06:10:28.208328+0000","last_active":"2023-11-24T06:10:29.194973+0000","last_peered":"2023-11-24T06:10:29.194973+0000","last_clean":"2023-11-24T06:10:29.194973+0000","last_became_active":"2023-11-24T06:10:28.208000+0000","last_became_peered":"2023-11-24T06:10:28.208000+0000","last_unstale":"2023-11-24T06:10:29.194973+0000","last_undegraded":"2023-11-24T06:10:29.194973+0000","last_fullsized":"2023-11-24T06:10:29.194973+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:27.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.197344+0000","last_change":"2023-11-24T06:10:27.194472+0000","last_active":"2023-11-24T06:10:28.197344+0000","last_peered":"2023-11-24T06:10:28.197344+0000","last_clean":"2023-11-24T06:10:28.197344+0000","last_became_active":"2023-11-24T06:10:27.194169+0000","last_became_peered":"2023-11-24T06:10:27.194169+0000","last_unstale":"2023-11-24T06:10:28.197344+0000","last_undegraded":"2023-11-24T06:10:28.197344+0000","last_fullsized":"2023-11-24T06:10:28.197344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:53.094666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208705+0000","last_change":"2023-11-24T06:10:28.204667+0000","last_active":"2023-11-24T06:10:28.208705+0000","last_peered":"2023-11-24T06:10:28.208705+0000","last_clean":"2023-11-24T06:10:28.208705+0000","last_became_active":"2023-11-24T06:10:28.204193+0000","last_became_peered":"2023-11-24T06:10:28.204193+0000","last_unstale":"2023-11-24T06:10:28.208705+0000","last_undegraded":"2023-11-24T06:10:28.208705+0000","last_fullsized":"2023-11-24T06:10:28.208705+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:53:30.744751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.511419+0000","last_change":"2023-11-24T06:10:27.198819+0000","last_active":"2023-11-24T06:10:28.511419+0000","last_peered":"2023-11-24T06:10:28.511419+0000","last_clean":"2023-11-24T06:10:28.511419+0000","last_became_active":"2023-11-24T06:10:27.198522+0000","last_became_peered":"2023-11-24T06:10:27.198522+0000","last_unstale":"2023-11-24T06:10:28.511419+0000","last_undegraded":"2023-11-24T06:10:28.511419+0000","last_fullsized":"2023-11-24T06:10:28.511419+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:15.013775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207165+0000","last_change":"2023-11-24T06:10:28.199999+0000","last_active":"2023-11-24T06:10:28.207165+0000","last_peered":"2023-11-24T06:10:28.207165+0000","last_clean":"2023-11-24T06:10:28.207165+0000","last_became_active":"2023-11-24T06:10:28.199645+0000","last_became_peered":"2023-11-24T06:10:28.199645+0000","last_unstale":"2023-11-24T06:10:28.207165+0000","last_undegraded":"2023-11-24T06:10:28.207165+0000","last_fullsized":"2023-11-24T06:10:28.207165+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:30.721899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194842+0000","last_change":"2023-11-24T06:10:27.190895+0000","last_active":"2023-11-24T06:10:29.194842+0000","last_peered":"2023-11-24T06:10:29.194842+0000","last_clean":"2023-11-24T06:10:29.194842+0000","last_became_active":"2023-11-24T06:10:27.190630+0000","last_became_peered":"2023-11-24T06:10:27.190630+0000","last_unstale":"2023-11-24T06:10:29.194842+0000","last_undegraded":"2023-11-24T06:10:29.194842+0000","last_fullsized":"2023-11-24T06:10:29.194842+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:26.210074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.190129+0000","last_change":"2023-11-24T06:10:28.192165+0000","last_active":"2023-11-24T06:10:29.190129+0000","last_peered":"2023-11-24T06:10:29.190129+0000","last_clean":"2023-11-24T06:10:29.190129+0000","last_became_active":"2023-11-24T06:10:28.191917+0000","last_became_peered":"2023-11-24T06:10:28.191917+0000","last_unstale":"2023-11-24T06:10:29.190129+0000","last_undegraded":"2023-11-24T06:10:29.190129+0000","last_fullsized":"2023-11-24T06:10:29.190129+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:41.612455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.196307+0000","last_change":"2023-11-24T06:10:27.202260+0000","last_active":"2023-11-24T06:10:28.196307+0000","last_peered":"2023-11-24T06:10:28.196307+0000","last_clean":"2023-11-24T06:10:28.196307+0000","last_became_active":"2023-11-24T06:10:27.202068+0000","last_became_peered":"2023-11-24T06:10:27.202068+0000","last_unstale":"2023-11-24T06:10:28.196307+0000","last_undegraded":"2023-11-24T06:10:28.196307+0000","last_fullsized":"2023-11-24T06:10:28.196307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:27.403951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208951+0000","last_change":"2023-11-24T06:10:28.206494+0000","last_active":"2023-11-24T06:10:28.208951+0000","last_peered":"2023-11-24T06:10:28.208951+0000","last_clean":"2023-11-24T06:10:28.208951+0000","last_became_active":"2023-11-24T06:10:28.206185+0000","last_became_peered":"2023-11-24T06:10:28.206185+0000","last_unstale":"2023-11-24T06:10:28.208951+0000","last_undegraded":"2023-11-24T06:10:28.208951+0000","last_fullsized":"2023-11-24T06:10:28.208951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:05.162020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333624+0000","last_change":"2023-11-24T06:10:27.200397+0000","last_active":"2023-11-24T06:10:29.333624+0000","last_peered":"2023-11-24T06:10:29.333624+0000","last_clean":"2023-11-24T06:10:29.333624+0000","last_became_active":"2023-11-24T06:10:27.200128+0000","last_became_peered":"2023-11-24T06:10:27.200128+0000","last_unstale":"2023-11-24T06:10:29.333624+0000","last_undegraded":"2023-11-24T06:10:29.333624+0000","last_fullsized":"2023-11-24T06:10:29.333624+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:21.696001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194534+0000","last_change":"2023-11-24T06:10:28.202948+0000","last_active":"2023-11-24T06:10:29.194534+0000","last_peered":"2023-11-24T06:10:29.194534+0000","last_clean":"2023-11-24T06:10:29.194534+0000","last_became_active":"2023-11-24T06:10:28.202681+0000","last_became_peered":"2023-11-24T06:10:28.202681+0000","last_unstale":"2023-11-24T06:10:29.194534+0000","last_undegraded":"2023-11-24T06:10:29.194534+0000","last_fullsized":"2023-11-24T06:10:29.194534+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:28.409249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.197408+0000","last_change":"2023-11-24T06:10:27.194281+0000","last_active":"2023-11-24T06:10:28.197408+0000","last_peered":"2023-11-24T06:10:28.197408+0000","last_clean":"2023-11-24T06:10:28.197408+0000","last_became_active":"2023-11-24T06:10:27.193593+0000","last_became_peered":"2023-11-24T06:10:27.193593+0000","last_unstale":"2023-11-24T06:10:28.197408+0000","last_undegraded":"2023-11-24T06:10:28.197408+0000","last_fullsized":"2023-11-24T06:10:28.197408+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:42.698073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.207292+0000","last_change":"2023-11-24T06:10:28.205820+0000","last_active":"2023-11-24T06:10:28.207292+0000","last_peered":"2023-11-24T06:10:28.207292+0000","last_clean":"2023-11-24T06:10:28.207292+0000","last_became_active":"2023-11-24T06:10:28.205438+0000","last_became_peered":"2023-11-24T06:10:28.205438+0000","last_unstale":"2023-11-24T06:10:28.207292+0000","last_undegraded":"2023-11-24T06:10:28.207292+0000","last_fullsized":"2023-11-24T06:10:28.207292+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:01.465488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.334066+0000","last_change":"2023-11-24T06:10:27.200126+0000","last_active":"2023-11-24T06:10:29.334066+0000","last_peered":"2023-11-24T06:10:29.334066+0000","last_clean":"2023-11-24T06:10:29.334066+0000","last_became_active":"2023-11-24T06:10:27.199846+0000","last_became_peered":"2023-11-24T06:10:27.199846+0000","last_unstale":"2023-11-24T06:10:29.334066+0000","last_undegraded":"2023-11-24T06:10:29.334066+0000","last_fullsized":"2023-11-24T06:10:29.334066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:05.195669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333990+0000","last_change":"2023-11-24T06:10:28.199429+0000","last_active":"2023-11-24T06:10:29.333990+0000","last_peered":"2023-11-24T06:10:29.333990+0000","last_clean":"2023-11-24T06:10:29.333990+0000","last_became_active":"2023-11-24T06:10:28.199149+0000","last_became_peered":"2023-11-24T06:10:28.199149+0000","last_unstale":"2023-11-24T06:10:29.333990+0000","last_undegraded":"2023-11-24T06:10:29.333990+0000","last_fullsized":"2023-11-24T06:10:29.333990+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:47:32.815240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.192043+0000","last_change":"2023-11-24T06:10:27.199260+0000","last_active":"2023-11-24T06:10:28.192043+0000","last_peered":"2023-11-24T06:10:28.192043+0000","last_clean":"2023-11-24T06:10:28.192043+0000","last_became_active":"2023-11-24T06:10:27.199002+0000","last_became_peered":"2023-11-24T06:10:27.199002+0000","last_unstale":"2023-11-24T06:10:28.192043+0000","last_undegraded":"2023-11-24T06:10:28.192043+0000","last_fullsized":"2023-11-24T06:10:28.192043+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:00.460189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208682+0000","last_change":"2023-11-24T06:10:28.204499+0000","last_active":"2023-11-24T06:10:28.208682+0000","last_peered":"2023-11-24T06:10:28.208682+0000","last_clean":"2023-11-24T06:10:28.208682+0000","last_became_active":"2023-11-24T06:10:28.203878+0000","last_became_peered":"2023-11-24T06:10:28.203878+0000","last_unstale":"2023-11-24T06:10:28.208682+0000","last_undegraded":"2023-11-24T06:10:28.208682+0000","last_fullsized":"2023-11-24T06:10:28.208682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:24.055216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.333924+0000","last_change":"2023-11-24T06:10:27.198660+0000","last_active":"2023-11-24T06:10:29.333924+0000","last_peered":"2023-11-24T06:10:29.333924+0000","last_clean":"2023-11-24T06:10:29.333924+0000","last_became_active":"2023-11-24T06:10:27.198406+0000","last_became_peered":"2023-11-24T06:10:27.198406+0000","last_unstale":"2023-11-24T06:10:29.333924+0000","last_undegraded":"2023-11-24T06:10:29.333924+0000","last_fullsized":"2023-11-24T06:10:29.333924+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:38.091507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.208177+0000","last_change":"2023-11-24T06:10:28.203247+0000","last_active":"2023-11-24T06:10:28.208177+0000","last_peered":"2023-11-24T06:10:28.208177+0000","last_clean":"2023-11-24T06:10:28.208177+0000","last_became_active":"2023-11-24T06:10:28.202976+0000","last_became_peered":"2023-11-24T06:10:28.202976+0000","last_unstale":"2023-11-24T06:10:28.208177+0000","last_undegraded":"2023-11-24T06:10:28.208177+0000","last_fullsized":"2023-11-24T06:10:28.208177+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:29:06.200967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.190810+0000","last_change":"2023-11-24T06:10:27.194184+0000","last_active":"2023-11-24T06:10:28.190810+0000","last_peered":"2023-11-24T06:10:28.190810+0000","last_clean":"2023-11-24T06:10:28.190810+0000","last_became_active":"2023-11-24T06:10:27.193859+0000","last_became_peered":"2023-11-24T06:10:27.193859+0000","last_unstale":"2023-11-24T06:10:28.190810+0000","last_undegraded":"2023-11-24T06:10:28.190810+0000","last_fullsized":"2023-11-24T06:10:28.190810+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:46:32.577027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194886+0000","last_change":"2023-11-24T06:10:28.199326+0000","last_active":"2023-11-24T06:10:29.194886+0000","last_peered":"2023-11-24T06:10:29.194886+0000","last_clean":"2023-11-24T06:10:29.194886+0000","last_became_active":"2023-11-24T06:10:28.199006+0000","last_became_peered":"2023-11-24T06:10:28.199006+0000","last_unstale":"2023-11-24T06:10:29.194886+0000","last_undegraded":"2023-11-24T06:10:29.194886+0000","last_fullsized":"2023-11-24T06:10:29.194886+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:21:41.131179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.191996+0000","last_change":"2023-11-24T06:10:27.198421+0000","last_active":"2023-11-24T06:10:28.191996+0000","last_peered":"2023-11-24T06:10:28.191996+0000","last_clean":"2023-11-24T06:10:28.191996+0000","last_became_active":"2023-11-24T06:10:27.198163+0000","last_became_peered":"2023-11-24T06:10:27.198163+0000","last_unstale":"2023-11-24T06:10:28.191996+0000","last_undegraded":"2023-11-24T06:10:28.191996+0000","last_fullsized":"2023-11-24T06:10:28.191996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:00.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.210897+0000","last_change":"2023-11-24T06:10:28.206091+0000","last_active":"2023-11-24T06:10:28.210897+0000","last_peered":"2023-11-24T06:10:28.210897+0000","last_clean":"2023-11-24T06:10:28.210897+0000","last_became_active":"2023-11-24T06:10:28.205411+0000","last_became_peered":"2023-11-24T06:10:28.205411+0000","last_unstale":"2023-11-24T06:10:28.210897+0000","last_undegraded":"2023-11-24T06:10:28.210897+0000","last_fullsized":"2023-11-24T06:10:28.210897+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:40.132683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.196387+0000","last_change":"2023-11-24T06:10:27.200706+0000","last_active":"2023-11-24T06:10:28.196387+0000","last_peered":"2023-11-24T06:10:28.196387+0000","last_clean":"2023-11-24T06:10:28.196387+0000","last_became_active":"2023-11-24T06:10:27.200456+0000","last_became_peered":"2023-11-24T06:10:27.200456+0000","last_unstale":"2023-11-24T06:10:28.196387+0000","last_undegraded":"2023-11-24T06:10:28.196387+0000","last_fullsized":"2023-11-24T06:10:28.196387+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:00.446987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.211453+0000","last_change":"2023-11-24T06:10:28.210256+0000","last_active":"2023-11-24T06:10:28.211453+0000","last_peered":"2023-11-24T06:10:28.211453+0000","last_clean":"2023-11-24T06:10:28.211453+0000","last_became_active":"2023-11-24T06:10:28.209816+0000","last_became_peered":"2023-11-24T06:10:28.209816+0000","last_unstale":"2023-11-24T06:10:28.211453+0000","last_undegraded":"2023-11-24T06:10:28.211453+0000","last_fullsized":"2023-11-24T06:10:28.211453+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:27.174042+0000","last_clean_scrub_stamp":"2023-11-24T06:10:27.174042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:27.215372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-11-24T06:10:29.194824+0000","last_change":"2023-11-24T06:10:27.188912+0000","last_active":"2023-11-24T06:10:29.194824+0000","last_peered":"2023-11-24T06:10:29.194824+0000","last_clean":"2023-11-24T06:10:29.194824+0000","last_became_active":"2023-11-24T06:10:27.188618+0000","last_became_peered":"2023-11-24T06:10:27.188618+0000","last_unstale":"2023-11-24T06:10:29.194824+0000","last_undegraded":"2023-11-24T06:10:29.194824+0000","last_fullsized":"2023-11-24T06:10:29.194824+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:10:26.168744+0000","last_clean_scrub_stamp":"2023-11-24T06:10:26.168744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:21.696001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T06:10:28.188583+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:10:28.188583+0000","last_peered":"2023-11-24T06:10:28.188583+0000","last_clean":"2023-11-24T06:10:28.188583+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:10:28.188583+0000","last_undegraded":"2023-11-24T06:10:28.188583+0000","last_fullsized":"2023-11-24T06:10:28.188583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":799257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.91500000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.747}]},{"osd":2,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.622},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.876}]},{"osd":5,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.97899999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.96699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":8.4190000000000005,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":9.5960000000000001,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.502}]},{"osd":2,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":8.4849999999999994,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.75}]},{"osd":5,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":208869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.26300000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.32800000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.372}]},{"osd":4,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.55100000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.622},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:09:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":799201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":12.378,"5min":12.378,"15min":12.378},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.58799999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":4.5199999999999996,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.6900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.502}]},{"osd":4,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":15.319000000000001,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":4.7590000000000003,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":15.356,"5min":15.356,"15min":15.356},"last":0.877}]},{"osd":6,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.755},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.726,"5min":4.726,"15min":4.726},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:09:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":4.7869999999999999,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":15.388,"5min":15.388,"15min":15.388},"last":0.78500000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":210433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.33400000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34999999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.374}]},{"osd":4,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.40899999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.443}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.48399999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":209437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.254},{"interface":"front","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.30299999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":1.304}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":1.161},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":1.1120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":208799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60999999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.622}]},{"osd":2,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.44},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:09:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.63400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":208869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":3.335,"5min":3.335,"15min":3.335},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.4580000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.42699999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":4.4489999999999998,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.4940000000000002,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":4.5069999999999997,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 06:09:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.60999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:10:32.969 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:10:32.969 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:10:33.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.129+0000 7f003df85700 1 Processor -- start 2023-11-24T06:10:33.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.129+0000 7f003df85700 1 -- start start 2023-11-24T06:10:33.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f003df85700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f003815e140 0x7f00380d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f003df85700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f00380b98c0 0x7f00380d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f003df85700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00380b80e0 con 0x7f00380d39d0 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f003df85700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003815e520 con 0x7f003815e140 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f003df85700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00380d2930 con 0x7f00380b98c0 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f00377fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f003815e140 0x7f00380d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f00377fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f003815e140 0x7f00380d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48044/0 (socket says 172.21.15.97:48044) 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f00377fe700 1 -- 172.21.15.97:0/1601226774 learned_addr learned my addr 172.21.15.97:0/1601226774 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f0037fff700 1 --2- 172.21.15.97:0/1601226774 >> v2:172.21.15.183:3300/0 conn(0x7f00380b98c0 0x7f00380d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f00377fe700 1 -- 172.21.15.97:0/1601226774 >> v2:172.21.15.183:3300/0 conn(0x7f00380b98c0 msgr2=0x7f00380d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f00377fe700 1 --2- 172.21.15.97:0/1601226774 >> v2:172.21.15.183:3300/0 conn(0x7f00380b98c0 0x7f00380d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f00377fe700 1 -- 172.21.15.97:0/1601226774 >> v1:172.21.15.183:6789/0 conn(0x7f00380d39d0 legacy=0x7f00380b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.130+0000 7f00377fe700 1 -- 172.21.15.97:0/1601226774 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00380d2c10 con 0x7f003815e140 2023-11-24T06:10:33.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f00377fe700 1 --2- 172.21.15.97:0/1601226774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003815e140 0x7f00380d3ec0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f00380b6df0 tx=0x7f002c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9421f97034790f83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f00367fc700 1 -- 172.21.15.97:0/1601226774 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00b870 con 0x7f003815e140 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f00367fc700 1 -- 172.21.15.97:0/1601226774 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c00b9d0 con 0x7f003815e140 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f00367fc700 1 -- 172.21.15.97:0/1601226774 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00bd60 con 0x7f003815e140 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f003df85700 1 -- 172.21.15.97:0/1601226774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003815e140 msgr2=0x7f00380d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f003df85700 1 --2- 172.21.15.97:0/1601226774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003815e140 0x7f00380d3ec0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f00380b6df0 tx=0x7f002c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f003df85700 1 -- 172.21.15.97:0/1601226774 shutdown_connections 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f003df85700 1 --2- 172.21.15.97:0/1601226774 >> v2:172.21.15.183:3300/0 conn(0x7f00380b98c0 0x7f00380d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f003df85700 1 --2- 172.21.15.97:0/1601226774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f003815e140 0x7f00380d3ec0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f003df85700 1 -- 172.21.15.97:0/1601226774 >> 172.21.15.97:0/1601226774 conn(0x7f003806b410 msgr2=0x7f003806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f003df85700 1 -- 172.21.15.97:0/1601226774 shutdown_connections 2023-11-24T06:10:33.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.131+0000 7f003df85700 1 -- 172.21.15.97:0/1601226774 wait complete. 2023-11-24T06:10:33.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.133+0000 7f003df85700 1 Processor -- start 2023-11-24T06:10:33.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.133+0000 7f003df85700 1 -- start start 2023-11-24T06:10:33.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f003df85700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380b98c0 0x7f0038154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f003df85700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00380d39d0 0x7f0038157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f003df85700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003815e140 0x7f00380c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f003df85700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003815a800 con 0x7f00380b98c0 2023-11-24T06:10:33.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f003df85700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00381565a0 con 0x7f003815e140 2023-11-24T06:10:33.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f003df85700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0038156880 con 0x7f00380d39d0 2023-11-24T06:10:33.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380b98c0 0x7f0038154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380b98c0 0x7f0038154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48050/0 (socket says 172.21.15.97:48050) 2023-11-24T06:10:33.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 -- 172.21.15.97:0/2451360794 learned_addr learned my addr 172.21.15.97:0/2451360794 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f0036ffd700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00380d39d0 0x7f0038157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f0037fff700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003815e140 0x7f00380c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 -- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00380d39d0 msgr2=0x7f0038157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00380d39d0 0x7f0038157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 -- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003815e140 msgr2=0x7f00380c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003815e140 0x7f00380c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 -- 172.21.15.97:0/2451360794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c009550 con 0x7f00380b98c0 2023-11-24T06:10:33.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f0037fff700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003815e140 0x7f00380c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:33.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.134+0000 7f00377fe700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380b98c0 0x7f0038154f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f002c000c00 tx=0x7f002c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:33.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.135+0000 7f001ffff700 1 -- 172.21.15.97:0/2451360794 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c004520 con 0x7f00380b98c0 2023-11-24T06:10:33.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.135+0000 7f001ffff700 1 -- 172.21.15.97:0/2451360794 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c01e070 con 0x7f00380b98c0 2023-11-24T06:10:33.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.135+0000 7f001ffff700 1 -- 172.21.15.97:0/2451360794 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0038a0 con 0x7f00380b98c0 2023-11-24T06:10:33.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.135+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0038156b60 con 0x7f00380b98c0 2023-11-24T06:10:33.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.135+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0038157120 con 0x7f00380b98c0 2023-11-24T06:10:33.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.136+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00381572b0 con 0x7f00380b98c0 2023-11-24T06:10:33.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.136+0000 7f001ffff700 1 -- 172.21.15.97:0/2451360794 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f002c003a00 con 0x7f00380b98c0 2023-11-24T06:10:33.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.136+0000 7f001ffff700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002406af60 0x7f002406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.136+0000 7f001ffff700 1 -- 172.21.15.97:0/2451360794 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f002c020070 con 0x7f00380b98c0 2023-11-24T06:10:33.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.136+0000 7f0036ffd700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002406af60 0x7f002406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.137+0000 7f0036ffd700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002406af60 0x7f002406d420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0028007fd0 tx=0x7f0028009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:33.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.141+0000 7f001ffff700 1 -- 172.21.15.97:0/2451360794 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f002c055c60 con 0x7f00380b98c0 2023-11-24T06:10:33.170 INFO:tasks.ceph.mds.c.smithi097.stderr:2023-11-24T06:10:33.167+0000 7fa9e1df8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:10:33.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.412+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f00380d1bb0 con 0x7f00380b98c0 2023-11-24T06:10:33.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.414+0000 7f001ffff700 1 -- 172.21.15.97:0/2451360794 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f002c015020 con 0x7f00380b98c0 2023-11-24T06:10:33.417 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-11-24T06:10:33.417 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA== 2023-11-24T06:10:33.417 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-11-24T06:10:33.417 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-11-24T06:10:33.417 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:10:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002406af60 msgr2=0x7f002406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002406af60 0x7f002406d420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0028007fd0 tx=0x7f0028009040 comp rx=0 tx=0).stop 2023-11-24T06:10:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380b98c0 msgr2=0x7f0038154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380b98c0 0x7f0038154f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f002c000c00 tx=0x7f002c004750 comp rx=0 tx=0).stop 2023-11-24T06:10:33.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 shutdown_connections 2023-11-24T06:10:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f003815e140 0x7f00380c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380b98c0 0x7f0038154f90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f002406af60 0x7f002406d420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 --2- 172.21.15.97:0/2451360794 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f00380d39d0 0x7f0038157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 >> 172.21.15.97:0/2451360794 conn(0x7f003806b410 msgr2=0x7f00380b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 shutdown_connections 2023-11-24T06:10:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.416+0000 7f003df85700 1 -- 172.21.15.97:0/2451360794 wait complete. 2023-11-24T06:10:33.420 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-11-24T06:10:33.431 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.550+0000 7f007d8d5700 1 Processor -- start 2023-11-24T06:10:33.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.550+0000 7f007d8d5700 1 -- start start 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.551+0000 7f007d8d5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f007815e080 0x7f00780b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.551+0000 7f007d8d5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f00780afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.551+0000 7f007d8d5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00780ae430 con 0x7f00780af4d0 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.551+0000 7f007d8d5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00780af0b0 con 0x7f007815e080 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.551+0000 7f007d8d5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780b05a0 con 0x7f007806c710 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.551+0000 7f0076ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f007815e080 0x7f00780b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.551+0000 7f0076ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f007815e080 0x7f00780b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48058/0 (socket says 172.21.15.97:48058) 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.551+0000 7f0076ffd700 1 -- 172.21.15.97:0/3049351019 learned_addr learned my addr 172.21.15.97:0/3049351019 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f00777fe700 1 --2- 172.21.15.97:0/3049351019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f00780afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f0076ffd700 1 -- 172.21.15.97:0/3049351019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 msgr2=0x7f00780afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f0076ffd700 1 --2- 172.21.15.97:0/3049351019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f00780afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f0076ffd700 1 -- 172.21.15.97:0/3049351019 >> v1:172.21.15.183:6789/0 conn(0x7f00780af4d0 legacy=0x7f007806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f0076ffd700 1 -- 172.21.15.97:0/3049351019 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780ae710 con 0x7f007815e080 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f00777fe700 1 --2- 172.21.15.97:0/3049351019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f00780afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f0076ffd700 1 --2- 172.21.15.97:0/3049351019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f00780b5050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f006c009920 tx=0x7f006c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672672017d756afc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f0075ffb700 1 -- 172.21.15.97:0/3049351019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c004230 con 0x7f007815e080 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f0075ffb700 1 -- 172.21.15.97:0/3049351019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c004390 con 0x7f007815e080 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f0075ffb700 1 -- 172.21.15.97:0/3049351019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c009e70 con 0x7f007815e080 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f007d8d5700 1 -- 172.21.15.97:0/3049351019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 msgr2=0x7f00780b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.552+0000 7f007d8d5700 1 --2- 172.21.15.97:0/3049351019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f00780b5050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f006c009920 tx=0x7f006c009c30 comp rx=0 tx=0).stop 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.553+0000 7f007d8d5700 1 -- 172.21.15.97:0/3049351019 shutdown_connections 2023-11-24T06:10:33.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.553+0000 7f007d8d5700 1 --2- 172.21.15.97:0/3049351019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f00780afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.553+0000 7f007d8d5700 1 --2- 172.21.15.97:0/3049351019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f00780b5050 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.553+0000 7f007d8d5700 1 -- 172.21.15.97:0/3049351019 >> 172.21.15.97:0/3049351019 conn(0x7f0078071b90 msgr2=0x7f0078071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.553+0000 7f007d8d5700 1 -- 172.21.15.97:0/3049351019 shutdown_connections 2023-11-24T06:10:33.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.553+0000 7f007d8d5700 1 -- 172.21.15.97:0/3049351019 wait complete. 2023-11-24T06:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.554+0000 7f007d8d5700 1 Processor -- start 2023-11-24T06:10:33.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.554+0000 7f007d8d5700 1 -- start start 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f007d8d5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f0078151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f007d8d5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00780af4d0 0x7f0078145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f007d8d5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f0078157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f007d8d5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0078154440 con 0x7f007815e080 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f007d8d5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00781534e0 con 0x7f00780af4d0 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f007d8d5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00781f8330 con 0x7f007806c710 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f0078157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f0078157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48068/0 (socket says 172.21.15.97:48068) 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 -- 172.21.15.97:0/33823441 learned_addr learned my addr 172.21.15.97:0/33823441 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f0076ffd700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f0078151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00767fc700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00780af4d0 0x7f0078145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 -- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 msgr2=0x7f0078151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f0078151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 -- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00780af4d0 msgr2=0x7f0078145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00780af4d0 0x7f0078145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 -- 172.21.15.97:0/33823441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c009550 con 0x7f007815e080 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f0076ffd700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f0078151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00767fc700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00780af4d0 0x7f0078145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f00777fe700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f0078157400 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f007000cea0 tx=0x7f007000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f005ffff700 1 -- 172.21.15.97:0/33823441 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000ae50 con 0x7f007815e080 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f005ffff700 1 -- 172.21.15.97:0/33823441 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0070003a20 con 0x7f007815e080 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f005ffff700 1 -- 172.21.15.97:0/33823441 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070003dd0 con 0x7f007815e080 2023-11-24T06:10:33.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0078153d80 con 0x7f007815e080 2023-11-24T06:10:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.555+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0078152510 con 0x7f007815e080 2023-11-24T06:10:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.556+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00780753f0 con 0x7f007815e080 2023-11-24T06:10:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.557+0000 7f005ffff700 1 -- 172.21.15.97:0/33823441 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0070003f30 con 0x7f007815e080 2023-11-24T06:10:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.557+0000 7f005ffff700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f006406af60 0x7f006406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.557+0000 7f0076ffd700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f006406af60 0x7f006406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.557+0000 7f005ffff700 1 -- 172.21.15.97:0/33823441 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0070089a50 con 0x7f007815e080 2023-11-24T06:10:33.563 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.558+0000 7f0076ffd700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f006406af60 0x7f006406d420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f006c009870 tx=0x7f006c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:33.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.562+0000 7f005ffff700 1 -- 172.21.15.97:0/33823441 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0070054b10 con 0x7f007815e080 2023-11-24T06:10:33.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.796+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00780d09b0 con 0x7f007815e080 2023-11-24T06:10:33.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.796+0000 7f005ffff700 1 -- 172.21.15.97:0/33823441 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f0070058130 con 0x7f007815e080 2023-11-24T06:10:33.799 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:33.799 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/163703518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":163703518},{"type":"v1","addr":"172.21.15.97:6837","nonce":163703518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/1688123009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1688123009},{"type":"v1","addr":"172.21.15.183:6834","nonce":1688123009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/310487911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":310487911},{"type":"v1","addr":"172.21.15.183:6835","nonce":310487911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:27.576219+0000","modified":"2023-11-24T06:10:30.898770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/19331625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":19331625},{"type":"v1","addr":"172.21.15.97:6835","nonce":19331625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:10:33.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f006406af60 msgr2=0x7f006406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f006406af60 0x7f006406d420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f006c009870 tx=0x7f006c008030 comp rx=0 tx=0).stop 2023-11-24T06:10:33.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 msgr2=0x7f0078157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:33.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f0078157400 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f007000cea0 tx=0x7f007000cf80 comp rx=0 tx=0).stop 2023-11-24T06:10:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 shutdown_connections 2023-11-24T06:10:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f007815e080 0x7f0078157400 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f006406af60 0x7f006406d420 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f007806c710 0x7f0078151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 --2- 172.21.15.97:0/33823441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00780af4d0 0x7f0078145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 >> 172.21.15.97:0/33823441 conn(0x7f0078071b90 msgr2=0x7f00780b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 shutdown_connections 2023-11-24T06:10:33.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:33.798+0000 7f007d8d5700 1 -- 172.21.15.97:0/33823441 wait complete. 2023-11-24T06:10:33.803 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2023-11-24T06:10:34.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:10:27.576219+0000', 'modified': '2023-11-24T06:10:30.898770+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5532}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5532': {'gid': 5532, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6835/19331625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 19331625}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 19331625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:10:34.216 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:10:34.216 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T06:10:34.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.337+0000 7fcb0f100700 1 Processor -- start 2023-11-24T06:10:34.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.337+0000 7fcb0f100700 1 -- start start 2023-11-24T06:10:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.337+0000 7fcb0f100700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb08158670 0x7fcb080622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.337+0000 7fcb0f100700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcb080c9fc0 0x7fcb080ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.337+0000 7fcb0f100700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb080cfff0 con 0x7fcb08158a50 2023-11-24T06:10:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.337+0000 7fcb0f100700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb080cfd10 con 0x7fcb080c9fc0 2023-11-24T06:10:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.337+0000 7fcb0f100700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08161f50 con 0x7fcb08158670 2023-11-24T06:10:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb07fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcb08158a50 0x7fcb080c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40526/0 (socket says 172.21.15.97:40526) 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb07fff700 1 -- 172.21.15.97:0/1006966738 learned_addr learned my addr 172.21.15.97:0/1006966738 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb08158670 0x7fcb080622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb0d69d700 1 --2- 172.21.15.97:0/1006966738 >> v2:172.21.15.183:3300/0 conn(0x7fcb080c9fc0 0x7fcb080ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294112580 0 0) 0x7fcb080cfff0 con 0x7fcb08158a50 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf80035b0 con 0x7fcb08158a50 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1364554963 0 0) 0x7fcaf80035b0 con 0x7fcb08158a50 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb08158670 msgr2=0x7fcb080622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb08158670 0x7fcb080622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 >> v2:172.21.15.183:3300/0 conn(0x7fcb080c9fc0 msgr2=0x7fcb080ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 --2- 172.21.15.97:0/1006966738 >> v2:172.21.15.183:3300/0 conn(0x7fcb080c9fc0 0x7fcb080ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb08162230 con 0x7fcb08158a50 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fcaf4001690 con 0x7fcb08158a50 2023-11-24T06:10:34.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 >> v1:172.21.15.97:6789/0 conn(0x7fcb08158a50 legacy=0x7fcb080c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcaf8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf8004850 0x7fcaf8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb08158670 0x7fcaf8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb080cfd10 con 0x7fcaf8003a00 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08161f50 con 0x7fcaf8004850 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf800ba10 con 0x7fcb08158670 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcaf8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb0d69d700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb08158670 0x7fcaf8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb07fff700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf8004850 0x7fcaf8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb08158670 msgr2=0x7fcaf8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb08158670 0x7fcaf8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf8004850 msgr2=0x7fcaf8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.338+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf8004850 0x7fcaf8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/1006966738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80035b0 con 0x7fcaf8003a00 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcaf8003e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009b90 tx=0x7fcaf8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ba04ab8229d782 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00e6c0 con 0x7fcaf8003a00 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc00e820 con 0x7fcaf8003a00 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb077fe700 1 -- 172.21.15.97:0/1006966738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00eb90 con 0x7fcaf8003a00 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0f100700 1 -- 172.21.15.97:0/1006966738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 msgr2=0x7fcaf8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0f100700 1 --2- 172.21.15.97:0/1006966738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcaf8003e20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009b90 tx=0x7fcaf8005880 comp rx=0 tx=0).stop 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0d69d700 1 -- 172.21.15.97:0/1006966738 reap_dead start 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0f100700 1 -- 172.21.15.97:0/1006966738 shutdown_connections 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0f100700 1 -- 172.21.15.97:0/1006966738 >> 172.21.15.97:0/1006966738 conn(0x7fcb080b57b0 msgr2=0x7fcb080b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0f100700 1 -- 172.21.15.97:0/1006966738 shutdown_connections 2023-11-24T06:10:34.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.339+0000 7fcb0f100700 1 -- 172.21.15.97:0/1006966738 wait complete. 2023-11-24T06:10:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.341+0000 7fcb0f100700 1 Processor -- start 2023-11-24T06:10:34.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.341+0000 7fcb0f100700 1 -- start start 2023-11-24T06:10:34.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.341+0000 7fcb0f100700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcb080cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.341+0000 7fcb0f100700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb08148750 0x7fcb080cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.341+0000 7fcb0f100700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb080b4290 0x7fcb080b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.341+0000 7fcb0f100700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb080b3e70 con 0x7fcaf8003a00 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.341+0000 7fcb0f100700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb080cdb10 con 0x7fcb08148750 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.341+0000 7fcb0f100700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08148bd0 con 0x7fcb080b4290 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcb080cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcb080cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48084/0 (socket says 172.21.15.97:48084) 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/4187847356 learned_addr learned my addr 172.21.15.97:0/4187847356 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb07fff700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb08148750 0x7fcb080cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0d69d700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb080b4290 0x7fcb080b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb080b4290 msgr2=0x7fcb080b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb080b4290 0x7fcb080b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb08148750 msgr2=0x7fcb080cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb08148750 0x7fcb080cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/4187847356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080cd850 con 0x7fcaf8003a00 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb07fff700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb08148750 0x7fcb080cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcb080cac20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e4b0 tx=0x7fcafc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb057fa700 1 -- 172.21.15.97:0/4187847356 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc00e6c0 con 0x7fcaf8003a00 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb057fa700 1 -- 172.21.15.97:0/4187847356 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcafc003ab0 con 0x7fcaf8003a00 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0f100700 1 -- 172.21.15.97:0/4187847356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb080b34d0 con 0x7fcaf8003a00 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb057fa700 1 -- 172.21.15.97:0/4187847356 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc003e60 con 0x7fcaf8003a00 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.342+0000 7fcb0f100700 1 -- 172.21.15.97:0/4187847356 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb080b3a90 con 0x7fcaf8003a00 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.344+0000 7fcb057fa700 1 -- 172.21.15.97:0/4187847356 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcafc013470 con 0x7fcaf8003a00 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.344+0000 7fcb057fa700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaec06af60 0x7fcaec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.344+0000 7fcb057fa700 1 -- 172.21.15.97:0/4187847356 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcafc08af20 con 0x7fcaf8003a00 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.344+0000 7fcb07fff700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaec06af60 0x7fcaec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.344+0000 7fcb07fff700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaec06af60 0x7fcaec06d420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb0806e440 tx=0x7fcaf4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.346+0000 7fcb1b2f9b80 1 Processor -- start 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.346+0000 7fcb1b2f9b80 1 -- start start 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 0x55d96a210840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a210d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1c7490 0x55d96a1f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96a13d770 con 0x55d96a1dfed0 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96a124750 con 0x55d96a1c7490 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d96a122070 con 0x55d96a1ca080 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a210d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a210d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48094/0 (socket says 172.21.15.97:48094) 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 -- 172.21.15.97:0/1678667123 learned_addr learned my addr 172.21.15.97:0/1678667123 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:34.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 0x55d96a210840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb0d69d700 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1c7490 0x55d96a1f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 -- 172.21.15.97:0/1678667123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 msgr2=0x55d96a210840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 0x55d96a210840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 -- 172.21.15.97:0/1678667123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1c7490 msgr2=0x55d96a1f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1c7490 0x55d96a1f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 -- 172.21.15.97:0/1678667123 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d969f9a5e0 con 0x55d96a1dfed0 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.347+0000 7fcb07fff700 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a210d80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4007270 tx=0x7fcaf4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299d1ff6eec75062 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcaf27fc700 1 -- 172.21.15.97:0/1678667123 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4015780 con 0x55d96a1dfed0 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcaf27fc700 1 -- 172.21.15.97:0/1678667123 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf40158e0 con 0x55d96a1dfed0 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcaf27fc700 1 -- 172.21.15.97:0/1678667123 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4015c70 con 0x55d96a1dfed0 2023-11-24T06:10:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1678667123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 msgr2=0x55d96a210d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a210d80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4007270 tx=0x7fcaf4014000 comp rx=0 tx=0).stop 2023-11-24T06:10:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1678667123 shutdown_connections 2023-11-24T06:10:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1c7490 0x55d96a1f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 0x55d96a210840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/1678667123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a210d80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1678667123 >> 172.21.15.97:0/1678667123 conn(0x55d96a0f8310 msgr2=0x55d96a20ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1678667123 shutdown_connections 2023-11-24T06:10:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.348+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1678667123 wait complete. 2023-11-24T06:10:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.349+0000 7fcb1b2f9b80 1 -- start start 2023-11-24T06:10:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.349+0000 7fcb1b2f9b80 1 Processor -- start 2023-11-24T06:10:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1c7490 0x55d96a1be330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1be870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 0x55d96a206200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96a11d960 con 0x55d96a1c7490 2023-11-24T06:10:34.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96a0d7510 con 0x55d96a1ca080 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d96a0d71b0 con 0x55d96a1dfed0 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb1b2f9b80 10 client.0 _ll_register_callbacks cb 0x55d96a002a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0ce9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1c7490 0x55d96a1be330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0ce9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1c7490 0x55d96a1be330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48098/0 (socket says 172.21.15.97:48098) 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/422201198 learned_addr learned my addr 172.21.15.97:0/422201198 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0d69d700 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 0x55d96a206200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb07fff700 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1be870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 msgr2=0x55d96a206200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 0x55d96a206200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 msgr2=0x55d96a1be870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1be870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb0ce9c700 1 -- 172.21.15.97:0/422201198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb08162230 con 0x55d96a1c7490 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.350+0000 7fcb07fff700 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1c7490 0x55d96a1be330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009b30 tx=0x7fcafc0263d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd703a8a709fde5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcae77fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcae77fe700 1 -- 172.21.15.97:0/422201198 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcafc059420 con 0x55d96a1c7490 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcae77fe700 1 -- 172.21.15.97:0/422201198 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcafc059d30 con 0x55d96a1c7490 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/422201198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d96a1bca40 con 0x55d96a1c7490 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcae77fe700 1 -- 172.21.15.97:0/422201198 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcafc05de80 con 0x55d96a1c7490 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 10 client.5667 fetch_fsmap learned FSMap version 26 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/422201198 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55d969f46d90 con 0x55d96a1c7490 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcae77fe700 1 -- 172.21.15.97:0/422201198 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d96a1bca40 con 0x55d96a1c7490 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcae77fe700 1 -- 172.21.15.97:0/422201198 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcafc061ba0 con 0x55d96a1c7490 2023-11-24T06:10:34.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 10 client.5667 fetch_fsmap finished waiting for FSMap version 26 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 10 client.5667 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 20 client.5667 populate_metadata read hostname 'smithi097' 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] conn(0x55d96a1f6060 0x55d96a206740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 4 client.5667 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/422201198 --> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d96a198840 con 0x55d96a1f6060 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.351+0000 7fcb07fff700 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] conn(0x55d96a1f6060 0x55d96a206740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.352+0000 7fcb07fff700 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] conn(0x55d96a1f6060 0x55d96a206740 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11644806d7d8ba84 server_cookie=9e1f67d2c73769bb in_seq=0 out_seq=0 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.352+0000 7fcae77fe700 10 client.5667 ms_handle_connect on v2:172.21.15.97:6834/19331625 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.352+0000 7fcae77fe700 1 -- 172.21.15.97:0/422201198 <== mds.0 v2:172.21.15.97:6834/19331625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d96a198840 con 0x55d96a1f6060 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.352+0000 7fcae77fe700 10 client.5667 handle_command_reply: tid=0 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.352+0000 7fcb1b2f9b80 1 client.5667 shutdown 2023-11-24T06:10:34.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/422201198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1c7490 msgr2=0x55d96a1be330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1c7490 0x55d96a1be330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009b30 tx=0x7fcafc0263d0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/422201198 shutdown_connections 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] conn(0x55d96a1f6060 0x55d96a206740 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 0x55d96a206200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcae77fe700 0 client.5667 ms_handle_reset on v2:172.21.15.97:6834/19331625 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1c7490 0x55d96a1be330 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/422201198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1be870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/422201198 >> 172.21.15.97:0/422201198 conn(0x55d96a0f8310 msgr2=0x55d96a212c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/422201198 shutdown_connections 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/422201198 wait complete. 2023-11-24T06:10:34.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.353+0000 7fcb1b2f9b80 20 client.5667 trim_cache size 0 max 16384 2023-11-24T06:10:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.371+0000 7fcb1b2f9b80 1 Processor -- start 2023-11-24T06:10:34.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.371+0000 7fcb1b2f9b80 1 -- start start 2023-11-24T06:10:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 0x55d96a207680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a20b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1f72b0 0x55d96a208fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d969f9c8d0 con 0x55d96a1dfed0 2023-11-24T06:10:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96a1c5a60 con 0x55d96a1f72b0 2023-11-24T06:10:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d96a221620 con 0x55d96a1ca080 2023-11-24T06:10:34.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a20b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a20b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48108/0 (socket says 172.21.15.97:48108) 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 -- 172.21.15.97:0/1433326178 learned_addr learned my addr 172.21.15.97:0/1433326178 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 -- 172.21.15.97:0/1433326178 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 msgr2=0x55d96a207680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 --2- 172.21.15.97:0/1433326178 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 0x55d96a207680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 -- 172.21.15.97:0/1433326178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1f72b0 msgr2=0x55d96a208fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 --2- 172.21.15.97:0/1433326178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1f72b0 0x55d96a208fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 -- 172.21.15.97:0/1433326178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d969f2fc10 con 0x55d96a1dfed0 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcb07fff700 1 --2- 172.21.15.97:0/1433326178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a20b930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4021000 tx=0x7fcaf4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41aa8708be65abe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcaf2ffd700 1 -- 172.21.15.97:0/1433326178 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4015d50 con 0x55d96a1dfed0 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcaf2ffd700 1 -- 172.21.15.97:0/1433326178 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf4001af0 con 0x55d96a1dfed0 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.372+0000 7fcaf2ffd700 1 -- 172.21.15.97:0/1433326178 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4032360 con 0x55d96a1dfed0 2023-11-24T06:10:34.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1433326178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 msgr2=0x55d96a20b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/1433326178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a20b930 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4021000 tx=0x7fcaf4015000 comp rx=0 tx=0).stop 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1433326178 shutdown_connections 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/1433326178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1f72b0 0x55d96a208fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/1433326178 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1ca080 0x55d96a207680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/1433326178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1dfed0 0x55d96a20b930 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1433326178 >> 172.21.15.97:0/1433326178 conn(0x55d96a0f8310 msgr2=0x55d96a1f77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1433326178 shutdown_connections 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.373+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/1433326178 wait complete. 2023-11-24T06:10:34.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.374+0000 7fcb1b2f9b80 1 -- start start 2023-11-24T06:10:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.374+0000 7fcb1b2f9b80 1 Processor -- start 2023-11-24T06:10:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1f2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 0x55d96a20cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1f72b0 0x55d96a212b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96a1f2880 con 0x55d96a1f72b0 2023-11-24T06:10:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d96a2123f0 con 0x55d96a1ca080 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d96a1f2cb0 con 0x55d96a1dfed0 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 10 client.0 _ll_register_callbacks cb 0x55d96a002a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1f72b0 0x55d96a212b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1f72b0 0x55d96a212b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48116/0 (socket says 172.21.15.97:48116) 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 -- 172.21.15.97:0/3683785637 learned_addr learned my addr 172.21.15.97:0/3683785637 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb07fff700 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 0x55d96a20cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 -- 172.21.15.97:0/3683785637 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 msgr2=0x55d96a20cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 0x55d96a20cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 -- 172.21.15.97:0/3683785637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 msgr2=0x55d96a1f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 -- 172.21.15.97:0/3683785637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d969f9a5e0 con 0x55d96a1f72b0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb0d69d700 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1f72b0 0x55d96a212b20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcb08158e30 tx=0x7fcb00004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0e1b65be1a940 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcae77fe700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcae77fe700 1 -- 172.21.15.97:0/3683785637 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0001c070 con 0x55d96a1f72b0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcae77fe700 1 -- 172.21.15.97:0/3683785637 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb00022ad0 con 0x55d96a1f72b0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/3683785637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcaf40158e0 con 0x55d96a1f72b0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcae77fe700 1 -- 172.21.15.97:0/3683785637 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb0000e040 con 0x55d96a1f72b0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 10 client.5673 fetch_fsmap learned FSMap version 26 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.375+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/3683785637 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55d969f6d200 con 0x55d96a1f72b0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcae77fe700 1 -- 172.21.15.97:0/3683785637 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf40158e0 con 0x55d96a1f72b0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcae77fe700 1 -- 172.21.15.97:0/3683785637 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb00022c90 con 0x55d96a1f72b0 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 10 client.5673 fetch_fsmap finished waiting for FSMap version 26 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 10 client.5673 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T06:10:34.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 20 client.5673 populate_metadata read hostname 'smithi097' 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] conn(0x55d96a2093e0 0x55d96a227830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 4 client.5673 mds_command: new command op to 5532 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/3683785637 --> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d96a0e3f80 con 0x55d96a2093e0 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] conn(0x55d96a2093e0 0x55d96a227830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcae77fe700 10 client.5673 ms_handle_connect on v2:172.21.15.97:6834/19331625 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb0ce9c700 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] conn(0x55d96a2093e0 0x55d96a227830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8abc9e359f7d75d1 server_cookie=4331720db9a96da in_seq=0 out_seq=0 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcae77fe700 1 -- 172.21.15.97:0/3683785637 <== mds.0 v2:172.21.15.97:6834/19331625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d96a0e3f80 con 0x55d96a2093e0 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcae77fe700 10 client.5673 handle_command_reply: tid=0 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 client.5673 shutdown 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/3683785637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1f72b0 msgr2=0x55d96a212b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1f72b0 0x55d96a212b20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcb08158e30 tx=0x7fcb00004690 comp rx=0 tx=0).stop 2023-11-24T06:10:34.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/3683785637 shutdown_connections 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d96a1f72b0 0x55d96a212b20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d96a1ca080 0x55d96a1f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.97:6834/19331625,v1:172.21.15.97:6835/19331625] conn(0x55d96a2093e0 0x55d96a227830 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 --2- 172.21.15.97:0/3683785637 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d96a1dfed0 0x55d96a20cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcae77fe700 0 client.5673 ms_handle_reset on v2:172.21.15.97:6834/19331625 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.376+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/3683785637 >> 172.21.15.97:0/3683785637 conn(0x55d96a0f8310 msgr2=0x55d96a1f76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/3683785637 shutdown_connections 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb1b2f9b80 1 -- 172.21.15.97:0/3683785637 wait complete. 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb1b2f9b80 20 client.5673 trim_cache size 0 max 16384 2023-11-24T06:10:34.386 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9", 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 5532, 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.8765049940000003, 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 45, 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 45, 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 12.127286717 2023-11-24T06:10:34.387 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:10:34.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 -- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaec06af60 msgr2=0x7fcaec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaec06af60 0x7fcaec06d420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb0806e440 tx=0x7fcaf4009040 comp rx=0 tx=0).stop 2023-11-24T06:10:34.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 -- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 msgr2=0x7fcb080cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcb080cac20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcafc00e4b0 tx=0x7fcafc0047c0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 -- 172.21.15.97:0/4187847356 shutdown_connections 2023-11-24T06:10:34.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaec06af60 0x7fcaec06d420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb080b4290 0x7fcb080b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf8003a00 0x7fcb080cac20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 --2- 172.21.15.97:0/4187847356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb08148750 0x7fcb080cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 -- 172.21.15.97:0/4187847356 >> 172.21.15.97:0/4187847356 conn(0x7fcb080b57b0 msgr2=0x7fcb0806e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 -- 172.21.15.97:0/4187847356 shutdown_connections 2023-11-24T06:10:34.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.377+0000 7fcb0f100700 1 -- 172.21.15.97:0/4187847356 wait complete. 2023-11-24T06:10:34.392 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:10:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.508+0000 7f372fd82700 1 Processor -- start 2023-11-24T06:10:34.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.508+0000 7f372fd82700 1 -- start start 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.509+0000 7f372fd82700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f37280adf00 0x7f372806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.509+0000 7f372fd82700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f372806c540 0x7f37280b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.509+0000 7f372fd82700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37280af950 con 0x7f37281446a0 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.509+0000 7f372fd82700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372806c9f0 con 0x7f372806c540 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.509+0000 7f372fd82700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3728144a80 con 0x7f37280adf00 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.509+0000 7f372e31f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f372806c540 0x7f37280b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.509+0000 7f372e31f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f372806c540 0x7f37280b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48124/0 (socket says 172.21.15.97:48124) 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.509+0000 7f372e31f700 1 -- 172.21.15.97:0/1509748086 learned_addr learned my addr 172.21.15.97:0/1509748086 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372d31d700 1 --2- 172.21.15.97:0/1509748086 >> v2:172.21.15.183:3300/0 conn(0x7f37280adf00 0x7f372806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372e31f700 1 -- 172.21.15.97:0/1509748086 >> v2:172.21.15.183:3300/0 conn(0x7f37280adf00 msgr2=0x7f372806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372e31f700 1 --2- 172.21.15.97:0/1509748086 >> v2:172.21.15.183:3300/0 conn(0x7f37280adf00 0x7f372806c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372e31f700 1 -- 172.21.15.97:0/1509748086 >> v1:172.21.15.183:6789/0 conn(0x7f37281446a0 legacy=0x7f37280b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372e31f700 1 -- 172.21.15.97:0/1509748086 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37280afc30 con 0x7f372806c540 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372d31d700 1 --2- 172.21.15.97:0/1509748086 >> v2:172.21.15.183:3300/0 conn(0x7f37280adf00 0x7f372806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372e31f700 1 --2- 172.21.15.97:0/1509748086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f372806c540 0x7f37280b3d40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f372809f030 tx=0x7f372000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c826facdcdb2416 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372cb1c700 1 -- 172.21.15.97:0/1509748086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720010070 con 0x7f372806c540 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372cb1c700 1 -- 172.21.15.97:0/1509748086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372000dd40 con 0x7f372806c540 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.510+0000 7f372cb1c700 1 -- 172.21.15.97:0/1509748086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372000b310 con 0x7f372806c540 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.511+0000 7f372fd82700 1 -- 172.21.15.97:0/1509748086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f372806c540 msgr2=0x7f37280b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.511+0000 7f372fd82700 1 --2- 172.21.15.97:0/1509748086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f372806c540 0x7f37280b3d40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f372809f030 tx=0x7f372000d460 comp rx=0 tx=0).stop 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.511+0000 7f372fd82700 1 -- 172.21.15.97:0/1509748086 shutdown_connections 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.511+0000 7f372fd82700 1 --2- 172.21.15.97:0/1509748086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f372806c540 0x7f37280b3d40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.511+0000 7f372fd82700 1 --2- 172.21.15.97:0/1509748086 >> v2:172.21.15.183:3300/0 conn(0x7f37280adf00 0x7f372806c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.511+0000 7f372fd82700 1 -- 172.21.15.97:0/1509748086 >> 172.21.15.97:0/1509748086 conn(0x7f3728071b90 msgr2=0x7f3728071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.511+0000 7f372fd82700 1 -- 172.21.15.97:0/1509748086 shutdown_connections 2023-11-24T06:10:34.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.511+0000 7f372fd82700 1 -- 172.21.15.97:0/1509748086 wait complete. 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.512+0000 7f372fd82700 1 Processor -- start 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.512+0000 7f372fd82700 1 -- start start 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372fd82700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f372806c540 0x7f372815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372fd82700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280adf00 0x7f3728154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372fd82700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281446a0 0x7f37280b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372fd82700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372813dfc0 con 0x7f37280adf00 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372fd82700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372815e210 con 0x7f37281446a0 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372fd82700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372813b4c0 con 0x7f372806c540 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280adf00 0x7f3728154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280adf00 0x7f3728154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48128/0 (socket says 172.21.15.97:48128) 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 -- 172.21.15.97:0/1653404891 learned_addr learned my addr 172.21.15.97:0/1653404891 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:34.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372db1e700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f372806c540 0x7f372815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372e31f700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281446a0 0x7f37280b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 -- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f372806c540 msgr2=0x7f372815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f372806c540 0x7f372815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 -- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281446a0 msgr2=0x7f37280b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281446a0 0x7f37280b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 -- 172.21.15.97:0/1653404891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372000bd40 con 0x7f37280adf00 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372e31f700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281446a0 0x7f37280b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372d31d700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280adf00 0x7f3728154f30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f372400c740 tx=0x7f372400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f371e7fc700 1 -- 172.21.15.97:0/1653404891 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724004220 con 0x7f37280adf00 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f371e7fc700 1 -- 172.21.15.97:0/1653404891 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3724004380 con 0x7f37280adf00 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f371e7fc700 1 -- 172.21.15.97:0/1653404891 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37240036a0 con 0x7f37280adf00 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372813d620 con 0x7f37280adf00 2023-11-24T06:10:34.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.513+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372813dbe0 con 0x7f37280adf00 2023-11-24T06:10:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.514+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37280753f0 con 0x7f37280adf00 2023-11-24T06:10:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.515+0000 7f371e7fc700 1 -- 172.21.15.97:0/1653404891 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3724003800 con 0x7f37280adf00 2023-11-24T06:10:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.515+0000 7f371e7fc700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370c06af60 0x7f370c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.515+0000 7f371e7fc700 1 -- 172.21.15.97:0/1653404891 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37240899a0 con 0x7f37280adf00 2023-11-24T06:10:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.515+0000 7f372db1e700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370c06af60 0x7f370c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:34.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.516+0000 7f372db1e700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370c06af60 0x7f370c06d420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3718009960 tx=0x7f37180090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:34.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.519+0000 7f371e7fc700 1 -- 172.21.15.97:0/1653404891 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3724054950 con 0x7f37280adf00 2023-11-24T06:10:34.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.757+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f372815a890 con 0x7f37280adf00 2023-11-24T06:10:34.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.757+0000 7f371e7fc700 1 -- 172.21.15.97:0/1653404891 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3724057f70 con 0x7f37280adf00 2023-11-24T06:10:34.760 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:10:34.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370c06af60 msgr2=0x7f370c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370c06af60 0x7f370c06d420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3718009960 tx=0x7f37180090e0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280adf00 msgr2=0x7f3728154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280adf00 0x7f3728154f30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f372400c740 tx=0x7f372400ca50 comp rx=0 tx=0).stop 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 shutdown_connections 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281446a0 0x7f37280b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370c06af60 0x7f370c06d420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f372806c540 0x7f372815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 --2- 172.21.15.97:0/1653404891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280adf00 0x7f3728154f30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 >> 172.21.15.97:0/1653404891 conn(0x7f3728071b90 msgr2=0x7f37280b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 shutdown_connections 2023-11-24T06:10:34.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:34.759+0000 7f372fd82700 1 -- 172.21.15.97:0/1653404891 wait complete. 2023-11-24T06:10:34.774 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:10:34.774 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:10:34.774 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-24T06:10:34.774 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:10:34.774 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:34.774 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:10:34.775 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:10:34.775 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:10:34.775 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:10:34.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:34.775 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-24T06:10:34.793 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:34.793 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:34.793 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:34.793 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:34.793 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:34.793 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:34.793 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:10:34.794 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42102sec preferred_lft 42102sec 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c9d4:801c:12b3:b866/64 scope link noprefixroute 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::7c25:b1ff:fe16:36f1/64 scope link 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:34.795 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:34.796 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether a2:28:0d:2d:82:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:10:34.796 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a028:dff:fe2d:822b/64 scope link 2023-11-24T06:10:34.796 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:34.796 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:34.796 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:10:34.796 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::1875:dff:fec0:4af7/64 scope link 2023-11-24T06:10:34.796 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:34.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:34.796 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:10:34.858 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-24T06:10:34.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:34.860 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-24T06:10:34.918 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-24T06:10:34.918 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:10:34.918 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:10:34.918 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:10:34.919 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:10:34.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:34.919 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:10:34.919 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:10:34.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:10:34.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:10:34.919 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:10:34.919 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:10:35.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:35.023 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:10:35.040 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:10:35.040 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:10:35.040 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:10:35.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:35.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-24T06:10:35.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:35.108 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:35.165 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:10:35.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:35.165 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:35.221 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:10:35.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:35.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:35.306 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:35.307 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:35.333 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:10:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:10:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:10:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:10:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:10:35.334 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:10:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:35.382 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:10:35.383 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:10:35.384 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:10:35.387 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:10:35.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:35.387 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:35.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:35.416 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:10:35.496 INFO:teuthology.orchestra.run.smithi097.stdout:client5682 2023-11-24T06:10:35.497 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:35.497 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:35.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:10:35.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.711+0000 7fa0f6a4e700 1 Processor -- start 2023-11-24T06:10:35.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.711+0000 7fa0f6a4e700 1 -- start start 2023-11-24T06:10:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.712+0000 7fa0f6a4e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa0f00d2240 0x7fa0f00b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.712+0000 7fa0f6a4e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f00b1d40 con 0x7fa0f015ddd0 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.712+0000 7fa0f6a4e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f00b28c0 con 0x7fa0f015e1b0 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.712+0000 7fa0f6a4e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f00b1f90 con 0x7fa0f00d2240 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.712+0000 7fa0effff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa0f015ddd0 0x7fa0f00d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40590/0 (socket says 172.21.15.97:40590) 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.712+0000 7fa0effff700 1 -- 172.21.15.97:0/99135609 learned_addr learned my addr 172.21.15.97:0/99135609 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.712+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1241839688 0 0) 0x7fa0f00b1d40 con 0x7fa0f015ddd0 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0ef7fe700 1 --2- 172.21.15.97:0/99135609 >> v2:172.21.15.183:3300/0 conn(0x7fa0f00d2240 0x7fa0f00b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d80035b0 con 0x7fa0f015ddd0 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520322021 0 0) 0x7fa0d80035b0 con 0x7fa0f015ddd0 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 >> v2:172.21.15.183:3300/0 conn(0x7fa0f00d2240 msgr2=0x7fa0f00b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:35.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 --2- 172.21.15.97:0/99135609 >> v2:172.21.15.183:3300/0 conn(0x7fa0f00d2240 0x7fa0f00b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 >> v1:172.21.15.183:6789/0 conn(0x7fa0f015e1b0 legacy=0x7fa0f006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00b2270 con 0x7fa0f015ddd0 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fa0e4001670 con 0x7fa0f015ddd0 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 >> v1:172.21.15.97:6789/0 conn(0x7fa0f015ddd0 legacy=0x7fa0f00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d8003a30 0x7fa0d8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8004850 0x7fa0d8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0d8005330 0x7fa0d800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00b1f90 con 0x7fa0d8003a30 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d800c230 con 0x7fa0d8005330 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d800c3f0 con 0x7fa0d8004850 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0effff700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d8003a30 0x7fa0d8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0ef7fe700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8004850 0x7fa0d8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0f4feb700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0d8005330 0x7fa0d800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0effff700 1 -- 172.21.15.97:0/99135609 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8004850 msgr2=0x7fa0d8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0effff700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8004850 0x7fa0d8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0effff700 1 -- 172.21.15.97:0/99135609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0d8005330 msgr2=0x7fa0d800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0effff700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0d8005330 0x7fa0d800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0f4feb700 1 -- 172.21.15.97:0/99135609 reap_dead start 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0effff700 1 -- 172.21.15.97:0/99135609 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d80035b0 con 0x7fa0d8003a30 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0effff700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d8003a30 0x7fa0d8003e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c210 tx=0x7fa0d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a72c49adfcb132 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4007c10 con 0x7fa0d8003a30 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.713+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e400cdb0 con 0x7fa0d8003a30 2023-11-24T06:10:35.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.714+0000 7fa0eeffd700 1 -- 172.21.15.97:0/99135609 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4005ba0 con 0x7fa0d8003a30 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.714+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/99135609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d8003a30 msgr2=0x7fa0d8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.714+0000 7fa0f6a4e700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d8003a30 0x7fa0d8003e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c210 tx=0x7fa0d8005880 comp rx=0 tx=0).stop 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.714+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/99135609 shutdown_connections 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.714+0000 7fa0f6a4e700 1 --2- 172.21.15.97:0/99135609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d8003a30 0x7fa0d8003e50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.714+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/99135609 >> 172.21.15.97:0/99135609 conn(0x7fa0f00731a0 msgr2=0x7fa0f009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.714+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/99135609 shutdown_connections 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.714+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/99135609 wait complete. 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.715+0000 7fa0f6a4e700 1 Processor -- start 2023-11-24T06:10:35.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.715+0000 7fa0f6a4e700 1 -- start start 2023-11-24T06:10:35.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f6a4e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8003a30 0x7fa0f01f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f6a4e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f0150050 0x7fa0f00b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f6a4e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0f009ecb0 0x7fa0f009f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f6a4e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00b6a40 con 0x7fa0f009ecb0 2023-11-24T06:10:35.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f6a4e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f009f6a0 con 0x7fa0f0150050 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f6a4e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01f7720 con 0x7fa0d8003a30 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f4feb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0f009ecb0 0x7fa0f009f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f4feb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0f009ecb0 0x7fa0f009f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48154/0 (socket says 172.21.15.97:48154) 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.716+0000 7fa0f4feb700 1 -- 172.21.15.97:0/2723514337 learned_addr learned my addr 172.21.15.97:0/2723514337 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0effff700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8003a30 0x7fa0f01f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0ef7fe700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f0150050 0x7fa0f00b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0f4feb700 1 -- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8003a30 msgr2=0x7fa0f01f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0f4feb700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8003a30 0x7fa0f01f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0f4feb700 1 -- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f0150050 msgr2=0x7fa0f00b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0f4feb700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f0150050 0x7fa0f00b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0f4feb700 1 -- 172.21.15.97:0/2723514337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00b2ed0 con 0x7fa0f009ecb0 2023-11-24T06:10:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0ef7fe700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f0150050 0x7fa0f00b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0f4feb700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0f009ecb0 0x7fa0f009f160 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0073130 tx=0x7fa0e800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0ecff9700 1 -- 172.21.15.97:0/2723514337 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8003c10 con 0x7fa0f009ecb0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0ecff9700 1 -- 172.21.15.97:0/2723514337 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e8003d90 con 0x7fa0f009ecb0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f01f7a00 con 0x7fa0f009ecb0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0ecff9700 1 -- 172.21.15.97:0/2723514337 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8005310 con 0x7fa0f009ecb0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.717+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f014db00 con 0x7fa0f009ecb0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.718+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f0067f60 con 0x7fa0f009ecb0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.718+0000 7fa0ecff9700 1 -- 172.21.15.97:0/2723514337 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa0e8005470 con 0x7fa0f009ecb0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.719+0000 7fa0ecff9700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa0dc06af60 0x7fa0dc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.719+0000 7fa0ecff9700 1 -- 172.21.15.97:0/2723514337 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0e8089730 con 0x7fa0f009ecb0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.719+0000 7fa0effff700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa0dc06af60 0x7fa0dc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:35.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.719+0000 7fa0effff700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa0dc06af60 0x7fa0dc06d420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4000c00 tx=0x7fa0e4007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:35.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.723+0000 7fa0ecff9700 1 -- 172.21.15.97:0/2723514337 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0e80546f0 con 0x7fa0f009ecb0 2023-11-24T06:10:35.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.957+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa0f00b1650 con 0x7fa0f009ecb0 2023-11-24T06:10:35.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.957+0000 7fa0ecff9700 1 -- 172.21.15.97:0/2723514337 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa0e801a310 con 0x7fa0f009ecb0 2023-11-24T06:10:35.960 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:10:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa0dc06af60 msgr2=0x7fa0dc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa0dc06af60 0x7fa0dc06d420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4000c00 tx=0x7fa0e4007000 comp rx=0 tx=0).stop 2023-11-24T06:10:35.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0f009ecb0 msgr2=0x7fa0f009f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0f009ecb0 0x7fa0f009f160 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0073130 tx=0x7fa0e800ee00 comp rx=0 tx=0).stop 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 shutdown_connections 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0f009ecb0 0x7fa0f009f160 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa0dc06af60 0x7fa0dc06d420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0d8003a30 0x7fa0f01f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 --2- 172.21.15.97:0/2723514337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa0f0150050 0x7fa0f00b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 >> 172.21.15.97:0/2723514337 conn(0x7fa0f00731a0 msgr2=0x7fa0f00aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 shutdown_connections 2023-11-24T06:10:35.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:35.959+0000 7fa0f6a4e700 1 -- 172.21.15.97:0/2723514337 wait complete. 2023-11-24T06:10:35.973 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:10:35.973 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:10:35.973 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-11-24T06:10:35.974 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:10:35.974 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:35.974 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:10:35.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:10:35.974 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:10:35.974 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:10:35.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:35.974 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-11-24T06:10:35.995 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:10:35.995 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:10:35.995 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:35.996 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42093sec preferred_lft 42093sec 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::5382:2bb9:1085:5950/64 scope link noprefixroute 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::b8cd:8aff:fe67:b24d/64 scope link 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether fe:4f:a7:e6:1c:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::fc4f:a7ff:fee6:1c7f/64 scope link 2023-11-24T06:10:35.997 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:35.998 INFO:teuthology.orchestra.run.smithi183.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:10:35.998 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:10:35.998 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::4034:29ff:fe35:b2a5/64 scope link 2023-11-24T06:10:35.998 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:10:35.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:35.998 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:10:36.065 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-11-24T06:10:36.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:36.067 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-11-24T06:10:36.128 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-11-24T06:10:36.128 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:10:36.128 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:10:36.128 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:10:36.128 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:10:36.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:36.128 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:10:36.129 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:10:36.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:10:36.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:10:36.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:10:36.129 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:10:36.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:36.238 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:10:36.255 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:10:36.255 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:10:36.255 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:10:36.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:36.256 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2023-11-24T06:10:36.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:36.322 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:36.379 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:10:36.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:10:36.379 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:36.435 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:10:36.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:36.435 DEBUG:teuthology.orchestra.run.smithi183:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:36.520 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:36.520 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:36.544 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:36.544 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:36.544 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:10:36.544 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:10:36.545 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:10:36.545 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:10:36.545 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:10:36.545 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:10:36.613 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:10:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:10:36.616 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:10:36.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:36.616 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:36.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:36.642 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:10:36.714 INFO:teuthology.orchestra.run.smithi183.stdout:client5356 2023-11-24T06:10:36.715 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:36.715 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:36.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:10:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.894+0000 7fb694640700 1 Processor -- start 2023-11-24T06:10:36.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.894+0000 7fb694640700 1 -- start start 2023-11-24T06:10:36.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb694640700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb68c0746a0 0x7fb68c06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb694640700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c0b5e50 con 0x7fb68c0b01e0 2023-11-24T06:10:36.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb694640700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c0b05c0 con 0x7fb68c0b59c0 2023-11-24T06:10:36.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb694640700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68c1470a0 con 0x7fb68c0746a0 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb691bdb700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb68c0746a0 0x7fb68c06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb6923dc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb68c0b01e0 0x7fb68c073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40610/0 (socket says 172.21.15.97:40610) 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb6923dc700 1 -- 172.21.15.97:0/2055170761 learned_addr learned my addr 172.21.15.97:0/2055170761 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb6913da700 1 -- 172.21.15.97:0/2055170761 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870989064 0 0) 0x7fb688001110 con 0x7fb68c0b01e0 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.895+0000 7fb6913da700 1 -- 172.21.15.97:0/2055170761 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6780035b0 con 0x7fb68c0b01e0 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb691bdb700 1 -- 172.21.15.97:0/2055170761 >> v1:172.21.15.183:6789/0 conn(0x7fb68c0b59c0 legacy=0x7fb68c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb691bdb700 1 -- 172.21.15.97:0/2055170761 >> v1:172.21.15.97:6789/0 conn(0x7fb68c0b01e0 legacy=0x7fb68c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb691bdb700 1 -- 172.21.15.97:0/2055170761 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c0b4250 con 0x7fb68c0746a0 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb691bdb700 1 --2- 172.21.15.97:0/2055170761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0746a0 0x7fb68c06e9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb680009840 tx=0x7fb680009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7767482bddcb2371 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb6913da700 1 -- 172.21.15.97:0/2055170761 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680011830 con 0x7fb68c0746a0 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb6913da700 1 -- 172.21.15.97:0/2055170761 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb680011990 con 0x7fb68c0746a0 2023-11-24T06:10:36.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb6913da700 1 -- 172.21.15.97:0/2055170761 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680011d20 con 0x7fb68c0746a0 2023-11-24T06:10:36.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb694640700 1 -- 172.21.15.97:0/2055170761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0746a0 msgr2=0x7fb68c06e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb694640700 1 --2- 172.21.15.97:0/2055170761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0746a0 0x7fb68c06e9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb680009840 tx=0x7fb680009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:36.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb694640700 1 -- 172.21.15.97:0/2055170761 shutdown_connections 2023-11-24T06:10:36.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb694640700 1 --2- 172.21.15.97:0/2055170761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0746a0 0x7fb68c06e9c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.896+0000 7fb694640700 1 -- 172.21.15.97:0/2055170761 >> 172.21.15.97:0/2055170761 conn(0x7fb68c0d0b40 msgr2=0x7fb68c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:36.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.897+0000 7fb694640700 1 -- 172.21.15.97:0/2055170761 shutdown_connections 2023-11-24T06:10:36.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.897+0000 7fb694640700 1 -- 172.21.15.97:0/2055170761 wait complete. 2023-11-24T06:10:36.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.898+0000 7fb694640700 1 Processor -- start 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.898+0000 7fb694640700 1 -- start start 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.898+0000 7fb694640700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb68c0746a0 0x7fb68c160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.898+0000 7fb694640700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c0b01e0 0x7fb68c142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.898+0000 7fb694640700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0b59c0 0x7fb68c1e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.898+0000 7fb694640700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c142aa0 con 0x7fb68c0b59c0 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.898+0000 7fb694640700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c1e89e0 con 0x7fb68c0b01e0 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.898+0000 7fb694640700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c1e8140 con 0x7fb68c0746a0 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0b59c0 0x7fb68c1e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0b59c0 0x7fb68c1e78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48168/0 (socket says 172.21.15.97:48168) 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 -- 172.21.15.97:0/972256300 learned_addr learned my addr 172.21.15.97:0/972256300 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:36.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb6923dc700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb68c0746a0 0x7fb68c160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb691bdb700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c0b01e0 0x7fb68c142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 -- 172.21.15.97:0/972256300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb68c0746a0 msgr2=0x7fb68c160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb68c0746a0 0x7fb68c160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 -- 172.21.15.97:0/972256300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c0b01e0 msgr2=0x7fb68c142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c0b01e0 0x7fb68c142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 -- 172.21.15.97:0/972256300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800094a0 con 0x7fb68c0b59c0 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb692bdd700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0b59c0 0x7fb68c1e78f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb68400cb50 tx=0x7fb68400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb67effd700 1 -- 172.21.15.97:0/972256300 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684003c00 con 0x7fb68c0b59c0 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb67effd700 1 -- 172.21.15.97:0/972256300 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb684003d60 con 0x7fb68c0b59c0 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c1e8420 con 0x7fb68c0b59c0 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb67effd700 1 -- 172.21.15.97:0/972256300 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68400aa20 con 0x7fb68c0b59c0 2023-11-24T06:10:36.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.899+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c1e9130 con 0x7fb68c0b59c0 2023-11-24T06:10:36.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.900+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c067f60 con 0x7fb68c0b59c0 2023-11-24T06:10:36.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.901+0000 7fb67effd700 1 -- 172.21.15.97:0/972256300 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb68400ab80 con 0x7fb68c0b59c0 2023-11-24T06:10:36.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.901+0000 7fb67effd700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb67006af60 0x7fb67006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:36.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.901+0000 7fb67effd700 1 -- 172.21.15.97:0/972256300 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb684089580 con 0x7fb68c0b59c0 2023-11-24T06:10:36.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.901+0000 7fb6923dc700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb67006af60 0x7fb67006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:36.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.901+0000 7fb6923dc700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb67006af60 0x7fb67006d420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb688007fd0 tx=0x7fb688007c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:36.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:36.906+0000 7fb67effd700 1 -- 172.21.15.97:0/972256300 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6840546d0 con 0x7fb68c0b59c0 2023-11-24T06:10:37.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.144+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb68c160cf0 con 0x7fb68c0b59c0 2023-11-24T06:10:37.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.551+0000 7fb67effd700 1 -- 172.21.15.97:0/972256300 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb684057cf0 con 0x7fb68c0b59c0 2023-11-24T06:10:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb67006af60 msgr2=0x7fb67006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb67006af60 0x7fb67006d420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb688007fd0 tx=0x7fb688007c40 comp rx=0 tx=0).stop 2023-11-24T06:10:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0b59c0 msgr2=0x7fb68c1e78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0b59c0 0x7fb68c1e78f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb68400cb50 tx=0x7fb68400ce60 comp rx=0 tx=0).stop 2023-11-24T06:10:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 shutdown_connections 2023-11-24T06:10:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb68c0b59c0 0x7fb68c1e78f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb67006af60 0x7fb67006d420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb68c0746a0 0x7fb68c160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 --2- 172.21.15.97:0/972256300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c0b01e0 0x7fb68c142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 >> 172.21.15.97:0/972256300 conn(0x7fb68c0d0b40 msgr2=0x7fb68c06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:37.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 shutdown_connections 2023-11-24T06:10:37.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.553+0000 7fb694640700 1 -- 172.21.15.97:0/972256300 wait complete. 2023-11-24T06:10:37.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:10:37.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.690+0000 7f8a1d6e8700 1 Processor -- start 2023-11-24T06:10:37.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.690+0000 7f8a1d6e8700 1 -- start start 2023-11-24T06:10:37.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a1d6e8700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8a180b3370 0x7f8a181604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a1d6e8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 0x7f8a180d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a1d6e8700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a180b22d0 con 0x7f8a1815bdd0 2023-11-24T06:10:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a1d6e8700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a180b2f50 con 0x7f8a180b3370 2023-11-24T06:10:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a1d6e8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180d2640 con 0x7f8a180d1c70 2023-11-24T06:10:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a16ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8a1815bdd0 0x7f8a180b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40624/0 (socket says 172.21.15.97:40624) 2023-11-24T06:10:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a16ffd700 1 -- 172.21.15.97:0/4286874795 learned_addr learned my addr 172.21.15.97:0/4286874795 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:37.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a177fe700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 0x7f8a180d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a167fc700 1 --2- 172.21.15.97:0/4286874795 >> v2:172.21.15.183:3300/0 conn(0x7f8a180b3370 0x7f8a181604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264484658 0 0) 0x7f8a180b22d0 con 0x7f8a1815bdd0 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a000035b0 con 0x7f8a1815bdd0 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.691+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570293414 0 0) 0x7f8a000035b0 con 0x7f8a1815bdd0 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 msgr2=0x7f8a180d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 0x7f8a180d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 >> v2:172.21.15.183:3300/0 conn(0x7f8a180b3370 msgr2=0x7f8a181604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 --2- 172.21.15.97:0/4286874795 >> v2:172.21.15.183:3300/0 conn(0x7f8a180b3370 0x7f8a181604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a180b25b0 con 0x7f8a1815bdd0 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f8a0c001670 con 0x7f8a1815bdd0 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 >> v1:172.21.15.97:6789/0 conn(0x7f8a1815bdd0 legacy=0x7f8a180b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a00003a00 0x7f8a00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a00004850 0x7f8a00004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 0x7f8a00005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180b2f50 con 0x7f8a00004850 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180d2640 con 0x7f8a00003a00 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0000ba10 con 0x7f8a180d1c70 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a167fc700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a00004850 0x7f8a00004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a16ffd700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a00003a00 0x7f8a00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a177fe700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 0x7f8a00005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a167fc700 1 -- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 msgr2=0x7f8a00005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a167fc700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 0x7f8a00005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a167fc700 1 -- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a00003a00 msgr2=0x7f8a00003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a167fc700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a00003a00 0x7f8a00003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a167fc700 1 -- 172.21.15.97:0/4286874795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a000035b0 con 0x7f8a00004850 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a177fe700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a180d1c70 0x7f8a00005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a167fc700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a00004850 0x7f8a00004cd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009b60 tx=0x7f8a00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1abd916ec004e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800e6c0 con 0x7f8a00004850 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0800e820 con 0x7f8a00004850 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.692+0000 7f8a15ffb700 1 -- 172.21.15.97:0/4286874795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800eb90 con 0x7f8a00004850 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.693+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/4286874795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a00004850 msgr2=0x7f8a00004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.693+0000 7f8a1d6e8700 1 --2- 172.21.15.97:0/4286874795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a00004850 0x7f8a00004cd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009b60 tx=0x7f8a00005880 comp rx=0 tx=0).stop 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.693+0000 7f8a177fe700 1 -- 172.21.15.97:0/4286874795 reap_dead start 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.693+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/4286874795 shutdown_connections 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.693+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/4286874795 >> 172.21.15.97:0/4286874795 conn(0x7f8a180731a0 msgr2=0x7f8a1809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.693+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/4286874795 shutdown_connections 2023-11-24T06:10:37.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.693+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/4286874795 wait complete. 2023-11-24T06:10:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.695+0000 7f8a1d6e8700 1 Processor -- start 2023-11-24T06:10:37.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.695+0000 7f8a1d6e8700 1 -- start start 2023-11-24T06:10:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.695+0000 7f8a1d6e8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a00004850 0x7f8a18145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.695+0000 7f8a1d6e8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1815c4b0 0x7f8a181e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.695+0000 7f8a1d6e8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a181e6b00 0x7f8a181e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.695+0000 7f8a1d6e8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18146520 con 0x7f8a181e6b00 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a1d6e8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180cc8b0 con 0x7f8a1815c4b0 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a1d6e8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1815c950 con 0x7f8a00004850 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a181e6b00 0x7f8a181e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a181e6b00 0x7f8a181e6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48188/0 (socket says 172.21.15.97:48188) 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 -- 172.21.15.97:0/2950821795 learned_addr learned my addr 172.21.15.97:0/2950821795 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a167fc700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1815c4b0 0x7f8a181e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a16ffd700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a00004850 0x7f8a18145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 -- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a00004850 msgr2=0x7f8a18145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a00004850 0x7f8a18145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 -- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1815c4b0 msgr2=0x7f8a181e60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1815c4b0 0x7f8a181e60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 -- 172.21.15.97:0/2950821795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a180ccb00 con 0x7f8a181e6b00 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a16ffd700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a00004850 0x7f8a18145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a177fe700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a181e6b00 0x7f8a181e6f90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8a180cf770 tx=0x7f8a100104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2950821795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000e050 con 0x7f8a181e6b00 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2950821795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1000f040 con 0x7f8a181e6b00 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a180ccde0 con 0x7f8a181e6b00 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2950821795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10010e30 con 0x7f8a181e6b00 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.696+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a18159380 con 0x7f8a181e6b00 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.697+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a180ccf70 con 0x7f8a181e6b00 2023-11-24T06:10:37.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.698+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2950821795 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8a10015440 con 0x7f8a181e6b00 2023-11-24T06:10:37.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.698+0000 7f89ff7fe700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a0406af60 0x7f8a0406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:37.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.698+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2950821795 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a100898c0 con 0x7f8a181e6b00 2023-11-24T06:10:37.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.698+0000 7f8a16ffd700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a0406af60 0x7f8a0406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:37.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.698+0000 7f8a16ffd700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a0406af60 0x7f8a0406d420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c000fb0 tx=0x7f8a0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:37.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.703+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2950821795 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a10054900 con 0x7f8a181e6b00 2023-11-24T06:10:37.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.918+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a180b2b70 con 0x7f8a0406af60 2023-11-24T06:10:37.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.920+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2950821795 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8a0c007e20 con 0x7f8a0406af60 2023-11-24T06:10:37.922 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:10:37.922 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:10:37.922 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephfs" 2023-11-24T06:10:37.923 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:10:37.923 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:10:37.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a0406af60 msgr2=0x7f8a0406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a0406af60 0x7f8a0406d420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c000fb0 tx=0x7f8a0c009040 comp rx=0 tx=0).stop 2023-11-24T06:10:37.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a181e6b00 msgr2=0x7f8a181e6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:37.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a181e6b00 0x7f8a181e6f90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8a180cf770 tx=0x7f8a100104a0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 shutdown_connections 2023-11-24T06:10:37.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a181e6b00 0x7f8a181e6f90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a0406af60 0x7f8a0406d420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a00004850 0x7f8a18145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 --2- 172.21.15.97:0/2950821795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a1815c4b0 0x7f8a181e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:37.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 >> 172.21.15.97:0/2950821795 conn(0x7f8a180731a0 msgr2=0x7f8a180b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:37.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 shutdown_connections 2023-11-24T06:10:37.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:37.922+0000 7f8a1d6e8700 1 -- 172.21.15.97:0/2950821795 wait complete. 2023-11-24T06:10:37.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:10:38.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.056+0000 7fcbf8575700 1 Processor -- start 2023-11-24T06:10:38.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.056+0000 7fcbf8575700 1 -- start start 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.057+0000 7fcbf8575700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbf015bf50 0x7fcbf00ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.057+0000 7fcbf8575700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcbf00ce830 0x7fcbf00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.057+0000 7fcbf8575700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00d0140 con 0x7fcbf00cf1f0 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.057+0000 7fcbf8575700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf015c330 con 0x7fcbf015bf50 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.057+0000 7fcbf8575700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf00d03e0 con 0x7fcbf00ce830 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf6311700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcbf015bf50 0x7fcbf00ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf6b12700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcbf00cf1f0 0x7fcbf00c5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40648/0 (socket says 172.21.15.97:40648) 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf6b12700 1 -- 172.21.15.97:0/3474943643 learned_addr learned my addr 172.21.15.97:0/3474943643 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:38.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf5b10700 1 --2- 172.21.15.97:0/3474943643 >> v2:172.21.15.183:3300/0 conn(0x7fcbf00ce830 0x7fcbf00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf6311700 1 -- 172.21.15.97:0/3474943643 >> v2:172.21.15.183:3300/0 conn(0x7fcbf00ce830 msgr2=0x7fcbf00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf6311700 1 --2- 172.21.15.97:0/3474943643 >> v2:172.21.15.183:3300/0 conn(0x7fcbf00ce830 0x7fcbf00cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf6311700 1 -- 172.21.15.97:0/3474943643 >> v1:172.21.15.97:6789/0 conn(0x7fcbf00cf1f0 legacy=0x7fcbf00c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf6311700 1 -- 172.21.15.97:0/3474943643 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00d06c0 con 0x7fcbf015bf50 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf530f700 1 -- 172.21.15.97:0/3474943643 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643182100 0 0) 0x7fcbf00d0140 con 0x7fcbf00cf1f0 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf6311700 1 --2- 172.21.15.97:0/3474943643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf015bf50 0x7fcbf00ce2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0072420 tx=0x7fcbe0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ff12ebee6caffb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf530f700 1 -- 172.21.15.97:0/3474943643 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0011830 con 0x7fcbf015bf50 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf530f700 1 -- 172.21.15.97:0/3474943643 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe0011990 con 0x7fcbf015bf50 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.058+0000 7fcbf530f700 1 -- 172.21.15.97:0/3474943643 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0011d20 con 0x7fcbf015bf50 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.059+0000 7fcbf8575700 1 -- 172.21.15.97:0/3474943643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf015bf50 msgr2=0x7fcbf00ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.059+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3474943643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf015bf50 0x7fcbf00ce2f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0072420 tx=0x7fcbe0009b20 comp rx=0 tx=0).stop 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.059+0000 7fcbf8575700 1 -- 172.21.15.97:0/3474943643 shutdown_connections 2023-11-24T06:10:38.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.059+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3474943643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf015bf50 0x7fcbf00ce2f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.059+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3474943643 >> v2:172.21.15.183:3300/0 conn(0x7fcbf00ce830 0x7fcbf00cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.059+0000 7fcbf8575700 1 -- 172.21.15.97:0/3474943643 >> 172.21.15.97:0/3474943643 conn(0x7fcbf00b3780 msgr2=0x7fcbf00b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.059+0000 7fcbf8575700 1 -- 172.21.15.97:0/3474943643 shutdown_connections 2023-11-24T06:10:38.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.059+0000 7fcbf8575700 1 -- 172.21.15.97:0/3474943643 wait complete. 2023-11-24T06:10:38.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.060+0000 7fcbf8575700 1 Processor -- start 2023-11-24T06:10:38.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.060+0000 7fcbf8575700 1 -- start start 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf8575700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00ce830 0x7fcbf014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf8575700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbf00cf1f0 0x7fcbf006b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf8575700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbf015bf50 0x7fcbf01e5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf8575700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01e5b80 con 0x7fcbf00ce830 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf8575700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006c4d0 con 0x7fcbf00cf1f0 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf8575700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf006b8c0 con 0x7fcbf015bf50 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00ce830 0x7fcbf014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00ce830 0x7fcbf014c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48198/0 (socket says 172.21.15.97:48198) 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 -- 172.21.15.97:0/3054890626 learned_addr learned my addr 172.21.15.97:0/3054890626 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 -- 172.21.15.97:0/3054890626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbf015bf50 msgr2=0x7fcbf01e5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbf015bf50 0x7fcbf01e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf5b10700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbf00cf1f0 0x7fcbf006b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 -- 172.21.15.97:0/3054890626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbf00cf1f0 msgr2=0x7fcbf006b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbf00cf1f0 0x7fcbf006b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 -- 172.21.15.97:0/3054890626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe00094a0 con 0x7fcbf00ce830 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf6311700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00ce830 0x7fcbf014c0f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009b60 tx=0x7fcbe0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbe6ffd700 1 -- 172.21.15.97:0/3054890626 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00038a0 con 0x7fcbf00ce830 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbe6ffd700 1 -- 172.21.15.97:0/3054890626 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe001e070 con 0x7fcbf00ce830 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.061+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf006bba0 con 0x7fcbf00ce830 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.062+0000 7fcbe6ffd700 1 -- 172.21.15.97:0/3054890626 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0003c00 con 0x7fcbf00ce830 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.062+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf006c160 con 0x7fcbf00ce830 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.062+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf0068fb0 con 0x7fcbf00ce830 2023-11-24T06:10:38.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.063+0000 7fcbe6ffd700 1 -- 172.21.15.97:0/3054890626 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcbe002d450 con 0x7fcbf00ce830 2023-11-24T06:10:38.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.063+0000 7fcbe6ffd700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcbd406af60 0x7fcbd406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.063+0000 7fcbe6ffd700 1 -- 172.21.15.97:0/3054890626 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbe008a630 con 0x7fcbf00ce830 2023-11-24T06:10:38.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.063+0000 7fcbf5b10700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcbd406af60 0x7fcbd406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.064+0000 7fcbf5b10700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcbd406af60 0x7fcbd406d420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcbf006d1d0 tx=0x7fcbec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.068+0000 7fcbe6ffd700 1 -- 172.21.15.97:0/3054890626 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcbe0055780 con 0x7fcbf00ce830 2023-11-24T06:10:38.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.260+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcbf00b7ab0 con 0x7fcbf00ce830 2023-11-24T06:10:38.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.260+0000 7fcbe6ffd700 1 -- 172.21.15.97:0/3054890626 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcbe0058da0 con 0x7fcbf00ce830 2023-11-24T06:10:38.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcbd406af60 msgr2=0x7fcbd406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcbd406af60 0x7fcbd406d420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcbf006d1d0 tx=0x7fcbec008040 comp rx=0 tx=0).stop 2023-11-24T06:10:38.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00ce830 msgr2=0x7fcbf014c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00ce830 0x7fcbf014c0f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009b60 tx=0x7fcbe0004750 comp rx=0 tx=0).stop 2023-11-24T06:10:38.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 shutdown_connections 2023-11-24T06:10:38.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcbd406af60 0x7fcbd406d420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcbf015bf50 0x7fcbf01e5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcbf00ce830 0x7fcbf014c0f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 --2- 172.21.15.97:0/3054890626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcbf00cf1f0 0x7fcbf006b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 >> 172.21.15.97:0/3054890626 conn(0x7fcbf00b3780 msgr2=0x7fcbf0070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 shutdown_connections 2023-11-24T06:10:38.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.262+0000 7fcbf8575700 1 -- 172.21.15.97:0/3054890626 wait complete. 2023-11-24T06:10:38.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp4467 2023-11-24T06:10:38.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.399+0000 7f32d5979700 1 Processor -- start 2023-11-24T06:10:38.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.399+0000 7f32d5979700 1 -- start start 2023-11-24T06:10:38.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.399+0000 7f32d5979700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32d006e930 0x7f32d00b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.399+0000 7f32d5979700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00b2a60 con 0x7f32d015ac40 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.399+0000 7f32d5979700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d015e970 con 0x7f32d00d05b0 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.399+0000 7f32d5979700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015ef30 con 0x7f32d006e930 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32ceffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f32d015ac40 0x7f32d00cfea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40658/0 (socket says 172.21.15.97:40658) 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32ceffd700 1 -- 172.21.15.97:0/4168233213 learned_addr learned my addr 172.21.15.97:0/4168233213 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cf7fe700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32d006e930 0x7f32d00b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550620793 0 0) 0x7f32d00b2a60 con 0x7f32d015ac40 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b80035b0 con 0x7f32d015ac40 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 490489178 0 0) 0x7f32b80035b0 con 0x7f32d015ac40 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32d006e930 msgr2=0x7f32d00b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32d006e930 0x7f32d00b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 >> v1:172.21.15.183:6789/0 conn(0x7f32d00d05b0 legacy=0x7f32d00ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00b2cb0 con 0x7f32d015ac40 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f32c0001670 con 0x7f32d015ac40 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 >> v1:172.21.15.97:6789/0 conn(0x7f32d015ac40 legacy=0x7f32d00cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32b8003a30 0x7f32b8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b8004850 0x7f32b8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32d006e930 0x7f32b8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015ef30 con 0x7f32b8004850 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b800bb70 con 0x7f32b8003a30 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b800df40 con 0x7f32d006e930 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32ce7fc700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b8004850 0x7f32b8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32ceffd700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32b8003a30 0x7f32b8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.400+0000 7f32cf7fe700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32d006e930 0x7f32b8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32ce7fc700 1 -- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32d006e930 msgr2=0x7f32b8005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32ce7fc700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32d006e930 0x7f32b8005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32ce7fc700 1 -- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32b8003a30 msgr2=0x7f32b8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32ce7fc700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32b8003a30 0x7f32b8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32ce7fc700 1 -- 172.21.15.97:0/4168233213 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80035b0 con 0x7f32b8004850 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32ce7fc700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b8004850 0x7f32b8004cd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f32c4001800 tx=0x7f32b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55aac3252539b62a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c40012f0 con 0x7f32b8004850 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c40052f0 con 0x7f32b8004850 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32cdffb700 1 -- 172.21.15.97:0/4168233213 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4006860 con 0x7f32b8004850 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32d5979700 1 -- 172.21.15.97:0/4168233213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b8004850 msgr2=0x7f32b8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32d5979700 1 --2- 172.21.15.97:0/4168233213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32b8004850 0x7f32b8004cd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f32c4001800 tx=0x7f32b8005880 comp rx=0 tx=0).stop 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32cf7fe700 1 -- 172.21.15.97:0/4168233213 reap_dead start 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32d5979700 1 -- 172.21.15.97:0/4168233213 shutdown_connections 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32d5979700 1 -- 172.21.15.97:0/4168233213 >> 172.21.15.97:0/4168233213 conn(0x7f32d006b410 msgr2=0x7f32d006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32d5979700 1 -- 172.21.15.97:0/4168233213 shutdown_connections 2023-11-24T06:10:38.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.401+0000 7f32d5979700 1 -- 172.21.15.97:0/4168233213 wait complete. 2023-11-24T06:10:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.403+0000 7f32d5979700 1 Processor -- start 2023-11-24T06:10:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.403+0000 7f32d5979700 1 -- start start 2023-11-24T06:10:38.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.403+0000 7f32d5979700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32b8004850 0x7f32d014cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.403+0000 7f32d5979700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d0158ad0 0x7f32d01f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.403+0000 7f32d5979700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d015c880 0x7f32d015cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.403+0000 7f32d5979700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d0150f80 con 0x7f32d015c880 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.403+0000 7f32d5979700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015d270 con 0x7f32d0158ad0 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.403+0000 7f32d5979700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01f2ef0 con 0x7f32b8004850 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d015c880 0x7f32d015cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d015c880 0x7f32d015cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48222/0 (socket says 172.21.15.97:48222) 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 -- 172.21.15.97:0/4016830104 learned_addr learned my addr 172.21.15.97:0/4016830104 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32ceffd700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32b8004850 0x7f32d014cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 -- 172.21.15.97:0/4016830104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32b8004850 msgr2=0x7f32d014cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32b8004850 0x7f32d014cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 -- 172.21.15.97:0/4016830104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d0158ad0 msgr2=0x7f32d01f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d0158ad0 0x7f32d01f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 -- 172.21.15.97:0/4016830104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d01f4000 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32cf7fe700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d015c880 0x7f32d015cd30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f32b80047d0 tx=0x7f32c800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32b7fff700 1 -- 172.21.15.97:0/4016830104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c800e040 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32b7fff700 1 -- 172.21.15.97:0/4016830104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c800f040 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d014fd60 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32b7fff700 1 -- 172.21.15.97:0/4016830104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c800fbd0 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.404+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d0159290 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.405+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d0068480 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.406+0000 7f32b7fff700 1 -- 172.21.15.97:0/4016830104 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f32c800fd30 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.406+0000 7f32b7fff700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f32bc06af60 0x7f32bc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.406+0000 7f32b7fff700 1 -- 172.21.15.97:0/4016830104 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32c8089ce0 con 0x7f32d015c880 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.406+0000 7f32ceffd700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f32bc06af60 0x7f32bc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.406+0000 7f32ceffd700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f32bc06af60 0x7f32bc06d420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f32d00cc1d0 tx=0x7f32c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.410+0000 7f32b7fff700 1 -- 172.21.15.97:0/4016830104 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32c8054c90 con 0x7f32d015c880 2023-11-24T06:10:38.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.628+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp4467", "target": ["mon-mgr", ""]}) v1 -- 0x7f32d015ebc0 con 0x7f32bc06af60 2023-11-24T06:10:38.636 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:38.633+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:38.636 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:38.633+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:38.636 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:38.633+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:38.636 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:38.633+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:38.636 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:38.633+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.648+0000 7f32b7fff700 1 -- 172.21.15.97:0/4016830104 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f32c0004de0 con 0x7f32bc06af60 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f32bc06af60 msgr2=0x7f32bc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f32bc06af60 0x7f32bc06d420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f32d00cc1d0 tx=0x7f32c0009040 comp rx=0 tx=0).stop 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d015c880 msgr2=0x7f32d015cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d015c880 0x7f32d015cd30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f32b80047d0 tx=0x7f32c800f490 comp rx=0 tx=0).stop 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 shutdown_connections 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32d015c880 0x7f32d015cd30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f32bc06af60 0x7f32bc06d420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32b8004850 0x7f32d014cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 --2- 172.21.15.97:0/4016830104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32d0158ad0 0x7f32d01f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.650+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 >> 172.21.15.97:0/4016830104 conn(0x7f32d006b410 msgr2=0x7f32d00b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.651+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 shutdown_connections 2023-11-24T06:10:38.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.651+0000 7f32d5979700 1 -- 172.21.15.97:0/4016830104 wait complete. 2023-11-24T06:10:38.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol1178 subvol_grp4467 2023-11-24T06:10:38.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.787+0000 7f8cda7cc700 1 Processor -- start 2023-11-24T06:10:38.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.787+0000 7f8cda7cc700 1 -- start start 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.787+0000 7f8cda7cc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8cd40ce5a0 0x7f8cd40ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.787+0000 7f8cda7cc700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd40aeff0 con 0x7f8cd415a510 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cda7cc700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd40adf50 con 0x7f8cd40ad720 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cda7cc700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd40aebd0 con 0x7f8cd40ce5a0 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8cd40ce5a0 0x7f8cd40ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd3fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8cd415a510 0x7f8cd40cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40692/0 (socket says 172.21.15.97:40692) 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd3fff700 1 -- 172.21.15.97:0/1372943333 learned_addr learned my addr 172.21.15.97:0/1372943333 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd37fe700 1 -- 172.21.15.97:0/1372943333 >> v1:172.21.15.183:6789/0 conn(0x7f8cd40ad720 legacy=0x7f8cd40afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd37fe700 1 -- 172.21.15.97:0/1372943333 >> v1:172.21.15.97:6789/0 conn(0x7f8cd415a510 legacy=0x7f8cd40cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd37fe700 1 -- 172.21.15.97:0/1372943333 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd40ae230 con 0x7f8cd40ce5a0 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd2ffd700 1 -- 172.21.15.97:0/1372943333 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66448886 0 0) 0x7f8cbc001110 con 0x7f8cd415a510 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd37fe700 1 --2- 172.21.15.97:0/1372943333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ce5a0 0x7f8cd40ad1e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400a9e0 tx=0x7f8cc400acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd0126a2d94f337 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd2ffd700 1 -- 172.21.15.97:0/1372943333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400cb30 con 0x7f8cd40ce5a0 2023-11-24T06:10:38.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd2ffd700 1 -- 172.21.15.97:0/1372943333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc400cc90 con 0x7f8cd40ce5a0 2023-11-24T06:10:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.788+0000 7f8cd2ffd700 1 -- 172.21.15.97:0/1372943333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4004030 con 0x7f8cd40ce5a0 2023-11-24T06:10:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.789+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1372943333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ce5a0 msgr2=0x7f8cd40ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.789+0000 7f8cda7cc700 1 --2- 172.21.15.97:0/1372943333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ce5a0 0x7f8cd40ad1e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400a9e0 tx=0x7f8cc400acf0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.789+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1372943333 shutdown_connections 2023-11-24T06:10:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.789+0000 7f8cda7cc700 1 --2- 172.21.15.97:0/1372943333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ce5a0 0x7f8cd40ad1e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.789+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1372943333 >> 172.21.15.97:0/1372943333 conn(0x7f8cd4072710 msgr2=0x7f8cd4072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.789+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1372943333 shutdown_connections 2023-11-24T06:10:38.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.789+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1372943333 wait complete. 2023-11-24T06:10:38.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.790+0000 7f8cda7cc700 1 Processor -- start 2023-11-24T06:10:38.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.791+0000 7f8cda7cc700 1 -- start start 2023-11-24T06:10:38.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.791+0000 7f8cda7cc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ad720 0x7f8cd40c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.791+0000 7f8cda7cc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cd40ce5a0 0x7f8cd40c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.791+0000 7f8cda7cc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cd415a510 0x7f8cd40ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.791+0000 7f8cda7cc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40c9010 con 0x7f8cd40ad720 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.791+0000 7f8cda7cc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd415c300 con 0x7f8cd415a510 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.791+0000 7f8cda7cc700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40c7f70 con 0x7f8cd40ce5a0 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ad720 0x7f8cd40c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ad720 0x7f8cd40c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48246/0 (socket says 172.21.15.97:48246) 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 -- 172.21.15.97:0/1474333200 learned_addr learned my addr 172.21.15.97:0/1474333200 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 -- 172.21.15.97:0/1474333200 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cd40ce5a0 msgr2=0x7f8cd40c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd37fe700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cd40ce5a0 0x7f8cd40c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cd40ce5a0 0x7f8cd40c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd8d69700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cd415a510 0x7f8cd40ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 -- 172.21.15.97:0/1474333200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cd415a510 msgr2=0x7f8cd40ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cd415a510 0x7f8cd40ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 -- 172.21.15.97:0/1474333200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc400a640 con 0x7f8cd40ad720 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd3fff700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ad720 0x7f8cd40c5a80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00c580 tx=0x7f8cbc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd0ff9700 1 -- 172.21.15.97:0/1474333200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc004c80 con 0x7f8cd40ad720 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd0ff9700 1 -- 172.21.15.97:0/1474333200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc004e00 con 0x7f8cd40ad720 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd40c8250 con 0x7f8cd40ad720 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cd0ff9700 1 -- 172.21.15.97:0/1474333200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc007310 con 0x7f8cd40ad720 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.792+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd40c8810 con 0x7f8cd40ad720 2023-11-24T06:10:38.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.793+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd4067f60 con 0x7f8cd40ad720 2023-11-24T06:10:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.794+0000 7f8cd0ff9700 1 -- 172.21.15.97:0/1474333200 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8cbc007470 con 0x7f8cd40ad720 2023-11-24T06:10:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.794+0000 7f8cd0ff9700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cb406af60 0x7f8cb406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.794+0000 7f8cd0ff9700 1 -- 172.21.15.97:0/1474333200 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cbc089140 con 0x7f8cd40ad720 2023-11-24T06:10:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.794+0000 7f8cd37fe700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cb406af60 0x7f8cb406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:38.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.794+0000 7f8cd37fe700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cb406af60 0x7f8cb406d420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4156110 tx=0x7f8cc401a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:38.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:38.798+0000 7f8cd0ff9700 1 -- 172.21.15.97:0/1474333200 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8cbc054200 con 0x7f8cd40ad720 2023-11-24T06:10:39.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.018+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol1178", "group_name": "subvol_grp4467", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cd40ae7f0 con 0x7f8cb406af60 2023-11-24T06:10:39.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.040+0000 7f8cd0ff9700 1 -- 172.21.15.97:0/1474333200 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8cc401e040 con 0x7f8cb406af60 2023-11-24T06:10:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.042+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cb406af60 msgr2=0x7f8cb406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cb406af60 0x7f8cb406d420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4156110 tx=0x7f8cc401a040 comp rx=0 tx=0).stop 2023-11-24T06:10:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ad720 msgr2=0x7f8cd40c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ad720 0x7f8cd40c5a80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00c580 tx=0x7f8cbc00c940 comp rx=0 tx=0).stop 2023-11-24T06:10:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 shutdown_connections 2023-11-24T06:10:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8cd415a510 0x7f8cd40ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cd40ad720 0x7f8cd40c5a80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8cb406af60 0x7f8cb406d420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 --2- 172.21.15.97:0/1474333200 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8cd40ce5a0 0x7f8cd40c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 >> 172.21.15.97:0/1474333200 conn(0x7f8cd4072710 msgr2=0x7f8cd40cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 shutdown_connections 2023-11-24T06:10:39.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.043+0000 7f8cda7cc700 1 -- 172.21.15.97:0/1474333200 wait complete. 2023-11-24T06:10:39.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol1178 subvol_snap3680 subvol_grp4467 2023-11-24T06:10:39.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.181+0000 7fb9a4b7c700 1 Processor -- start 2023-11-24T06:10:39.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.181+0000 7fb9a4b7c700 1 -- start start 2023-11-24T06:10:39.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb9a4b7c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb9a015e160 0x7fb9a00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb9a4b7c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 0x7fb9a0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb9a4b7c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9a00744d0 0x7fb9a006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb9a4b7c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a00b4500 con 0x7fb9a00744d0 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb9a4b7c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a0074960 con 0x7fb9a015e160 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb9a4b7c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00b3c60 con 0x7fb9a00afc90 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb99ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9a00744d0 0x7fb9a006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb99ed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9a00744d0 0x7fb9a006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48260/0 (socket says 172.21.15.97:48260) 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.182+0000 7fb99ed9d700 1 -- 172.21.15.97:0/1671510581 learned_addr learned my addr 172.21.15.97:0/1671510581 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99dd9b700 1 --2- 172.21.15.97:0/1671510581 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 0x7fb9a0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99e59c700 1 --2- 172.21.15.97:0/1671510581 >> v2:172.21.15.183:3300/0 conn(0x7fb9a015e160 0x7fb9a00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99ed9d700 1 -- 172.21.15.97:0/1671510581 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 msgr2=0x7fb9a0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99ed9d700 1 --2- 172.21.15.97:0/1671510581 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 0x7fb9a0073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99ed9d700 1 -- 172.21.15.97:0/1671510581 >> v2:172.21.15.183:3300/0 conn(0x7fb9a015e160 msgr2=0x7fb9a00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99ed9d700 1 --2- 172.21.15.97:0/1671510581 >> v2:172.21.15.183:3300/0 conn(0x7fb9a015e160 0x7fb9a00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99ed9d700 1 -- 172.21.15.97:0/1671510581 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a00b58c0 con 0x7fb9a00744d0 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99e59c700 1 --2- 172.21.15.97:0/1671510581 >> v2:172.21.15.183:3300/0 conn(0x7fb9a015e160 0x7fb9a00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99ed9d700 1 --2- 172.21.15.97:0/1671510581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a00744d0 0x7fb9a006d930 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb9a009efa0 tx=0x7fb99800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d07427f3b20e732 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99d59a700 1 -- 172.21.15.97:0/1671510581 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99800e070 con 0x7fb9a00744d0 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99d59a700 1 -- 172.21.15.97:0/1671510581 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99800b910 con 0x7fb9a00744d0 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.183+0000 7fb99d59a700 1 -- 172.21.15.97:0/1671510581 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99800bcc0 con 0x7fb9a00744d0 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1671510581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a00744d0 msgr2=0x7fb9a006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1671510581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a00744d0 0x7fb9a006d930 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb9a009efa0 tx=0x7fb99800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1671510581 shutdown_connections 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1671510581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a00744d0 0x7fb9a006d930 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1671510581 >> v2:172.21.15.183:3300/0 conn(0x7fb9a015e160 0x7fb9a00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1671510581 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 0x7fb9a0073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1671510581 >> 172.21.15.97:0/1671510581 conn(0x7fb9a00d0bf0 msgr2=0x7fb9a00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1671510581 shutdown_connections 2023-11-24T06:10:39.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.184+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1671510581 wait complete. 2023-11-24T06:10:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.185+0000 7fb9a4b7c700 1 Processor -- start 2023-11-24T06:10:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.185+0000 7fb9a4b7c700 1 -- start start 2023-11-24T06:10:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb9a4b7c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a00744d0 0x7fb9a014f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb9a4b7c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 0x7fb9a00b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb9a4b7c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a015e160 0x7fb9a015e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb9a4b7c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a0157c00 con 0x7fb9a015e160 2023-11-24T06:10:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb9a4b7c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a0155250 con 0x7fb9a00744d0 2023-11-24T06:10:39.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb9a4b7c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a0155530 con 0x7fb9a00afc90 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb99ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a015e160 0x7fb9a015e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb99ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a015e160 0x7fb9a015e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48266/0 (socket says 172.21.15.97:48266) 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.186+0000 7fb99ed9d700 1 -- 172.21.15.97:0/1283554291 learned_addr learned my addr 172.21.15.97:0/1283554291 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99dd9b700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 0x7fb9a00b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99e59c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a00744d0 0x7fb9a014f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99ed9d700 1 -- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 msgr2=0x7fb9a00b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99ed9d700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 0x7fb9a00b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99ed9d700 1 -- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a00744d0 msgr2=0x7fb9a014f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99ed9d700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a00744d0 0x7fb9a014f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99ed9d700 1 -- 172.21.15.97:0/1283554291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb998009b30 con 0x7fb9a015e160 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99e59c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a00744d0 0x7fb9a014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:39.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb99ed9d700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a015e160 0x7fb9a015e590 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0072ed0 tx=0x7fb998004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb98effd700 1 -- 172.21.15.97:0/1283554291 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980092e0 con 0x7fb9a015e160 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb98effd700 1 -- 172.21.15.97:0/1283554291 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb998004250 con 0x7fb9a015e160 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a0157890 con 0x7fb9a015e160 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb98effd700 1 -- 172.21.15.97:0/1283554291 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998004600 con 0x7fb9a015e160 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.187+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a0157540 con 0x7fb9a015e160 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.188+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a0068fb0 con 0x7fb9a015e160 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.189+0000 7fb98effd700 1 -- 172.21.15.97:0/1283554291 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb998004760 con 0x7fb9a015e160 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.189+0000 7fb98effd700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb98006af60 0x7fb98006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.189+0000 7fb98effd700 1 -- 172.21.15.97:0/1283554291 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb99808abd0 con 0x7fb9a015e160 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.189+0000 7fb99e59c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb98006af60 0x7fb98006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.189+0000 7fb99e59c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb98006af60 0x7fb98006d420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb994009820 tx=0x7fb994008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.193+0000 7fb98effd700 1 -- 172.21.15.97:0/1283554291 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb998055d30 con 0x7fb9a015e160 2023-11-24T06:10:39.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.420+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol1178", "snap_name": "subvol_snap3680", "group_name": "subvol_grp4467", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9a00b4190 con 0x7fb98006af60 2023-11-24T06:10:39.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.432+0000 7fb98effd700 1 -- 172.21.15.97:0/1283554291 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb994009590 con 0x7fb98006af60 2023-11-24T06:10:39.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb98006af60 msgr2=0x7fb98006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb98006af60 0x7fb98006d420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb994009820 tx=0x7fb994008040 comp rx=0 tx=0).stop 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a015e160 msgr2=0x7fb9a015e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a015e160 0x7fb9a015e590 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0072ed0 tx=0x7fb998004ad0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 shutdown_connections 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9a015e160 0x7fb9a015e590 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9a00744d0 0x7fb9a014f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb98006af60 0x7fb98006d420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 --2- 172.21.15.97:0/1283554291 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb9a00afc90 0x7fb9a00b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 >> 172.21.15.97:0/1283554291 conn(0x7fb9a00d0bf0 msgr2=0x7fb9a006e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.434+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 shutdown_connections 2023-11-24T06:10:39.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.435+0000 7fb9a4b7c700 1 -- 172.21.15.97:0/1283554291 wait complete. 2023-11-24T06:10:39.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot metadata set cephfs subvol1178 subvol_snap3680 key value subvol_grp4467 2023-11-24T06:10:39.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.581+0000 7f9aab362700 1 Processor -- start 2023-11-24T06:10:39.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.581+0000 7f9aab362700 1 -- start start 2023-11-24T06:10:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aab362700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9aa40b4230 0x7f9aa406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aab362700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9aa40ceb70 0x7f9aa40cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aab362700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa40cf550 con 0x7f9aa415e2d0 2023-11-24T06:10:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aab362700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa415e6b0 con 0x7f9aa40ceb70 2023-11-24T06:10:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aab362700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa40cc9b0 con 0x7f9aa40b4230 2023-11-24T06:10:39.586 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa98ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9aa40ceb70 0x7f9aa40cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa90fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9aa415e2d0 0x7f9aa40ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40722/0 (socket says 172.21.15.97:40722) 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa98ff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9aa40ceb70 0x7f9aa40cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48274/0 (socket says 172.21.15.97:48274) 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa90fe700 1 -- 172.21.15.97:0/4044632601 learned_addr learned my addr 172.21.15.97:0/4044632601 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa88fd700 1 --2- 172.21.15.97:0/4044632601 >> v2:172.21.15.183:3300/0 conn(0x7f9aa40b4230 0x7f9aa406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa98ff700 1 -- 172.21.15.97:0/4044632601 >> v2:172.21.15.183:3300/0 conn(0x7f9aa40b4230 msgr2=0x7f9aa406ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa98ff700 1 --2- 172.21.15.97:0/4044632601 >> v2:172.21.15.183:3300/0 conn(0x7f9aa40b4230 0x7f9aa406ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa98ff700 1 -- 172.21.15.97:0/4044632601 >> v1:172.21.15.97:6789/0 conn(0x7f9aa415e2d0 legacy=0x7f9aa40ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa98ff700 1 -- 172.21.15.97:0/4044632601 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa40cf020 con 0x7f9aa40ceb70 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9a9bfff700 1 -- 172.21.15.97:0/4044632601 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985154157 0 0) 0x7f9aa40cf550 con 0x7f9aa415e2d0 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa88fd700 1 --2- 172.21.15.97:0/4044632601 >> v2:172.21.15.183:3300/0 conn(0x7f9aa40b4230 0x7f9aa406ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.582+0000 7f9aa98ff700 1 --2- 172.21.15.97:0/4044632601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa40cce40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40709e0 tx=0x7f9a9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7737183ca755fb2a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9a9bfff700 1 -- 172.21.15.97:0/4044632601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c00e070 con 0x7f9aa40ceb70 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9a9bfff700 1 -- 172.21.15.97:0/4044632601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9c00bbd0 con 0x7f9aa40ceb70 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9a9bfff700 1 -- 172.21.15.97:0/4044632601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c012430 con 0x7f9aa40ceb70 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9aab362700 1 -- 172.21.15.97:0/4044632601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 msgr2=0x7f9aa40cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9aab362700 1 --2- 172.21.15.97:0/4044632601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa40cce40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40709e0 tx=0x7f9a9c00b330 comp rx=0 tx=0).stop 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9aab362700 1 -- 172.21.15.97:0/4044632601 shutdown_connections 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9aab362700 1 --2- 172.21.15.97:0/4044632601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa40cce40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9aab362700 1 --2- 172.21.15.97:0/4044632601 >> v2:172.21.15.183:3300/0 conn(0x7f9aa40b4230 0x7f9aa406ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9aab362700 1 -- 172.21.15.97:0/4044632601 >> 172.21.15.97:0/4044632601 conn(0x7f9aa40b3710 msgr2=0x7f9aa40b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9aab362700 1 -- 172.21.15.97:0/4044632601 shutdown_connections 2023-11-24T06:10:39.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.583+0000 7f9aab362700 1 -- 172.21.15.97:0/4044632601 wait complete. 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.584+0000 7f9aab362700 1 Processor -- start 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aab362700 1 -- start start 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aab362700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aa40b4230 0x7f9aa40cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aab362700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa41f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aab362700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aa415e2d0 0x7f9aa4152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aab362700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa41f7f60 con 0x7f9aa40ceb70 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aab362700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa4160a40 con 0x7f9aa415e2d0 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aab362700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa41f38f0 con 0x7f9aa40b4230 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aa88fd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa41f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aa88fd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa41f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48278/0 (socket says 172.21.15.97:48278) 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.585+0000 7f9aa88fd700 1 -- 172.21.15.97:0/655112304 learned_addr learned my addr 172.21.15.97:0/655112304 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa90fe700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aa40b4230 0x7f9aa40cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa98ff700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aa415e2d0 0x7f9aa4152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa88fd700 1 -- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aa40b4230 msgr2=0x7f9aa40cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa88fd700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aa40b4230 0x7f9aa40cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa88fd700 1 -- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aa415e2d0 msgr2=0x7f9aa4152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa88fd700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aa415e2d0 0x7f9aa4152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa88fd700 1 -- 172.21.15.97:0/655112304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c009a70 con 0x7f9aa40ceb70 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa90fe700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aa40b4230 0x7f9aa40cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa98ff700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aa415e2d0 0x7f9aa4152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aa88fd700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa41f7a20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009fd0 tx=0x7f9aa000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9a99ffb700 1 -- 172.21.15.97:0/655112304 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0013610 con 0x7f9aa40ceb70 2023-11-24T06:10:39.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9a99ffb700 1 -- 172.21.15.97:0/655112304 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa0013770 con 0x7f9aa40ceb70 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa4152f50 con 0x7f9aa40ceb70 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9a99ffb700 1 -- 172.21.15.97:0/655112304 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0013b20 con 0x7f9aa40ceb70 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.586+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa41603f0 con 0x7f9aa40ceb70 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.587+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa4160bd0 con 0x7f9aa40ceb70 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.588+0000 7f9a99ffb700 1 -- 172.21.15.97:0/655112304 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f9aa0013c80 con 0x7f9aa40ceb70 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.588+0000 7f9a99ffb700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9a8806aff0 0x7f9a8806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.588+0000 7f9a99ffb700 1 -- 172.21.15.97:0/655112304 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9aa00895d0 con 0x7f9aa40ceb70 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.588+0000 7f9aa90fe700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9a8806aff0 0x7f9a8806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:39.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.589+0000 7f9aa90fe700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9a8806aff0 0x7f9a8806d4b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9a94007fd0 tx=0x7f9a94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:39.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.593+0000 7f9a99ffb700 1 -- 172.21.15.97:0/655112304 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9aa0054420 con 0x7f9aa40ceb70 2023-11-24T06:10:39.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.820+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot metadata set", "vol_name": "cephfs", "sub_name": "subvol1178", "snap_name": "subvol_snap3680", "key_name": "key", "value": "value", "group_name": "subvol_grp4467", "target": ["mon-mgr", ""]}) v1 -- 0x7f9aa415d9e0 con 0x7f9a8806aff0 2023-11-24T06:10:39.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.836+0000 7f9a99ffb700 1 -- 172.21.15.97:0/655112304 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9aa415d9e0 con 0x7f9a8806aff0 2023-11-24T06:10:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9a8806aff0 msgr2=0x7f9a8806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9a8806aff0 0x7f9a8806d4b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9a94007fd0 tx=0x7f9a94009040 comp rx=0 tx=0).stop 2023-11-24T06:10:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 msgr2=0x7f9aa41f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa41f7a20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009fd0 tx=0x7f9aa000eb70 comp rx=0 tx=0).stop 2023-11-24T06:10:39.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 shutdown_connections 2023-11-24T06:10:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aa415e2d0 0x7f9aa4152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9a8806aff0 0x7f9a8806d4b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9aa40b4230 0x7f9aa40cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 --2- 172.21.15.97:0/655112304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aa40ceb70 0x7f9aa41f7a20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 >> 172.21.15.97:0/655112304 conn(0x7f9aa40b3710 msgr2=0x7f9aa40b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 shutdown_connections 2023-11-24T06:10:39.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:39.838+0000 7f9aab362700 1 -- 172.21.15.97:0/655112304 wait complete. 2023-11-24T06:10:39.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:39.853 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp4467/subvol1178/.meta ./volumes/subvol_grp4467/subvol1178/.meta.stale_snap_section) 2023-11-24T06:10:42.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol1178 subvol_snap3680 subvol_grp4467 2023-11-24T06:10:42.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.402+0000 7fc8e4a34700 1 Processor -- start 2023-11-24T06:10:42.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.402+0000 7fc8e4a34700 1 -- start start 2023-11-24T06:10:42.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.403+0000 7fc8e4a34700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc8e00746a0 0x7fc8e00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.403+0000 7fc8e4a34700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8e00afc90 0x7fc8e006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.403+0000 7fc8e4a34700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e00b3c60 con 0x7fc8e015e160 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.403+0000 7fc8e4a34700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00b4500 con 0x7fc8e00afc90 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.403+0000 7fc8e4a34700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00b58c0 con 0x7fc8e00746a0 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.403+0000 7fc8ded9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8e00afc90 0x7fc8e006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.403+0000 7fc8ded9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8e00afc90 0x7fc8e006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48290/0 (socket says 172.21.15.97:48290) 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.403+0000 7fc8ded9d700 1 -- 172.21.15.97:0/648180203 learned_addr learned my addr 172.21.15.97:0/648180203 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8ddd9b700 1 --2- 172.21.15.97:0/648180203 >> v2:172.21.15.183:3300/0 conn(0x7fc8e00746a0 0x7fc8e00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8ded9d700 1 -- 172.21.15.97:0/648180203 >> v2:172.21.15.183:3300/0 conn(0x7fc8e00746a0 msgr2=0x7fc8e00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8ded9d700 1 --2- 172.21.15.97:0/648180203 >> v2:172.21.15.183:3300/0 conn(0x7fc8e00746a0 0x7fc8e00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8ded9d700 1 -- 172.21.15.97:0/648180203 >> v1:172.21.15.183:6789/0 conn(0x7fc8e015e160 legacy=0x7fc8e0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8ded9d700 1 -- 172.21.15.97:0/648180203 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e006bdc0 con 0x7fc8e00afc90 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8ddd9b700 1 --2- 172.21.15.97:0/648180203 >> v2:172.21.15.183:3300/0 conn(0x7fc8e00746a0 0x7fc8e00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8ded9d700 1 --2- 172.21.15.97:0/648180203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00afc90 0x7fc8e006d930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00d0880 tx=0x7fc8d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4eb0a2f325ea96 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8dd59a700 1 -- 172.21.15.97:0/648180203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800e070 con 0x7fc8e00afc90 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8dd59a700 1 -- 172.21.15.97:0/648180203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d800b9c0 con 0x7fc8e00afc90 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.404+0000 7fc8dd59a700 1 -- 172.21.15.97:0/648180203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800bd70 con 0x7fc8e00afc90 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.405+0000 7fc8e4a34700 1 -- 172.21.15.97:0/648180203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00afc90 msgr2=0x7fc8e006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.405+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/648180203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00afc90 0x7fc8e006d930 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00d0880 tx=0x7fc8d800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.405+0000 7fc8e4a34700 1 -- 172.21.15.97:0/648180203 shutdown_connections 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.405+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/648180203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00afc90 0x7fc8e006d930 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.405+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/648180203 >> v2:172.21.15.183:3300/0 conn(0x7fc8e00746a0 0x7fc8e00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.405+0000 7fc8e4a34700 1 -- 172.21.15.97:0/648180203 >> 172.21.15.97:0/648180203 conn(0x7fc8e00d0bf0 msgr2=0x7fc8e00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.405+0000 7fc8e4a34700 1 -- 172.21.15.97:0/648180203 shutdown_connections 2023-11-24T06:10:42.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.405+0000 7fc8e4a34700 1 -- 172.21.15.97:0/648180203 wait complete. 2023-11-24T06:10:42.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.406+0000 7fc8e4a34700 1 Processor -- start 2023-11-24T06:10:42.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.406+0000 7fc8e4a34700 1 -- start start 2023-11-24T06:10:42.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8e4a34700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00746a0 0x7fc8e0146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8e4a34700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8e00afc90 0x7fc8e006db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8e4a34700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8e015e160 0x7fc8e006e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8e4a34700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0159750 con 0x7fc8e00746a0 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8e4a34700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e015fb90 con 0x7fc8e015e160 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8e4a34700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e015fe70 con 0x7fc8e00afc90 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00746a0 0x7fc8e0146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00746a0 0x7fc8e0146a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48292/0 (socket says 172.21.15.97:48292) 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 -- 172.21.15.97:0/2515981020 learned_addr learned my addr 172.21.15.97:0/2515981020 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8ded9d700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8e015e160 0x7fc8e006e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8ddd9b700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8e00afc90 0x7fc8e006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 -- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8e00afc90 msgr2=0x7fc8e006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8e00afc90 0x7fc8e006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 -- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8e015e160 msgr2=0x7fc8e006e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8e015e160 0x7fc8e006e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 -- 172.21.15.97:0/2515981020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d8009b30 con 0x7fc8e00746a0 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8ddd9b700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8e00afc90 0x7fc8e006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8de59c700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00746a0 0x7fc8e0146a80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c450 tx=0x7fc8d400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.407+0000 7fc8ceffd700 1 -- 172.21.15.97:0/2515981020 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40052d0 con 0x7fc8e00746a0 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.408+0000 7fc8ceffd700 1 -- 172.21.15.97:0/2515981020 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4005430 con 0x7fc8e00746a0 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.408+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e0160150 con 0x7fc8e00746a0 2023-11-24T06:10:42.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.408+0000 7fc8ceffd700 1 -- 172.21.15.97:0/2515981020 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40038c0 con 0x7fc8e00746a0 2023-11-24T06:10:42.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.408+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e0159270 con 0x7fc8e00746a0 2023-11-24T06:10:42.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.409+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e0068fb0 con 0x7fc8e00746a0 2023-11-24T06:10:42.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.409+0000 7fc8ceffd700 1 -- 172.21.15.97:0/2515981020 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc8d4003a20 con 0x7fc8e00746a0 2023-11-24T06:10:42.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.409+0000 7fc8ceffd700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc8c006aff0 0x7fc8c006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.409+0000 7fc8ceffd700 1 -- 172.21.15.97:0/2515981020 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc8d4088a60 con 0x7fc8e00746a0 2023-11-24T06:10:42.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.409+0000 7fc8ddd9b700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc8c006aff0 0x7fc8c006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.410+0000 7fc8ddd9b700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc8c006aff0 0x7fc8c006d4b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0147d10 tx=0x7fc8d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.414+0000 7fc8ceffd700 1 -- 172.21.15.97:0/2515981020 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8d4053a50 con 0x7fc8e00746a0 2023-11-24T06:10:42.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.642+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol1178", "snap_name": "subvol_snap3680", "group_name": "subvol_grp4467", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8e015e540 con 0x7fc8c006aff0 2023-11-24T06:10:42.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.673+0000 7fc8ceffd700 1 -- 172.21.15.97:0/2515981020 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc8d000c050 con 0x7fc8c006aff0 2023-11-24T06:10:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc8c006aff0 msgr2=0x7fc8c006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc8c006aff0 0x7fc8c006d4b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0147d10 tx=0x7fc8d0008040 comp rx=0 tx=0).stop 2023-11-24T06:10:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00746a0 msgr2=0x7fc8e0146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00746a0 0x7fc8e0146a80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c450 tx=0x7fc8d400c810 comp rx=0 tx=0).stop 2023-11-24T06:10:42.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 shutdown_connections 2023-11-24T06:10:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8e015e160 0x7fc8e006e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e00746a0 0x7fc8e0146a80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc8c006aff0 0x7fc8c006d4b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 --2- 172.21.15.97:0/2515981020 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc8e00afc90 0x7fc8e006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 >> 172.21.15.97:0/2515981020 conn(0x7fc8e00d0bf0 msgr2=0x7fc8e015d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 shutdown_connections 2023-11-24T06:10:42.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.675+0000 7fc8e4a34700 1 -- 172.21.15.97:0/2515981020 wait complete. 2023-11-24T06:10:42.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:42.691 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp4467/subvol1178/.meta.stale_snap_section ./volumes/subvol_grp4467/subvol1178/.meta) 2023-11-24T06:10:42.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:42.727 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap3680 ./volumes/subvol_grp4467/subvol1178/.meta) 2023-11-24T06:10:42.793 INFO:teuthology.orchestra.run.smithi097.stdout:[SNAP_METADATA_subvol_snap3680] 2023-11-24T06:10:42.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol1178 subvol_grp4467 2023-11-24T06:10:42.958 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.956+0000 7f561bfd6700 1 Processor -- start 2023-11-24T06:10:42.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.956+0000 7f561bfd6700 1 -- start start 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f561bfd6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b01e0 0x7f561406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f561bfd6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f56140b59c0 0x7f56140b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f561bfd6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56140b5e40 con 0x7f5614073f90 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f561bfd6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56141470a0 con 0x7f56140b59c0 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f561bfd6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56140b05c0 con 0x7f56140b01e0 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5619571700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f56140b59c0 0x7f56140b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5619571700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f56140b59c0 0x7f56140b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48302/0 (socket says 172.21.15.97:48302) 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5619571700 1 -- 172.21.15.97:0/824611311 learned_addr learned my addr 172.21.15.97:0/824611311 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:42.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5619571700 1 -- 172.21.15.97:0/824611311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b01e0 msgr2=0x7f561406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5619571700 1 --2- 172.21.15.97:0/824611311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b01e0 0x7f561406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5619571700 1 -- 172.21.15.97:0/824611311 >> v1:172.21.15.183:6789/0 conn(0x7f5614073f90 legacy=0x7f561406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5619571700 1 -- 172.21.15.97:0/824611311 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140b4250 con 0x7f56140b59c0 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5619571700 1 --2- 172.21.15.97:0/824611311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b59c0 0x7f56140b3b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f56040098f0 tx=0x7f5604009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac9eb56b1cbf9b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.957+0000 7f5618d70700 1 -- 172.21.15.97:0/824611311 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604004230 con 0x7f56140b59c0 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f5618d70700 1 -- 172.21.15.97:0/824611311 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5604004390 con 0x7f56140b59c0 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f5618d70700 1 -- 172.21.15.97:0/824611311 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56040036a0 con 0x7f56140b59c0 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f561bfd6700 1 -- 172.21.15.97:0/824611311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b59c0 msgr2=0x7f56140b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f561bfd6700 1 --2- 172.21.15.97:0/824611311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b59c0 0x7f56140b3b50 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f56040098f0 tx=0x7f5604009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f561bfd6700 1 -- 172.21.15.97:0/824611311 shutdown_connections 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f561bfd6700 1 --2- 172.21.15.97:0/824611311 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b01e0 0x7f561406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f561bfd6700 1 --2- 172.21.15.97:0/824611311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b59c0 0x7f56140b3b50 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f561bfd6700 1 -- 172.21.15.97:0/824611311 >> 172.21.15.97:0/824611311 conn(0x7f56140d0b40 msgr2=0x7f56140d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:42.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f561bfd6700 1 -- 172.21.15.97:0/824611311 shutdown_connections 2023-11-24T06:10:42.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.958+0000 7f561bfd6700 1 -- 172.21.15.97:0/824611311 wait complete. 2023-11-24T06:10:42.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.959+0000 7f561bfd6700 1 Processor -- start 2023-11-24T06:10:42.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.959+0000 7f561bfd6700 1 -- start start 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f561bfd6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5614073f90 0x7f5614160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f561bfd6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b01e0 0x7f5614141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f561bfd6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b59c0 0x7f5614142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f561bfd6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5614140660 con 0x7f56140b01e0 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f561bfd6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5614140940 con 0x7f5614073f90 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f561bfd6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5614140c20 con 0x7f56140b59c0 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619571700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b01e0 0x7f5614141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619571700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b01e0 0x7f5614141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48310/0 (socket says 172.21.15.97:48310) 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619571700 1 -- 172.21.15.97:0/2133513492 learned_addr learned my addr 172.21.15.97:0/2133513492 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619d72700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5614073f90 0x7f5614160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f561a573700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b59c0 0x7f5614142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619571700 1 -- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b59c0 msgr2=0x7f5614142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619571700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b59c0 0x7f5614142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619571700 1 -- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5614073f90 msgr2=0x7f5614160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619571700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5614073f90 0x7f5614160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f5619571700 1 -- 172.21.15.97:0/2133513492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604009550 con 0x7f56140b01e0 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.960+0000 7f561a573700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b59c0 0x7f5614142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.961+0000 7f5619571700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b01e0 0x7f5614141d40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5604000c00 tx=0x7f5604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.961+0000 7f560a7fc700 1 -- 172.21.15.97:0/2133513492 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400b660 con 0x7f56140b01e0 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.961+0000 7f560a7fc700 1 -- 172.21.15.97:0/2133513492 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560400b7c0 con 0x7f56140b01e0 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.961+0000 7f560a7fc700 1 -- 172.21.15.97:0/2133513492 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400bb70 con 0x7f56140b01e0 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.961+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5614140f00 con 0x7f56140b01e0 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.961+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5614161210 con 0x7f56140b01e0 2023-11-24T06:10:42.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.962+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56140b5430 con 0x7f56140b01e0 2023-11-24T06:10:42.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.962+0000 7f560a7fc700 1 -- 172.21.15.97:0/2133513492 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f5604021070 con 0x7f56140b01e0 2023-11-24T06:10:42.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.963+0000 7f560a7fc700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f55f806aff0 0x7f55f806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:42.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.963+0000 7f5619d72700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f55f806aff0 0x7f55f806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:42.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.963+0000 7f560a7fc700 1 -- 172.21.15.97:0/2133513492 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f560408b040 con 0x7f56140b01e0 2023-11-24T06:10:42.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.963+0000 7f5619d72700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f55f806aff0 0x7f55f806d4b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f561406c700 tx=0x7f5610009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:42.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:42.967+0000 7f560a7fc700 1 -- 172.21.15.97:0/2133513492 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5604056030 con 0x7f56140b01e0 2023-11-24T06:10:43.194 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.192+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol1178", "group_name": "subvol_grp4467", "target": ["mon-mgr", ""]}) v1 -- 0x7f5614140ab0 con 0x7f55f806aff0 2023-11-24T06:10:43.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.226+0000 7f560a7fc700 1 -- 172.21.15.97:0/2133513492 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+704 (secure 0 0 0) 0x7f5614140ab0 con 0x7f55f806aff0 2023-11-24T06:10:43.229 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "atime": "2023-11-24 06:10:39", 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "bytes_pcent": "undefined", 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "bytes_quota": "infinite", 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "bytes_used": 0, 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "created_at": "2023-11-24 06:10:39", 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "ctime": "2023-11-24 06:10:39", 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "data_pool": "cephfs_data", 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "features": [ 2023-11-24T06:10:43.230 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot-clone", 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot-autoprotect", 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot-retention" 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 0, 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": 16877, 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "mon_addrs": [ 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "172.21.15.97:6789", 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "172.21.15.183:6789", 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "172.21.15.183:6790" 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "mtime": "2023-11-24 06:10:39", 2023-11-24T06:10:43.231 INFO:teuthology.orchestra.run.smithi097.stdout: "path": "/volumes/subvol_grp4467/subvol1178/548d60c1-44be-4481-9206-a63793f5c4bb", 2023-11-24T06:10:43.232 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_namespace": "", 2023-11-24T06:10:43.232 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "complete", 2023-11-24T06:10:43.232 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "subvolume", 2023-11-24T06:10:43.232 INFO:teuthology.orchestra.run.smithi097.stdout: "uid": 0 2023-11-24T06:10:43.232 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:10:43.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f55f806aff0 msgr2=0x7f55f806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f55f806aff0 0x7f55f806d4b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f561406c700 tx=0x7f5610009190 comp rx=0 tx=0).stop 2023-11-24T06:10:43.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b01e0 msgr2=0x7f5614141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b01e0 0x7f5614141d40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5604000c00 tx=0x7f5604004750 comp rx=0 tx=0).stop 2023-11-24T06:10:43.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 shutdown_connections 2023-11-24T06:10:43.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f55f806aff0 0x7f55f806d4b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f56140b59c0 0x7f5614142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5614073f90 0x7f5614160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 --2- 172.21.15.97:0/2133513492 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56140b01e0 0x7f5614141d40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 >> 172.21.15.97:0/2133513492 conn(0x7f56140d0b40 msgr2=0x7f5614162000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:43.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 shutdown_connections 2023-11-24T06:10:43.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.229+0000 7f561bfd6700 1 -- 172.21.15.97:0/2133513492 wait complete. 2023-11-24T06:10:43.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:43.243 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap3680 ./volumes/subvol_grp4467/subvol1178/.meta) 2023-11-24T06:10:43.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:10:43.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol1178 subvol_grp4467 2023-11-24T06:10:43.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.436+0000 7f8a06c3e700 1 Processor -- start 2023-11-24T06:10:43.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.436+0000 7f8a06c3e700 1 -- start start 2023-11-24T06:10:43.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a06c3e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8a000cf040 0x7f8a0006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a06c3e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a000d21c0 0x7f8a000af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a06c3e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a000d01c0 con 0x7f8a000702e0 2023-11-24T06:10:43.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a06c3e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a00070910 con 0x7f8a000d21c0 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a06c3e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a00070bf0 con 0x7f8a000cf040 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f89fffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8a000702e0 0x7f8a000d1ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40774/0 (socket says 172.21.15.97:40774) 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a051db700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a000d21c0 0x7f8a000af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f89fffff700 1 -- 172.21.15.97:0/2772759622 learned_addr learned my addr 172.21.15.97:0/2772759622 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a049da700 1 --2- 172.21.15.97:0/2772759622 >> v2:172.21.15.183:3300/0 conn(0x7f8a000cf040 0x7f8a0006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a051db700 1 -- 172.21.15.97:0/2772759622 >> v2:172.21.15.183:3300/0 conn(0x7f8a000cf040 msgr2=0x7f8a0006fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a051db700 1 --2- 172.21.15.97:0/2772759622 >> v2:172.21.15.183:3300/0 conn(0x7f8a000cf040 0x7f8a0006fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a051db700 1 -- 172.21.15.97:0/2772759622 >> v1:172.21.15.97:6789/0 conn(0x7f8a000702e0 legacy=0x7f8a000d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.437+0000 7f8a051db700 1 -- 172.21.15.97:0/2772759622 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a000aebb0 con 0x7f8a000d21c0 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f8a051db700 1 --2- 172.21.15.97:0/2772759622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000d21c0 0x7f8a000af3b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8a000cf690 tx=0x7f89f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1e419572502bd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2772759622 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f800e070 con 0x7f8a000d21c0 2023-11-24T06:10:43.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2772759622 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f800bbd0 con 0x7f8a000d21c0 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f89ff7fe700 1 -- 172.21.15.97:0/2772759622 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8012430 con 0x7f8a000d21c0 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f8a06c3e700 1 -- 172.21.15.97:0/2772759622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000d21c0 msgr2=0x7f8a000af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/2772759622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000d21c0 0x7f8a000af3b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8a000cf690 tx=0x7f89f800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f8a06c3e700 1 -- 172.21.15.97:0/2772759622 shutdown_connections 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/2772759622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000d21c0 0x7f8a000af3b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/2772759622 >> v2:172.21.15.183:3300/0 conn(0x7f8a000cf040 0x7f8a0006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f8a06c3e700 1 -- 172.21.15.97:0/2772759622 >> 172.21.15.97:0/2772759622 conn(0x7f8a0009aa00 msgr2=0x7f8a0009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.438+0000 7f8a06c3e700 1 -- 172.21.15.97:0/2772759622 shutdown_connections 2023-11-24T06:10:43.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.439+0000 7f8a06c3e700 1 -- 172.21.15.97:0/2772759622 wait complete. 2023-11-24T06:10:43.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.440+0000 7f8a06c3e700 1 Processor -- start 2023-11-24T06:10:43.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.440+0000 7f8a06c3e700 1 -- start start 2023-11-24T06:10:43.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f8a06c3e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a000702e0 0x7f8a00150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f8a06c3e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000cf040 0x7f8a000cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f8a06c3e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a000d21c0 0x7f8a000b6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f8a06c3e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000b5f20 con 0x7f8a000cf040 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f8a06c3e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000b52a0 con 0x7f8a000702e0 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f8a06c3e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000b5580 con 0x7f8a000d21c0 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f89fffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000cf040 0x7f8a000cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f89fffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000cf040 0x7f8a000cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48330/0 (socket says 172.21.15.97:48330) 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f89fffff700 1 -- 172.21.15.97:0/3769727409 learned_addr learned my addr 172.21.15.97:0/3769727409 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f8a049da700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a000702e0 0x7f8a00150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.441+0000 7f8a051db700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a000d21c0 0x7f8a000b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fffff700 1 -- 172.21.15.97:0/3769727409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a000d21c0 msgr2=0x7f8a000b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fffff700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a000d21c0 0x7f8a000b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fffff700 1 -- 172.21.15.97:0/3769727409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a000702e0 msgr2=0x7f8a00150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fffff700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a000702e0 0x7f8a00150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fffff700 1 -- 172.21.15.97:0/3769727409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f8009a70 con 0x7f8a000cf040 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fffff700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000cf040 0x7f8a000cb550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c520 tx=0x7f89ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fd7fa700 1 -- 172.21.15.97:0/3769727409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec005d70 con 0x7f8a000cf040 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fd7fa700 1 -- 172.21.15.97:0/3769727409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec0070d0 con 0x7f8a000cf040 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a000b5860 con 0x7f8a000cf040 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f89fd7fa700 1 -- 172.21.15.97:0/3769727409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec007440 con 0x7f8a000cf040 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.442+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a000cba90 con 0x7f8a000cf040 2023-11-24T06:10:43.447 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.443+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a000cdd10 con 0x7f8a000cf040 2023-11-24T06:10:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.443+0000 7f89fd7fa700 1 -- 172.21.15.97:0/3769727409 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f89ec0075a0 con 0x7f8a000cf040 2023-11-24T06:10:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.444+0000 7f89fd7fa700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f89e406aff0 0x7f89e406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.444+0000 7f8a049da700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f89e406aff0 0x7f89e406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.444+0000 7f89fd7fa700 1 -- 172.21.15.97:0/3769727409 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f89ec088a10 con 0x7f8a000cf040 2023-11-24T06:10:43.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.444+0000 7f8a049da700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f89e406aff0 0x7f89e406d4b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89f400ab60 tx=0x7f89f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.448+0000 7f89fd7fa700 1 -- 172.21.15.97:0/3769727409 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89ec053a20 con 0x7f8a000cf040 2023-11-24T06:10:43.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.668+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol1178", "group_name": "subvol_grp4467", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a000cf420 con 0x7f89e406aff0 2023-11-24T06:10:43.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.683+0000 7f89fd7fa700 1 -- 172.21.15.97:0/3769727409 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f89f400c040 con 0x7f89e406aff0 2023-11-24T06:10:43.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f89e406aff0 msgr2=0x7f89e406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f89e406aff0 0x7f89e406d4b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89f400ab60 tx=0x7f89f4008040 comp rx=0 tx=0).stop 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000cf040 msgr2=0x7f8a000cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000cf040 0x7f8a000cb550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c520 tx=0x7f89ec00c830 comp rx=0 tx=0).stop 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 shutdown_connections 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f89e406aff0 0x7f89e406d4b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a000d21c0 0x7f8a000b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a000702e0 0x7f8a00150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 --2- 172.21.15.97:0/3769727409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a000cf040 0x7f8a000cb550 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 >> 172.21.15.97:0/3769727409 conn(0x7f8a0009aa00 msgr2=0x7f8a000b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:43.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 shutdown_connections 2023-11-24T06:10:43.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.685+0000 7f8a06c3e700 1 -- 172.21.15.97:0/3769727409 wait complete. 2023-11-24T06:10:43.690 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.686+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.690 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.686+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.690 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.686+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.691 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.686+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.691 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.686+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.702 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.700+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.702 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.700+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.702 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.700+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.703 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.700+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.703 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:10:43.700+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:10:43.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp4467 2023-11-24T06:10:43.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.826+0000 7fb6d6a92700 1 Processor -- start 2023-11-24T06:10:43.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.826+0000 7fb6d6a92700 1 -- start start 2023-11-24T06:10:43.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6d6a92700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6d015f9e0 0x7fb6d00add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6d6a92700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fb6d0073f90 0x7fb6d0074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6d6a92700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d006b080 con 0x7fb6d00cf610 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6d6a92700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d0074960 con 0x7fb6d015f9e0 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6d6a92700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d015c290 con 0x7fb6d0073f90 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6d482e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb6d00cf610 0x7fb6d00b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40816/0 (socket says 172.21.15.97:40816) 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6cffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6d015f9e0 0x7fb6d00add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6d482e700 1 -- 172.21.15.97:0/697425700 learned_addr learned my addr 172.21.15.97:0/697425700 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6d502f700 1 --2- 172.21.15.97:0/697425700 >> v2:172.21.15.183:3300/0 conn(0x7fb6d0073f90 0x7fb6d0074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6cffff700 1 -- 172.21.15.97:0/697425700 >> v2:172.21.15.183:3300/0 conn(0x7fb6d0073f90 msgr2=0x7fb6d0074420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.827+0000 7fb6cffff700 1 --2- 172.21.15.97:0/697425700 >> v2:172.21.15.183:3300/0 conn(0x7fb6d0073f90 0x7fb6d0074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6cffff700 1 -- 172.21.15.97:0/697425700 >> v1:172.21.15.97:6789/0 conn(0x7fb6d00cf610 legacy=0x7fb6d00b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6cf7fe700 1 -- 172.21.15.97:0/697425700 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159076363 0 0) 0x7fb6d006b080 con 0x7fb6d00cf610 2023-11-24T06:10:43.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6cffff700 1 -- 172.21.15.97:0/697425700 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d00cf9f0 con 0x7fb6d015f9e0 2023-11-24T06:10:43.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6cffff700 1 --2- 172.21.15.97:0/697425700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015f9e0 0x7fb6d00add40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009840 tx=0x7fb6bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629f3af4fd34173a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6cf7fe700 1 -- 172.21.15.97:0/697425700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc011830 con 0x7fb6d015f9e0 2023-11-24T06:10:43.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6cf7fe700 1 -- 172.21.15.97:0/697425700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc011990 con 0x7fb6d015f9e0 2023-11-24T06:10:43.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6cf7fe700 1 -- 172.21.15.97:0/697425700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc011d20 con 0x7fb6d015f9e0 2023-11-24T06:10:43.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6d6a92700 1 -- 172.21.15.97:0/697425700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015f9e0 msgr2=0x7fb6d00add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/697425700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015f9e0 0x7fb6d00add40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc009840 tx=0x7fb6bc009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6d6a92700 1 -- 172.21.15.97:0/697425700 shutdown_connections 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/697425700 >> v2:172.21.15.183:3300/0 conn(0x7fb6d0073f90 0x7fb6d0074420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/697425700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d015f9e0 0x7fb6d00add40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6d6a92700 1 -- 172.21.15.97:0/697425700 >> 172.21.15.97:0/697425700 conn(0x7fb6d00d0bf0 msgr2=0x7fb6d00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6d6a92700 1 -- 172.21.15.97:0/697425700 shutdown_connections 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.828+0000 7fb6d6a92700 1 -- 172.21.15.97:0/697425700 wait complete. 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.830+0000 7fb6d6a92700 1 Processor -- start 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.830+0000 7fb6d6a92700 1 -- start start 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.830+0000 7fb6d6a92700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d0073f90 0x7fb6d015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.830+0000 7fb6d6a92700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6d00cf610 0x7fb6d00b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d6a92700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6d015f9e0 0x7fb6d00ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d6a92700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00b2220 con 0x7fb6d0073f90 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d6a92700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d015e510 con 0x7fb6d00cf610 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d6a92700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d015e7f0 con 0x7fb6d015f9e0 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d0073f90 0x7fb6d015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d0073f90 0x7fb6d015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48370/0 (socket says 172.21.15.97:48370) 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 -- 172.21.15.97:0/3197132043 learned_addr learned my addr 172.21.15.97:0/3197132043 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 -- 172.21.15.97:0/3197132043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6d015f9e0 msgr2=0x7fb6d00ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6d015f9e0 0x7fb6d00ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 -- 172.21.15.97:0/3197132043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6d00cf610 msgr2=0x7fb6d00b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6cffff700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6d00cf610 0x7fb6d00b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6d00cf610 0x7fb6d00b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 -- 172.21.15.97:0/3197132043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc0094a0 con 0x7fb6d0073f90 2023-11-24T06:10:43.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d482e700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d0073f90 0x7fb6d015dec0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40011e0 tx=0x7fb6c400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6cd7fa700 1 -- 172.21.15.97:0/3197132043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4004c80 con 0x7fb6d0073f90 2023-11-24T06:10:43.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6cd7fa700 1 -- 172.21.15.97:0/3197132043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c4004e00 con 0x7fb6d0073f90 2023-11-24T06:10:43.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d015ead0 con 0x7fb6d0073f90 2023-11-24T06:10:43.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6cd7fa700 1 -- 172.21.15.97:0/3197132043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4007310 con 0x7fb6d0073f90 2023-11-24T06:10:43.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.831+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d006d9d0 con 0x7fb6d0073f90 2023-11-24T06:10:43.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.832+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d0068fb0 con 0x7fb6d0073f90 2023-11-24T06:10:43.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.833+0000 7fb6cd7fa700 1 -- 172.21.15.97:0/3197132043 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fb6c4007470 con 0x7fb6d0073f90 2023-11-24T06:10:43.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.833+0000 7fb6cd7fa700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb6b406aff0 0x7fb6b406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:43.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.833+0000 7fb6cd7fa700 1 -- 172.21.15.97:0/3197132043 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb6c40894d0 con 0x7fb6d0073f90 2023-11-24T06:10:43.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.833+0000 7fb6cffff700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb6b406aff0 0x7fb6b406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:43.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.833+0000 7fb6cffff700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb6b406aff0 0x7fb6b406d4b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00adf00 tx=0x7fb6bc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:43.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:43.838+0000 7fb6cd7fa700 1 -- 172.21.15.97:0/3197132043 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6c40544c0 con 0x7fb6d0073f90 2023-11-24T06:10:44.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.055+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp4467", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6d0160540 con 0x7fb6b406aff0 2023-11-24T06:10:44.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.059+0000 7fb6cd7fa700 1 -- 172.21.15.97:0/3197132043 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb6bc011b80 con 0x7fb6b406aff0 2023-11-24T06:10:44.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb6b406aff0 msgr2=0x7fb6b406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb6b406aff0 0x7fb6b406d4b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb6d00adf00 tx=0x7fb6bc008030 comp rx=0 tx=0).stop 2023-11-24T06:10:44.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d0073f90 msgr2=0x7fb6d015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d0073f90 0x7fb6d015dec0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40011e0 tx=0x7fb6c400cae0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 shutdown_connections 2023-11-24T06:10:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb6b406aff0 0x7fb6b406d4b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb6d015f9e0 0x7fb6d00ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d0073f90 0x7fb6d015dec0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 --2- 172.21.15.97:0/3197132043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb6d00cf610 0x7fb6d00b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 >> 172.21.15.97:0/3197132043 conn(0x7fb6d00d0bf0 msgr2=0x7fb6d00b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 shutdown_connections 2023-11-24T06:10:44.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.061+0000 7fb6d6a92700 1 -- 172.21.15.97:0/3197132043 wait complete. 2023-11-24T06:10:44.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:44.076 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T06:10:44.110 INFO:teuthology.orchestra.run.smithi097.stdout:2 2023-11-24T06:10:44.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:44.111 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rm -f ./volumes/subvol_grp4467/subvol1178/.meta.stale_snap_section) 2023-11-24T06:10:44.178 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:10:44.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:10:44.266 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:10:44.310 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:10:44.311 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:44.311 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:44.328 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:44.328 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:44.386 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:44.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.544+0000 7f037d518700 1 Processor -- start 2023-11-24T06:10:44.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.544+0000 7f037d518700 1 -- start start 2023-11-24T06:10:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f037d518700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f037815a3b0 0x7f03780b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f037d518700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f037815deb0 0x7f037815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f037d518700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f037815a790 con 0x7f03780cf360 2023-11-24T06:10:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f037d518700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03780cf8b0 con 0x7f037815deb0 2023-11-24T06:10:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f037d518700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03780b4c40 con 0x7f037815a3b0 2023-11-24T06:10:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f03777fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f037815deb0 0x7f037815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f03767fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f03780cf360 0x7f03780b3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40830/0 (socket says 172.21.15.97:40830) 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f03767fc700 1 -- 172.21.15.97:0/529069999 learned_addr learned my addr 172.21.15.97:0/529069999 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f0376ffd700 1 --2- 172.21.15.97:0/529069999 >> v2:172.21.15.183:3300/0 conn(0x7f037815a3b0 0x7f03780b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f03777fe700 1 -- 172.21.15.97:0/529069999 >> v2:172.21.15.183:3300/0 conn(0x7f037815a3b0 msgr2=0x7f03780b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f03777fe700 1 --2- 172.21.15.97:0/529069999 >> v2:172.21.15.183:3300/0 conn(0x7f037815a3b0 0x7f03780b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f03777fe700 1 -- 172.21.15.97:0/529069999 >> v1:172.21.15.97:6789/0 conn(0x7f03780cf360 legacy=0x7f03780b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f0375ffb700 1 -- 172.21.15.97:0/529069999 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718362292 0 0) 0x7f037815a790 con 0x7f03780cf360 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f03777fe700 1 -- 172.21.15.97:0/529069999 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0378070c60 con 0x7f037815deb0 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f0376ffd700 1 --2- 172.21.15.97:0/529069999 >> v2:172.21.15.183:3300/0 conn(0x7f037815a3b0 0x7f03780b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f03777fe700 1 --2- 172.21.15.97:0/529069999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037815deb0 0x7f037815e340 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f037804fca0 tx=0x7f037000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8924e44d857fff1f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.545+0000 7f0375ffb700 1 -- 172.21.15.97:0/529069999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037000e070 con 0x7f037815deb0 2023-11-24T06:10:44.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f0375ffb700 1 -- 172.21.15.97:0/529069999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037000bbd0 con 0x7f037815deb0 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f0375ffb700 1 -- 172.21.15.97:0/529069999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370012430 con 0x7f037815deb0 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f037d518700 1 -- 172.21.15.97:0/529069999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037815deb0 msgr2=0x7f037815e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f037d518700 1 --2- 172.21.15.97:0/529069999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037815deb0 0x7f037815e340 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f037804fca0 tx=0x7f037000b330 comp rx=0 tx=0).stop 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f037d518700 1 -- 172.21.15.97:0/529069999 shutdown_connections 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f037d518700 1 --2- 172.21.15.97:0/529069999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f037815deb0 0x7f037815e340 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f037d518700 1 --2- 172.21.15.97:0/529069999 >> v2:172.21.15.183:3300/0 conn(0x7f037815a3b0 0x7f03780b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f037d518700 1 -- 172.21.15.97:0/529069999 >> 172.21.15.97:0/529069999 conn(0x7f03780b1740 msgr2=0x7f03780b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f037d518700 1 -- 172.21.15.97:0/529069999 shutdown_connections 2023-11-24T06:10:44.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.546+0000 7f037d518700 1 -- 172.21.15.97:0/529069999 wait complete. 2023-11-24T06:10:44.553 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.547+0000 7f037d518700 1 Processor -- start 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.547+0000 7f037d518700 1 -- start start 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f037d518700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03780cf360 0x7f0378156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f037d518700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f037815a3b0 0x7f03780cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f037d518700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f037815deb0 0x7f03780ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f037d518700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03781469d0 con 0x7f03780cf360 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f037d518700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037815be10 con 0x7f037815a3b0 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f037d518700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037815c0f0 con 0x7f037815deb0 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f0376ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03780cf360 0x7f0378156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f0376ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03780cf360 0x7f0378156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48384/0 (socket says 172.21.15.97:48384) 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f0376ffd700 1 -- 172.21.15.97:0/938915843 learned_addr learned my addr 172.21.15.97:0/938915843 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:44.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f0376ffd700 1 -- 172.21.15.97:0/938915843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f037815deb0 msgr2=0x7f03780ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f0376ffd700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f037815deb0 0x7f03780ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f0376ffd700 1 -- 172.21.15.97:0/938915843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f037815a3b0 msgr2=0x7f03780cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.548+0000 7f0376ffd700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f037815a3b0 0x7f03780cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.549+0000 7f0376ffd700 1 -- 172.21.15.97:0/938915843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0370009a70 con 0x7f03780cf360 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.549+0000 7f0376ffd700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03780cf360 0x7f0378156fa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f036c00c7f0 tx=0x7f036c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.549+0000 7f035f7fe700 1 -- 172.21.15.97:0/938915843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c011800 con 0x7f03780cf360 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.549+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0378146c90 con 0x7f03780cf360 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.549+0000 7f035f7fe700 1 -- 172.21.15.97:0/938915843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036c011960 con 0x7f03780cf360 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.549+0000 7f035f7fe700 1 -- 172.21.15.97:0/938915843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c011d10 con 0x7f03780cf360 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.549+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03781574e0 con 0x7f03780cf360 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.550+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037815c2e0 con 0x7f03780cf360 2023-11-24T06:10:44.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.550+0000 7f035f7fe700 1 -- 172.21.15.97:0/938915843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f036c018360 con 0x7f03780cf360 2023-11-24T06:10:44.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.550+0000 7f035f7fe700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f036406aff0 0x7f036406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.551+0000 7f035f7fe700 1 -- 172.21.15.97:0/938915843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f036c089950 con 0x7f03780cf360 2023-11-24T06:10:44.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.551+0000 7f03767fc700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f036406aff0 0x7f036406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.551+0000 7f03767fc700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f036406aff0 0x7f036406d4b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f03780cf300 tx=0x7f03680079f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:44.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.555+0000 7f035f7fe700 1 -- 172.21.15.97:0/938915843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f036c054720 con 0x7f03780cf360 2023-11-24T06:10:44.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.753+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03780751e0 con 0x7f03780cf360 2023-11-24T06:10:44.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.754+0000 7f035f7fe700 1 -- 172.21.15.97:0/938915843 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16677 (secure 0 0 0) 0x7f036c057d40 con 0x7f03780cf360 2023-11-24T06:10:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:44.757 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:40.568208+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:25.335173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:26.547561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.37","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:44.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.756+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f036406aff0 msgr2=0x7f036406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.756+0000 7f037d518700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f036406aff0 0x7f036406d4b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f03780cf300 tx=0x7f03680079f0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.756+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03780cf360 msgr2=0x7f0378156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.756+0000 7f037d518700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03780cf360 0x7f0378156fa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f036c00c7f0 tx=0x7f036c00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.757+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 shutdown_connections 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.757+0000 7f037d518700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f036406aff0 0x7f036406d4b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.757+0000 7f037d518700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f037815deb0 0x7f03780ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.757+0000 7f037d518700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03780cf360 0x7f0378156fa0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.757+0000 7f037d518700 1 --2- 172.21.15.97:0/938915843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f037815a3b0 0x7f03780cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.757+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 >> 172.21.15.97:0/938915843 conn(0x7f03780b1740 msgr2=0x7f037806e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.757+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 shutdown_connections 2023-11-24T06:10:44.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.757+0000 7f037d518700 1 -- 172.21.15.97:0/938915843 wait complete. 2023-11-24T06:10:44.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:25.335173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:26.547561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:44.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.892+0000 7ff1f99c1700 1 Processor -- start 2023-11-24T06:10:44.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.892+0000 7ff1f99c1700 1 -- start start 2023-11-24T06:10:44.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.893+0000 7ff1f99c1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f40746a0 0x7ff1f40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.893+0000 7ff1f99c1700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff1f40afc90 0x7ff1f406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.893+0000 7ff1f99c1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f40b3c60 con 0x7ff1f415e160 2023-11-24T06:10:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.893+0000 7ff1f99c1700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f40b4500 con 0x7ff1f40afc90 2023-11-24T06:10:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.893+0000 7ff1f99c1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f40b58c0 con 0x7ff1f40746a0 2023-11-24T06:10:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.893+0000 7ff1f37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff1f40afc90 0x7ff1f406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.893+0000 7ff1f37fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff1f40afc90 0x7ff1f406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48394/0 (socket says 172.21.15.97:48394) 2023-11-24T06:10:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.893+0000 7ff1f37fe700 1 -- 172.21.15.97:0/423126554 learned_addr learned my addr 172.21.15.97:0/423126554 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:44.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f27fc700 1 --2- 172.21.15.97:0/423126554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f40746a0 0x7ff1f40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f37fe700 1 -- 172.21.15.97:0/423126554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f40746a0 msgr2=0x7ff1f40b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f37fe700 1 --2- 172.21.15.97:0/423126554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f40746a0 0x7ff1f40b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f37fe700 1 -- 172.21.15.97:0/423126554 >> v1:172.21.15.183:6789/0 conn(0x7ff1f415e160 legacy=0x7ff1f4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f37fe700 1 -- 172.21.15.97:0/423126554 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f40b0140 con 0x7ff1f40afc90 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f37fe700 1 --2- 172.21.15.97:0/423126554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40afc90 0x7ff1f406d930 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009e50 tx=0x7ff1ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e1fb9af7157117 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f1ffb700 1 -- 172.21.15.97:0/423126554 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00e070 con 0x7ff1f40afc90 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f1ffb700 1 -- 172.21.15.97:0/423126554 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ec00b9c0 con 0x7ff1f40afc90 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f1ffb700 1 -- 172.21.15.97:0/423126554 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00bd70 con 0x7ff1f40afc90 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f99c1700 1 -- 172.21.15.97:0/423126554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40afc90 msgr2=0x7ff1f406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.894+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/423126554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40afc90 0x7ff1f406d930 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec009e50 tx=0x7ff1ec00b330 comp rx=0 tx=0).stop 2023-11-24T06:10:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.895+0000 7ff1f99c1700 1 -- 172.21.15.97:0/423126554 shutdown_connections 2023-11-24T06:10:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.895+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/423126554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40afc90 0x7ff1f406d930 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.895+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/423126554 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f40746a0 0x7ff1f40b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.895+0000 7ff1f99c1700 1 -- 172.21.15.97:0/423126554 >> 172.21.15.97:0/423126554 conn(0x7ff1f40d0bf0 msgr2=0x7ff1f40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.895+0000 7ff1f99c1700 1 -- 172.21.15.97:0/423126554 shutdown_connections 2023-11-24T06:10:44.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.895+0000 7ff1f99c1700 1 -- 172.21.15.97:0/423126554 wait complete. 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.896+0000 7ff1f99c1700 1 Processor -- start 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.896+0000 7ff1f99c1700 1 -- start start 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f99c1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40746a0 0x7ff1f41f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f99c1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f40afc90 0x7ff1f414a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f99c1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f415e160 0x7ff1f40add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f99c1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f414a400 con 0x7ff1f40746a0 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f99c1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f41f7dc0 con 0x7ff1f40afc90 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f99c1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f414cca0 con 0x7ff1f415e160 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40746a0 0x7ff1f41f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40746a0 0x7ff1f41f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48396/0 (socket says 172.21.15.97:48396) 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 -- 172.21.15.97:0/877974657 learned_addr learned my addr 172.21.15.97:0/877974657 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f27fc700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f40afc90 0x7ff1f414a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f37fe700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f415e160 0x7ff1f40add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 -- 172.21.15.97:0/877974657 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f415e160 msgr2=0x7ff1f40add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f415e160 0x7ff1f40add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 -- 172.21.15.97:0/877974657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f40afc90 msgr2=0x7ff1f414a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f40afc90 0x7ff1f414a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 -- 172.21.15.97:0/877974657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec009b30 con 0x7ff1f40746a0 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f2ffd700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40746a0 0x7ff1f41f7880 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c450 tx=0x7ff1e800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1dbfff700 1 -- 172.21.15.97:0/877974657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80052d0 con 0x7ff1f40746a0 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1dbfff700 1 -- 172.21.15.97:0/877974657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e8005430 con 0x7ff1f40746a0 2023-11-24T06:10:44.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f414cf80 con 0x7ff1f40746a0 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1dbfff700 1 -- 172.21.15.97:0/877974657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80038c0 con 0x7ff1f40746a0 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.897+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f414d540 con 0x7ff1f40746a0 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.898+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f414d6d0 con 0x7ff1f40746a0 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.899+0000 7ff1dbfff700 1 -- 172.21.15.97:0/877974657 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7ff1e8003a20 con 0x7ff1f40746a0 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.899+0000 7ff1dbfff700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff1e006aff0 0x7ff1e006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.899+0000 7ff1dbfff700 1 -- 172.21.15.97:0/877974657 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff1e8088af0 con 0x7ff1f40746a0 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.899+0000 7ff1f27fc700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff1e006aff0 0x7ff1e006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.900+0000 7ff1f27fc700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff1e006aff0 0x7ff1e006d4b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009930 tx=0x7ff1e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:44.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:44.904+0000 7ff1dbfff700 1 -- 172.21.15.97:0/877974657 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1e8053a50 con 0x7ff1f40746a0 2023-11-24T06:10:45.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.093+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1f4068fb0 con 0x7ff1f40746a0 2023-11-24T06:10:45.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.093+0000 7ff1dbfff700 1 -- 172.21.15.97:0/877974657 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1e8057070 con 0x7ff1f40746a0 2023-11-24T06:10:45.096 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff1e006aff0 msgr2=0x7ff1e006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff1e006aff0 0x7ff1e006d4b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009930 tx=0x7ff1e4008040 comp rx=0 tx=0).stop 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40746a0 msgr2=0x7ff1f41f7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40746a0 0x7ff1f41f7880 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c450 tx=0x7ff1e800c810 comp rx=0 tx=0).stop 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 shutdown_connections 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff1e006aff0 0x7ff1e006d4b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1f415e160 0x7ff1f40add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1f40746a0 0x7ff1f41f7880 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 --2- 172.21.15.97:0/877974657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1f40afc90 0x7ff1f414a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 >> 172.21.15.97:0/877974657 conn(0x7ff1f40d0bf0 msgr2=0x7ff1f4073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:45.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 shutdown_connections 2023-11-24T06:10:45.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.095+0000 7ff1f99c1700 1 -- 172.21.15.97:0/877974657 wait complete. 2023-11-24T06:10:45.110 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:45.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.230+0000 7f078e143700 1 Processor -- start 2023-11-24T06:10:45.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.230+0000 7f078e143700 1 -- start start 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f078e143700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f078815bf50 0x7f07880ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f078e143700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07880d0140 con 0x7f07880ce830 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f078e143700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07880cf3c0 con 0x7f07880c6c30 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f078e143700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078815c330 con 0x7f078815bf50 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f0786ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f07880ce830 0x7f07880cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40856/0 (socket says 172.21.15.97:40856) 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f0786ffd700 1 -- 172.21.15.97:0/23415292 learned_addr learned my addr 172.21.15.97:0/23415292 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570959037 0 0) 0x7f07880d0140 con 0x7f07880ce830 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07740035b0 con 0x7f07880ce830 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f07877fe700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f078815bf50 0x7f07880ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.231+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2493305231 0 0) 0x7f07740035b0 con 0x7f07880ce830 2023-11-24T06:10:45.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f078815bf50 msgr2=0x7f07880ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f078815bf50 0x7f07880ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 >> v1:172.21.15.183:6789/0 conn(0x7f07880c6c30 legacy=0x7f07880cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880d03e0 con 0x7f07880ce830 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f0778001670 con 0x7f07880ce830 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 >> v1:172.21.15.97:6789/0 conn(0x7f07880ce830 legacy=0x7f07880cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0774003a30 0x7f0774003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774004850 0x7f0774004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f078815bf50 0x7f0774005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078815c330 con 0x7f0774003a30 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077400bac0 con 0x7f0774004850 2023-11-24T06:10:45.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077400bc80 con 0x7f078815bf50 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07877fe700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0774003a30 0x7f0774003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f0786ffd700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774004850 0x7f0774004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f0787fff700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f078815bf50 0x7f0774005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07877fe700 1 -- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f078815bf50 msgr2=0x7f0774005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07877fe700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f078815bf50 0x7f0774005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07877fe700 1 -- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774004850 msgr2=0x7f0774004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07877fe700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774004850 0x7f0774004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07877fe700 1 -- 172.21.15.97:0/23415292 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07740035b0 con 0x7f0774003a30 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07877fe700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0774003a30 0x7f0774003e50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0770009b30 tx=0x7f0774005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3969daddc67a9e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077000e6c0 con 0x7f0774003a30 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077000e820 con 0x7f0774003a30 2023-11-24T06:10:45.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.232+0000 7f07867fc700 1 -- 172.21.15.97:0/23415292 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077000eb90 con 0x7f0774003a30 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.233+0000 7f078e143700 1 -- 172.21.15.97:0/23415292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0774003a30 msgr2=0x7f0774003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.233+0000 7f078e143700 1 --2- 172.21.15.97:0/23415292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0774003a30 0x7f0774003e50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0770009b30 tx=0x7f0774005880 comp rx=0 tx=0).stop 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.233+0000 7f0787fff700 1 -- 172.21.15.97:0/23415292 reap_dead start 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.233+0000 7f078e143700 1 -- 172.21.15.97:0/23415292 shutdown_connections 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.233+0000 7f078e143700 1 -- 172.21.15.97:0/23415292 >> 172.21.15.97:0/23415292 conn(0x7f07880b3780 msgr2=0x7f07880b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.233+0000 7f078e143700 1 -- 172.21.15.97:0/23415292 shutdown_connections 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.233+0000 7f078e143700 1 -- 172.21.15.97:0/23415292 wait complete. 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.234+0000 7f078e143700 1 Processor -- start 2023-11-24T06:10:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.234+0000 7f078e143700 1 -- start start 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f078e143700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774003a30 0x7f07880c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f078e143700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0788155440 0x7f07880b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f078e143700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07880cad30 0x7f07880cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f078e143700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07880cb720 con 0x7f07880cad30 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f078e143700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07880c9090 con 0x7f0774003a30 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f078e143700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07880b6190 con 0x7f0788155440 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0787fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07880cad30 0x7f07880cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0787fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07880cad30 0x7f07880cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48418/0 (socket says 172.21.15.97:48418) 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0787fff700 1 -- 172.21.15.97:0/1599317274 learned_addr learned my addr 172.21.15.97:0/1599317274 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f07877fe700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774003a30 0x7f07880c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0786ffd700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0788155440 0x7f07880b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0787fff700 1 -- 172.21.15.97:0/1599317274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0788155440 msgr2=0x7f07880b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0787fff700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0788155440 0x7f07880b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0787fff700 1 -- 172.21.15.97:0/1599317274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774003a30 msgr2=0x7f07880c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0787fff700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774003a30 0x7f07880c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.235+0000 7f0787fff700 1 -- 172.21.15.97:0/1599317274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880b5bd0 con 0x7f07880cad30 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.236+0000 7f0787fff700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07880cad30 0x7f07880cb1e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0780000f80 tx=0x7f078000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.236+0000 7f077ffff700 1 -- 172.21.15.97:0/1599317274 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780003c10 con 0x7f07880cad30 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.236+0000 7f077ffff700 1 -- 172.21.15.97:0/1599317274 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0780003d70 con 0x7f07880cad30 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.236+0000 7f077ffff700 1 -- 172.21.15.97:0/1599317274 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780005310 con 0x7f07880cad30 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.236+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07880b5eb0 con 0x7f07880cad30 2023-11-24T06:10:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.236+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0788156660 con 0x7f07880cad30 2023-11-24T06:10:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.237+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0788068fb0 con 0x7f07880cad30 2023-11-24T06:10:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.237+0000 7f077ffff700 1 -- 172.21.15.97:0/1599317274 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f0780005470 con 0x7f07880cad30 2023-11-24T06:10:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.237+0000 7f077ffff700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f076806aff0 0x7f076806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.238+0000 7f07877fe700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f076806aff0 0x7f076806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.238+0000 7f077ffff700 1 -- 172.21.15.97:0/1599317274 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0780089640 con 0x7f07880cad30 2023-11-24T06:10:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.238+0000 7f07877fe700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f076806aff0 0x7f076806d4b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07700090b0 tx=0x7f077001f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.242+0000 7f077ffff700 1 -- 172.21.15.97:0/1599317274 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0780054630 con 0x7f07880cad30 2023-11-24T06:10:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.432+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07880c91d0 con 0x7f07880cad30 2023-11-24T06:10:45.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.432+0000 7f077ffff700 1 -- 172.21.15.97:0/1599317274 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0780057c50 con 0x7f07880cad30 2023-11-24T06:10:45.435 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:10:45.437 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f076806aff0 msgr2=0x7f076806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f076806aff0 0x7f076806d4b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f07700090b0 tx=0x7f077001f040 comp rx=0 tx=0).stop 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07880cad30 msgr2=0x7f07880cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07880cad30 0x7f07880cb1e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0780000f80 tx=0x7f078000edd0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 shutdown_connections 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07880cad30 0x7f07880cb1e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0774003a30 0x7f07880c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f076806aff0 0x7f076806d4b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 --2- 172.21.15.97:0/1599317274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0788155440 0x7f07880b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.434+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 >> 172.21.15.97:0/1599317274 conn(0x7f07880b3780 msgr2=0x7f078806ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.435+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 shutdown_connections 2023-11-24T06:10:45.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.435+0000 7f078e143700 1 -- 172.21.15.97:0/1599317274 wait complete. 2023-11-24T06:10:45.449 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:45.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.571+0000 7f145f9a7700 1 Processor -- start 2023-11-24T06:10:45.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.571+0000 7f145f9a7700 1 -- start start 2023-11-24T06:10:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145f9a7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f145815e160 0x7f14580b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145f9a7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14580b3c60 con 0x7f14580afc90 2023-11-24T06:10:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145f9a7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14580b58c0 con 0x7f14580746a0 2023-11-24T06:10:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145f9a7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14580b4500 con 0x7f145815e160 2023-11-24T06:10:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145d743700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f145815e160 0x7f14580b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145cf42700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f14580afc90 0x7f1458073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40874/0 (socket says 172.21.15.97:40874) 2023-11-24T06:10:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145cf42700 1 -- 172.21.15.97:0/4071053562 learned_addr learned my addr 172.21.15.97:0/4071053562 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:45.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145d743700 1 --2- 172.21.15.97:0/4071053562 >> v2:172.21.15.97:3300/0 conn(0x7f145815e160 0x7f14580b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48422/0 (socket says 172.21.15.97:48422) 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145d743700 1 -- 172.21.15.97:0/4071053562 >> v1:172.21.15.183:6789/0 conn(0x7f14580746a0 legacy=0x7f145806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145d743700 1 -- 172.21.15.97:0/4071053562 >> v1:172.21.15.97:6789/0 conn(0x7f14580afc90 legacy=0x7f1458073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145d743700 1 -- 172.21.15.97:0/4071053562 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14580b0140 con 0x7f145815e160 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f144ffff700 1 -- 172.21.15.97:0/4071053562 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605018321 0 0) 0x7f14580b3c60 con 0x7f14580afc90 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.572+0000 7f145d743700 1 --2- 172.21.15.97:0/4071053562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f145815e160 0x7f14580b5da0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1448009840 tx=0x7f1448009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fe174d796228ab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f144ffff700 1 -- 172.21.15.97:0/4071053562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1448004230 con 0x7f145815e160 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f144ffff700 1 -- 172.21.15.97:0/4071053562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1448004390 con 0x7f145815e160 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f144ffff700 1 -- 172.21.15.97:0/4071053562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14480036a0 con 0x7f145815e160 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f145f9a7700 1 -- 172.21.15.97:0/4071053562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f145815e160 msgr2=0x7f14580b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f145f9a7700 1 --2- 172.21.15.97:0/4071053562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f145815e160 0x7f14580b5da0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1448009840 tx=0x7f1448009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f145f9a7700 1 -- 172.21.15.97:0/4071053562 shutdown_connections 2023-11-24T06:10:45.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f145f9a7700 1 --2- 172.21.15.97:0/4071053562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f145815e160 0x7f14580b5da0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f145f9a7700 1 -- 172.21.15.97:0/4071053562 >> 172.21.15.97:0/4071053562 conn(0x7f14580d0bf0 msgr2=0x7f14580d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:45.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f145f9a7700 1 -- 172.21.15.97:0/4071053562 shutdown_connections 2023-11-24T06:10:45.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.573+0000 7f145f9a7700 1 -- 172.21.15.97:0/4071053562 wait complete. 2023-11-24T06:10:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.574+0000 7f145f9a7700 1 Processor -- start 2023-11-24T06:10:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.574+0000 7f145f9a7700 1 -- start start 2023-11-24T06:10:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145f9a7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14580746a0 0x7f145814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145f9a7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14580afc90 0x7f145814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145f9a7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f145815e160 0x7f14581619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145f9a7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145814acc0 con 0x7f14580afc90 2023-11-24T06:10:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145f9a7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145814b9a0 con 0x7f145815e160 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145f9a7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145814bc80 con 0x7f14580746a0 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145cf42700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14580afc90 0x7f145814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145cf42700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14580afc90 0x7f145814af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48428/0 (socket says 172.21.15.97:48428) 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145cf42700 1 -- 172.21.15.97:0/2836791144 learned_addr learned my addr 172.21.15.97:0/2836791144 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145d743700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14580746a0 0x7f145814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.575+0000 7f145df44700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f145815e160 0x7f14581619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f145cf42700 1 -- 172.21.15.97:0/2836791144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14580746a0 msgr2=0x7f145814a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f145cf42700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14580746a0 0x7f145814a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f145cf42700 1 -- 172.21.15.97:0/2836791144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f145815e160 msgr2=0x7f14581619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f145cf42700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f145815e160 0x7f14581619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f145cf42700 1 -- 172.21.15.97:0/2836791144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14480094a0 con 0x7f14580afc90 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f145cf42700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14580afc90 0x7f145814af50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f14540011e0 tx=0x7f145400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f144dffb700 1 -- 172.21.15.97:0/2836791144 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454004c80 con 0x7f14580afc90 2023-11-24T06:10:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f144dffb700 1 -- 172.21.15.97:0/2836791144 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454004e00 con 0x7f14580afc90 2023-11-24T06:10:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145814bf60 con 0x7f14580afc90 2023-11-24T06:10:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f144dffb700 1 -- 172.21.15.97:0/2836791144 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454007310 con 0x7f14580afc90 2023-11-24T06:10:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.576+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145814c520 con 0x7f14580afc90 2023-11-24T06:10:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.577+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145814c6b0 con 0x7f14580afc90 2023-11-24T06:10:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.577+0000 7f144dffb700 1 -- 172.21.15.97:0/2836791144 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f1454007470 con 0x7f14580afc90 2023-11-24T06:10:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.578+0000 7f144dffb700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f143c06aff0 0x7f143c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.578+0000 7f144dffb700 1 -- 172.21.15.97:0/2836791144 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f14540893d0 con 0x7f14580afc90 2023-11-24T06:10:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.578+0000 7f145d743700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f143c06aff0 0x7f143c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.578+0000 7f145d743700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f143c06aff0 0x7f143c06d4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14480090b0 tx=0x7f1448008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.582+0000 7f144dffb700 1 -- 172.21.15.97:0/2836791144 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1454054330 con 0x7f14580afc90 2023-11-24T06:10:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.600+0000 7f144dffb700 1 -- 172.21.15.97:0/2836791144 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f14540878b0 con 0x7f14580afc90 2023-11-24T06:10:45.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.776+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f14580b2780 con 0x7f14580afc90 2023-11-24T06:10:45.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.777+0000 7f144dffb700 1 -- 172.21.15.97:0/2836791144 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f14540739a0 con 0x7f14580afc90 2023-11-24T06:10:45.790 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:10:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f143c06aff0 msgr2=0x7f143c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f143c06aff0 0x7f143c06d4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14480090b0 tx=0x7f1448008030 comp rx=0 tx=0).stop 2023-11-24T06:10:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14580afc90 msgr2=0x7f145814af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14580afc90 0x7f145814af50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f14540011e0 tx=0x7f145400ca30 comp rx=0 tx=0).stop 2023-11-24T06:10:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 shutdown_connections 2023-11-24T06:10:45.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f145815e160 0x7f14581619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f143c06aff0 0x7f143c06d4b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f14580746a0 0x7f145814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 --2- 172.21.15.97:0/2836791144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f14580afc90 0x7f145814af50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 >> 172.21.15.97:0/2836791144 conn(0x7f14580d0bf0 msgr2=0x7f145815d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:45.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 shutdown_connections 2023-11-24T06:10:45.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.779+0000 7f145f9a7700 1 -- 172.21.15.97:0/2836791144 wait complete. 2023-11-24T06:10:45.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:10:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.911+0000 7fe3c45ba700 1 Processor -- start 2023-11-24T06:10:45.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.912+0000 7fe3c45ba700 1 -- start start 2023-11-24T06:10:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.912+0000 7fe3c45ba700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc06c710 0x7fe3bc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.912+0000 7fe3c45ba700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe3bc0af4d0 0x7fe3bc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c45ba700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc0ae430 con 0x7fe3bc15e080 2023-11-24T06:10:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c45ba700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc0af0b0 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c45ba700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0b05a0 con 0x7fe3bc06c710 2023-11-24T06:10:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c2b57700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe3bc0af4d0 0x7fe3bc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c2356700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fe3bc15e080 0x7fe3bc06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:50208/0 (socket says 172.21.15.97:50208) 2023-11-24T06:10:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c1b55700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc06c710 0x7fe3bc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c2356700 1 -- 172.21.15.97:0/3503591567 learned_addr learned my addr 172.21.15.97:0/3503591567 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c2b57700 1 -- 172.21.15.97:0/3503591567 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc06c710 msgr2=0x7fe3bc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c2b57700 1 --2- 172.21.15.97:0/3503591567 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc06c710 0x7fe3bc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c2b57700 1 -- 172.21.15.97:0/3503591567 >> v1:172.21.15.183:6789/0 conn(0x7fe3bc15e080 legacy=0x7fe3bc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c2b57700 1 -- 172.21.15.97:0/3503591567 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc0ae710 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.913+0000 7fe3c2b57700 1 --2- 172.21.15.97:0/3503591567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc0afb90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009dc0 tx=0x7fe3b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da42fea4dfaf7b11 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c1354700 1 -- 172.21.15.97:0/3503591567 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400e070 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c1354700 1 -- 172.21.15.97:0/3503591567 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b400bae0 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c1354700 1 -- 172.21.15.97:0/3503591567 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400be90 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3503591567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc0af4d0 msgr2=0x7fe3bc0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3503591567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc0afb90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009dc0 tx=0x7fe3b400b330 comp rx=0 tx=0).stop 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3503591567 shutdown_connections 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3503591567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc0afb90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3503591567 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc06c710 0x7fe3bc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3503591567 >> 172.21.15.97:0/3503591567 conn(0x7fe3bc071b90 msgr2=0x7fe3bc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3503591567 shutdown_connections 2023-11-24T06:10:45.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.914+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3503591567 wait complete. 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.915+0000 7fe3c45ba700 1 Processor -- start 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.915+0000 7fe3c45ba700 1 -- start start 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c45ba700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc06c710 0x7fe3bc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c45ba700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc1f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c45ba700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc15e080 0x7fe3bc0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c45ba700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0d02e0 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c45ba700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0cfb00 con 0x7fe3bc06c710 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c45ba700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc157ed0 con 0x7fe3bc15e080 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c1b55700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc1f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c1b55700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc1f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48442/0 (socket says 172.21.15.97:48442) 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c1b55700 1 -- 172.21.15.97:0/3245229625 learned_addr learned my addr 172.21.15.97:0/3245229625 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c2356700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc06c710 0x7fe3bc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.916+0000 7fe3c2b57700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc15e080 0x7fe3bc0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3c1b55700 1 -- 172.21.15.97:0/3245229625 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc15e080 msgr2=0x7fe3bc0b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3c1b55700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc15e080 0x7fe3bc0b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3c1b55700 1 -- 172.21.15.97:0/3245229625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc06c710 msgr2=0x7fe3bc14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3c1b55700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc06c710 0x7fe3bc14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3c1b55700 1 -- 172.21.15.97:0/3245229625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4009a70 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3c1b55700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc1f7880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800e9e0 tx=0x7fe3b800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3b2ffd700 1 -- 172.21.15.97:0/3245229625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8009710 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3b2ffd700 1 -- 172.21.15.97:0/3245229625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b800ee80 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.922 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc14df50 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3b2ffd700 1 -- 172.21.15.97:0/3245229625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800ca50 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.917+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc1f7e10 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.918+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc1581b0 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.918+0000 7fe3b2ffd700 1 -- 172.21.15.97:0/3245229625 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe3b800cbb0 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.919+0000 7fe3b2ffd700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe3a006b110 0x7fe3a006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.919+0000 7fe3b2ffd700 1 -- 172.21.15.97:0/3245229625 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe3b8089990 con 0x7fe3bc0af4d0 2023-11-24T06:10:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.919+0000 7fe3c2356700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe3a006b110 0x7fe3a006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:45.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.919+0000 7fe3c2356700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe3a006b110 0x7fe3a006d5d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac005910 tx=0x7fe3ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:45.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:45.923+0000 7fe3b2ffd700 1 -- 172.21.15.97:0/3245229625 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3b8054730 con 0x7fe3bc0af4d0 2023-11-24T06:10:46.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.129+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe3bc0753f0 con 0x7fe3bc0af4d0 2023-11-24T06:10:46.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.130+0000 7fe3b2ffd700 1 -- 172.21.15.97:0/3245229625 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe3b8021080 con 0x7fe3bc0af4d0 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe3a006b110 msgr2=0x7fe3a006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe3a006b110 0x7fe3a006d5d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac005910 tx=0x7fe3ac009040 comp rx=0 tx=0).stop 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0af4d0 msgr2=0x7fe3bc1f7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc1f7880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800e9e0 tx=0x7fe3b800ecf0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 shutdown_connections 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe3a006b110 0x7fe3a006d5d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe3bc15e080 0x7fe3bc0b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3bc06c710 0x7fe3bc14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 --2- 172.21.15.97:0/3245229625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3bc0af4d0 0x7fe3bc1f7880 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 >> 172.21.15.97:0/3245229625 conn(0x7fe3bc071b90 msgr2=0x7fe3bc06e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:46.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 shutdown_connections 2023-11-24T06:10:46.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.132+0000 7fe3c45ba700 1 -- 172.21.15.97:0/3245229625 wait complete. 2023-11-24T06:10:46.136 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:10:46.146 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:10:46.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:46.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T06:10:46.187 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:10:46.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:46.187 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:10:46.243 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:46.243 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:46.264 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:46.265 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:46.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.443+0000 7fd535017700 1 Processor -- start 2023-11-24T06:10:46.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.443+0000 7fd535017700 1 -- start start 2023-11-24T06:10:46.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd535017700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd53015c160 0x7fd5300622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd535017700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 0x7fd5300afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd535017700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53015c540 con 0x7fd5300b02e0 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd535017700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300b2cf0 con 0x7fd53015c160 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd535017700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300b0770 con 0x7fd5300b2870 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52f59e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd5300b02e0 0x7fd5300ae1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40904/0 (socket says 172.21.15.97:40904) 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52f59e700 1 -- 172.21.15.97:0/3071546191 learned_addr learned my addr 172.21.15.97:0/3071546191 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021004420 0 0) 0x7fd53015c540 con 0x7fd5300b02e0 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52ed9d700 1 --2- 172.21.15.97:0/3071546191 >> v2:172.21.15.183:3300/0 conn(0x7fd53015c160 0x7fd5300622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52e59c700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 0x7fd5300afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5180035b0 con 0x7fd5300b02e0 2023-11-24T06:10:46.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613613393 0 0) 0x7fd5180035b0 con 0x7fd5300b02e0 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 msgr2=0x7fd5300afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 0x7fd5300afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 >> v2:172.21.15.183:3300/0 conn(0x7fd53015c160 msgr2=0x7fd5300622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 --2- 172.21.15.97:0/3071546191 >> v2:172.21.15.183:3300/0 conn(0x7fd53015c160 0x7fd5300622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300b2ee0 con 0x7fd5300b02e0 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.444+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fd5280019f0 con 0x7fd5300b02e0 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 >> v1:172.21.15.97:6789/0 conn(0x7fd5300b02e0 legacy=0x7fd5300ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd518003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 0x7fd518004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd518004d90 0x7fd518005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300b0770 con 0x7fd518003a00 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51800ba10 con 0x7fd518004d90 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51800bb50 con 0x7fd5300b2870 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52ed9d700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd518003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52f59e700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd518004d90 0x7fd518005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52e59c700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 0x7fd518004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52ed9d700 1 -- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 msgr2=0x7fd518004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52ed9d700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 0x7fd518004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52ed9d700 1 -- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd518004d90 msgr2=0x7fd518005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:46.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52ed9d700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd518004d90 0x7fd518005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52ed9d700 1 -- 172.21.15.97:0/3071546191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180035b0 con 0x7fd518003a00 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52e59c700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5300b2870 0x7fd518004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52f59e700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd518004d90 0x7fd518005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52ed9d700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd518003e20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd524009b60 tx=0x7fd518005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0beadf88a272568 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400e6c0 con 0x7fd518003a00 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52400e820 con 0x7fd518003a00 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.445+0000 7fd52dd9b700 1 -- 172.21.15.97:0/3071546191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400eb90 con 0x7fd518003a00 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.446+0000 7fd535017700 1 -- 172.21.15.97:0/3071546191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 msgr2=0x7fd518003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.446+0000 7fd535017700 1 --2- 172.21.15.97:0/3071546191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd518003e20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd524009b60 tx=0x7fd518005880 comp rx=0 tx=0).stop 2023-11-24T06:10:46.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.446+0000 7fd52f59e700 1 -- 172.21.15.97:0/3071546191 reap_dead start 2023-11-24T06:10:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.446+0000 7fd535017700 1 -- 172.21.15.97:0/3071546191 shutdown_connections 2023-11-24T06:10:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.446+0000 7fd535017700 1 -- 172.21.15.97:0/3071546191 >> 172.21.15.97:0/3071546191 conn(0x7fd5300b57b0 msgr2=0x7fd5300b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.446+0000 7fd535017700 1 -- 172.21.15.97:0/3071546191 shutdown_connections 2023-11-24T06:10:46.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.446+0000 7fd535017700 1 -- 172.21.15.97:0/3071546191 wait complete. 2023-11-24T06:10:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.447+0000 7fd535017700 1 Processor -- start 2023-11-24T06:10:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.447+0000 7fd535017700 1 -- start start 2023-11-24T06:10:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd535017700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd53015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd535017700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5301f4820 0x7fd53013e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd535017700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53013e9a0 0x7fd5300ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd535017700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53014de30 con 0x7fd518003a00 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd535017700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300cbc70 con 0x7fd53013e9a0 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd535017700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300cbe80 con 0x7fd5301f4820 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd53015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd53015cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48460/0 (socket says 172.21.15.97:48460) 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52ed9d700 1 -- 172.21.15.97:0/2572249290 learned_addr learned my addr 172.21.15.97:0/2572249290 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52f59e700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53013e9a0 0x7fd5300ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52e59c700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5301f4820 0x7fd53013e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52ed9d700 1 -- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5301f4820 msgr2=0x7fd53013e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52ed9d700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5301f4820 0x7fd53013e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52ed9d700 1 -- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53013e9a0 msgr2=0x7fd5300ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52ed9d700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53013e9a0 0x7fd5300ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52ed9d700 1 -- 172.21.15.97:0/2572249290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300cc160 con 0x7fd518003a00 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.448+0000 7fd52f59e700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53013e9a0 0x7fd5300ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.449+0000 7fd52ed9d700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd53015cc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd5180046e0 tx=0x7fd5240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.449+0000 7fd5177fe700 1 -- 172.21.15.97:0/2572249290 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400e6c0 con 0x7fd518003a00 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.449+0000 7fd5177fe700 1 -- 172.21.15.97:0/2572249290 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524003ab0 con 0x7fd518003a00 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.449+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5300cc440 con 0x7fd518003a00 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.449+0000 7fd5177fe700 1 -- 172.21.15.97:0/2572249290 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524003e60 con 0x7fd518003a00 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.449+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5301f4ca0 con 0x7fd518003a00 2023-11-24T06:10:46.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.450+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5301f4e30 con 0x7fd518003a00 2023-11-24T06:10:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.450+0000 7fd5177fe700 1 -- 172.21.15.97:0/2572249290 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd524013470 con 0x7fd518003a00 2023-11-24T06:10:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.451+0000 7fd5177fe700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd51c06b110 0x7fd51c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.451+0000 7fd5177fe700 1 -- 172.21.15.97:0/2572249290 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd52408b3f0 con 0x7fd518003a00 2023-11-24T06:10:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.451+0000 7fd52e59c700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd51c06b110 0x7fd51c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.451+0000 7fd52e59c700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd51c06b110 0x7fd51c06d5d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd53009e590 tx=0x7fd520008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:46.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.455+0000 7fd5177fe700 1 -- 172.21.15.97:0/2572249290 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd524056190 con 0x7fd518003a00 2023-11-24T06:10:46.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.654+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd530067f60 con 0x7fd518003a00 2023-11-24T06:10:46.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.655+0000 7fd5177fe700 1 -- 172.21.15.97:0/2572249290 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16677 (secure 0 0 0) 0x7fd5240597b0 con 0x7fd518003a00 2023-11-24T06:10:46.658 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:40.568208+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:25.335173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:26.547561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.37","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd51c06b110 msgr2=0x7fd51c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd51c06b110 0x7fd51c06d5d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd53009e590 tx=0x7fd520008040 comp rx=0 tx=0).stop 2023-11-24T06:10:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 msgr2=0x7fd53015cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd53015cc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd5180046e0 tx=0x7fd5240047c0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 shutdown_connections 2023-11-24T06:10:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd53013e9a0 0x7fd5300ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd518003a00 0x7fd53015cc60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd51c06b110 0x7fd51c06d5d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 --2- 172.21.15.97:0/2572249290 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5301f4820 0x7fd53013e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 >> 172.21.15.97:0/2572249290 conn(0x7fd5300b57b0 msgr2=0x7fd5300ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 shutdown_connections 2023-11-24T06:10:46.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.657+0000 7fd535017700 1 -- 172.21.15.97:0/2572249290 wait complete. 2023-11-24T06:10:46.672 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:25.335173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:26.547561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:46.673 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.793+0000 7f671b6cd700 1 Processor -- start 2023-11-24T06:10:46.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.793+0000 7f671b6cd700 1 -- start start 2023-11-24T06:10:46.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f671b6cd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67140746a0 0x7f67140b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f671b6cd700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67140b3c60 con 0x7f671415e160 2023-11-24T06:10:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f671b6cd700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67140b58c0 con 0x7f67140afc90 2023-11-24T06:10:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f671b6cd700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140b4500 con 0x7f67140746a0 2023-11-24T06:10:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f6718c68700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67140746a0 0x7f67140b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f6718c68700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67140746a0 0x7f67140b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48472/0 (socket says 172.21.15.97:48472) 2023-11-24T06:10:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f6718c68700 1 -- 172.21.15.97:0/1567552862 learned_addr learned my addr 172.21.15.97:0/1567552862 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:46.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f6718c68700 1 -- 172.21.15.97:0/1567552862 >> v1:172.21.15.183:6789/0 conn(0x7f67140afc90 legacy=0x7f671406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.794+0000 7f6718c68700 1 -- 172.21.15.97:0/1567552862 >> v1:172.21.15.97:6789/0 conn(0x7f671415e160 legacy=0x7f6714073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f6718c68700 1 -- 172.21.15.97:0/1567552862 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671406bdc0 con 0x7f67140746a0 2023-11-24T06:10:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f6718c68700 1 --2- 172.21.15.97:0/1567552862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f67140b5da0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f67100098f0 tx=0x7f6710009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7637c88eb96fee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f670bfff700 1 -- 172.21.15.97:0/1567552862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710004230 con 0x7f67140746a0 2023-11-24T06:10:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f670bfff700 1 -- 172.21.15.97:0/1567552862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6710004390 con 0x7f67140746a0 2023-11-24T06:10:46.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f670bfff700 1 -- 172.21.15.97:0/1567552862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67100036a0 con 0x7f67140746a0 2023-11-24T06:10:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f671b6cd700 1 -- 172.21.15.97:0/1567552862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 msgr2=0x7f67140b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f671b6cd700 1 --2- 172.21.15.97:0/1567552862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f67140b5da0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f67100098f0 tx=0x7f6710009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f671b6cd700 1 -- 172.21.15.97:0/1567552862 shutdown_connections 2023-11-24T06:10:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f671b6cd700 1 --2- 172.21.15.97:0/1567552862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f67140b5da0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f671b6cd700 1 -- 172.21.15.97:0/1567552862 >> 172.21.15.97:0/1567552862 conn(0x7f67140d0bf0 msgr2=0x7f67140d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f671b6cd700 1 -- 172.21.15.97:0/1567552862 shutdown_connections 2023-11-24T06:10:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.795+0000 7f671b6cd700 1 -- 172.21.15.97:0/1567552862 wait complete. 2023-11-24T06:10:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f671b6cd700 1 Processor -- start 2023-11-24T06:10:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f671b6cd700 1 -- start start 2023-11-24T06:10:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f671b6cd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f671414f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f671b6cd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67140afc90 0x7f671409be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f671b6cd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f671415e160 0x7f671414bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f671b6cd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671414c600 con 0x7f67140746a0 2023-11-24T06:10:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f671b6cd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671414f590 con 0x7f67140afc90 2023-11-24T06:10:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f671b6cd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671414c810 con 0x7f671415e160 2023-11-24T06:10:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f6719469700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f671414f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.797+0000 7f6719469700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f671414f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48476/0 (socket says 172.21.15.97:48476) 2023-11-24T06:10:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719469700 1 -- 172.21.15.97:0/1751540338 learned_addr learned my addr 172.21.15.97:0/1751540338 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719c6a700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f671415e160 0x7f671414bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6718c68700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67140afc90 0x7f671409be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719469700 1 -- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f671415e160 msgr2=0x7f671414bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719469700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f671415e160 0x7f671414bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719469700 1 -- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67140afc90 msgr2=0x7f671409be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719469700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67140afc90 0x7f671409be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719469700 1 -- 172.21.15.97:0/1751540338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009550 con 0x7f67140746a0 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719c6a700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f671415e160 0x7f671414bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6718c68700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67140afc90 0x7f671409be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6719469700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f671414f050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f670400c430 tx=0x7f670400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6709ffb700 1 -- 172.21.15.97:0/1751540338 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040052d0 con 0x7f67140746a0 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6709ffb700 1 -- 172.21.15.97:0/1751540338 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6704005450 con 0x7f67140746a0 2023-11-24T06:10:46.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671414caf0 con 0x7f67140746a0 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f6709ffb700 1 -- 172.21.15.97:0/1751540338 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704006aa0 con 0x7f67140746a0 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.798+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671414d0b0 con 0x7f67140746a0 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.799+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714068fb0 con 0x7f67140746a0 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.800+0000 7f6709ffb700 1 -- 172.21.15.97:0/1751540338 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6704003680 con 0x7f67140746a0 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.800+0000 7f6709ffb700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66f806b110 0x7f66f806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.800+0000 7f6709ffb700 1 -- 172.21.15.97:0/1751540338 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6704088bc0 con 0x7f67140746a0 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.800+0000 7f6718c68700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66f806b110 0x7f66f806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.800+0000 7f6718c68700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66f806b110 0x7f66f806d5d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f671409ede0 tx=0x7f6710008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:46.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.804+0000 7f6709ffb700 1 -- 172.21.15.97:0/1751540338 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67040539a0 con 0x7f67140746a0 2023-11-24T06:10:46.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.992+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f671406c140 con 0x7f67140746a0 2023-11-24T06:10:46.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.993+0000 7f6709ffb700 1 -- 172.21.15.97:0/1751540338 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6704056fc0 con 0x7f67140746a0 2023-11-24T06:10:46.995 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66f806b110 msgr2=0x7f66f806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66f806b110 0x7f66f806d5d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f671409ede0 tx=0x7f6710008030 comp rx=0 tx=0).stop 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 msgr2=0x7f671414f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f671414f050 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f670400c430 tx=0x7f670400c7f0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 shutdown_connections 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f66f806b110 0x7f66f806d5d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f671415e160 0x7f671414bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67140746a0 0x7f671414f050 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 --2- 172.21.15.97:0/1751540338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67140afc90 0x7f671409be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 >> 172.21.15.97:0/1751540338 conn(0x7f67140d0bf0 msgr2=0x7f67140ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 shutdown_connections 2023-11-24T06:10:46.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:46.995+0000 7f671b6cd700 1 -- 172.21.15.97:0/1751540338 wait complete. 2023-11-24T06:10:47.010 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:47.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.131+0000 7fe8615c6700 1 Processor -- start 2023-11-24T06:10:47.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.131+0000 7fe8615c6700 1 -- start start 2023-11-24T06:10:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe8615c6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe85c0d0890 0x7fe85c0d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe8615c6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0d0c70 0x7fe85c0b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe8615c6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c0cf570 con 0x7fe85c0cefb0 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe8615c6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85c0cf850 con 0x7fe85c0d0890 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe8615c6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0d2b30 con 0x7fe85c0d0c70 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe85affd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe85c0d0890 0x7fe85c0d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe85b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0d0c70 0x7fe85c0b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe85b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0d0c70 0x7fe85c0b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:35524/0 (socket says 172.21.15.97:35524) 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe85affd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe85c0d0890 0x7fe85c0d3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:42644/0 (socket says 172.21.15.97:42644) 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe85b7fe700 1 -- 172.21.15.97:0/530132021 learned_addr learned my addr 172.21.15.97:0/530132021 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.132+0000 7fe85a7fc700 1 --1- 172.21.15.97:0/530132021 >> v1:172.21.15.183:6789/0 conn(0x7fe85c0cefb0 0x7fe85c0b3200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:50260/0 (socket says 172.21.15.97:50260) 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe85b7fe700 1 -- 172.21.15.97:0/530132021 >> v2:172.21.15.183:3300/0 conn(0x7fe85c0d0890 msgr2=0x7fe85c0d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe85b7fe700 1 --2- 172.21.15.97:0/530132021 >> v2:172.21.15.183:3300/0 conn(0x7fe85c0d0890 0x7fe85c0d3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe85b7fe700 1 -- 172.21.15.97:0/530132021 >> v1:172.21.15.183:6789/0 conn(0x7fe85c0cefb0 legacy=0x7fe85c0b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe85b7fe700 1 -- 172.21.15.97:0/530132021 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c0d2d40 con 0x7fe85c0d0c70 2023-11-24T06:10:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe85b7fe700 1 --2- 172.21.15.97:0/530132021 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0d0c70 0x7fe85c0b7160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe854009e50 tx=0x7fe85400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d097aa0367da55f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe859ffb700 1 -- 172.21.15.97:0/530132021 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85400e070 con 0x7fe85c0d0c70 2023-11-24T06:10:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe859ffb700 1 -- 172.21.15.97:0/530132021 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85400b910 con 0x7fe85c0d0c70 2023-11-24T06:10:47.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe859ffb700 1 -- 172.21.15.97:0/530132021 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85400bcc0 con 0x7fe85c0d0c70 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe8615c6700 1 -- 172.21.15.97:0/530132021 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0d0c70 msgr2=0x7fe85c0b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe8615c6700 1 --2- 172.21.15.97:0/530132021 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0d0c70 0x7fe85c0b7160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe854009e50 tx=0x7fe85400b330 comp rx=0 tx=0).stop 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe8615c6700 1 -- 172.21.15.97:0/530132021 shutdown_connections 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe8615c6700 1 --2- 172.21.15.97:0/530132021 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0d0c70 0x7fe85c0b7160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe8615c6700 1 --2- 172.21.15.97:0/530132021 >> v2:172.21.15.183:3300/0 conn(0x7fe85c0d0890 0x7fe85c0d3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe8615c6700 1 -- 172.21.15.97:0/530132021 >> 172.21.15.97:0/530132021 conn(0x7fe85c06b410 msgr2=0x7fe85c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe8615c6700 1 -- 172.21.15.97:0/530132021 shutdown_connections 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.133+0000 7fe8615c6700 1 -- 172.21.15.97:0/530132021 wait complete. 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.135+0000 7fe8615c6700 1 Processor -- start 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.135+0000 7fe8615c6700 1 -- start start 2023-11-24T06:10:47.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.135+0000 7fe8615c6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0cefb0 0x7fe85c0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.135+0000 7fe8615c6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85c0d0890 0x7fe85c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.135+0000 7fe8615c6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe85c0d0c70 0x7fe85c1f6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.135+0000 7fe8615c6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0c7f80 con 0x7fe85c0d0890 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.135+0000 7fe8615c6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0c7670 con 0x7fe85c0d0c70 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.135+0000 7fe8615c6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0cc6f0 con 0x7fe85c0cefb0 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85c0d0890 0x7fe85c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85c0d0890 0x7fe85c0cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48492/0 (socket says 172.21.15.97:48492) 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 -- 172.21.15.97:0/2051135971 learned_addr learned my addr 172.21.15.97:0/2051135971 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85affd700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0cefb0 0x7fe85c0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85b7fe700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe85c0d0c70 0x7fe85c1f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 -- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0cefb0 msgr2=0x7fe85c0b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0cefb0 0x7fe85c0b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 -- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe85c0d0c70 msgr2=0x7fe85c1f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe85c0d0c70 0x7fe85c1f6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 -- 172.21.15.97:0/2051135971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe854009b30 con 0x7fe85c0d0890 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85affd700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0cefb0 0x7fe85c0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85b7fe700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe85c0d0c70 0x7fe85c1f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe85a7fc700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85c0d0890 0x7fe85c0cbcd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00c4c0 tx=0x7fe84c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe8437fe700 1 -- 172.21.15.97:0/2051135971 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c005cb0 con 0x7fe85c0d0890 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe8437fe700 1 -- 172.21.15.97:0/2051135971 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c005e10 con 0x7fe85c0d0890 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c0c78c0 con 0x7fe85c0d0890 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe8437fe700 1 -- 172.21.15.97:0/2051135971 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0072d0 con 0x7fe85c0d0890 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.136+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c0b62b0 con 0x7fe85c0d0890 2023-11-24T06:10:47.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.137+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c068440 con 0x7fe85c0d0890 2023-11-24T06:10:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.138+0000 7fe8437fe700 1 -- 172.21.15.97:0/2051135971 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe84c007430 con 0x7fe85c0d0890 2023-11-24T06:10:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.138+0000 7fe8437fe700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe84806b110 0x7fe84806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.138+0000 7fe8437fe700 1 -- 172.21.15.97:0/2051135971 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe84c088cf0 con 0x7fe85c0d0890 2023-11-24T06:10:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.138+0000 7fe85affd700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe84806b110 0x7fe84806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.138+0000 7fe85affd700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe84806b110 0x7fe84806d5d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe850009b60 tx=0x7fe850008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.142+0000 7fe8437fe700 1 -- 172.21.15.97:0/2051135971 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe84c08d070 con 0x7fe85c0d0890 2023-11-24T06:10:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.335+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe85c0c80c0 con 0x7fe85c0d0890 2023-11-24T06:10:47.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.336+0000 7fe8437fe700 1 -- 172.21.15.97:0/2051135971 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe85c0c80c0 con 0x7fe85c0d0890 2023-11-24T06:10:47.339 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe84806b110 msgr2=0x7fe84806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe84806b110 0x7fe84806d5d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe850009b60 tx=0x7fe850008040 comp rx=0 tx=0).stop 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85c0d0890 msgr2=0x7fe85c0cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85c0d0890 0x7fe85c0cbcd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00c4c0 tx=0x7fe84c00c880 comp rx=0 tx=0).stop 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 shutdown_connections 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe85c0d0c70 0x7fe85c1f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe84806b110 0x7fe84806d5d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe85c0cefb0 0x7fe85c0b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 --2- 172.21.15.97:0/2051135971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe85c0d0890 0x7fe85c0cbcd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 >> 172.21.15.97:0/2051135971 conn(0x7fe85c06b410 msgr2=0x7fe85c06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 shutdown_connections 2023-11-24T06:10:47.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.338+0000 7fe8615c6700 1 -- 172.21.15.97:0/2051135971 wait complete. 2023-11-24T06:10:47.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:47.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.472+0000 7fcb0d550700 1 Processor -- start 2023-11-24T06:10:47.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.473+0000 7fcb0d550700 1 -- start start 2023-11-24T06:10:47.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb0d550700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcb080b5950 0x7fcb080b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb0d550700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb080d1aa0 con 0x7fcb080cf240 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb0d550700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb080b62a0 con 0x7fcb0815e1d0 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb0d550700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb080d3b40 con 0x7fcb080b5950 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb077fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcb080b5950 0x7fcb080b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb077fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcb080b5950 0x7fcb080b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48502/0 (socket says 172.21.15.97:48502) 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb077fe700 1 -- 172.21.15.97:0/1342447405 learned_addr learned my addr 172.21.15.97:0/1342447405 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb077fe700 1 -- 172.21.15.97:0/1342447405 >> v1:172.21.15.183:6789/0 conn(0x7fcb0815e1d0 legacy=0x7fcb080d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb077fe700 1 -- 172.21.15.97:0/1342447405 >> v1:172.21.15.97:6789/0 conn(0x7fcb080cf240 legacy=0x7fcb080d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb077fe700 1 -- 172.21.15.97:0/1342447405 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb080d1cf0 con 0x7fcb080b5950 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb077fe700 1 --2- 172.21.15.97:0/1342447405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080b5d60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009e50 tx=0x7fcb0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b738901eecac3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb05ffb700 1 -- 172.21.15.97:0/1342447405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000e070 con 0x7fcb080b5950 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb05ffb700 1 -- 172.21.15.97:0/1342447405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb00003a20 con 0x7fcb080b5950 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.474+0000 7fcb05ffb700 1 -- 172.21.15.97:0/1342447405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00003dd0 con 0x7fcb080b5950 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.475+0000 7fcb0d550700 1 -- 172.21.15.97:0/1342447405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 msgr2=0x7fcb080b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.475+0000 7fcb0d550700 1 --2- 172.21.15.97:0/1342447405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080b5d60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009e50 tx=0x7fcb0000b330 comp rx=0 tx=0).stop 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.475+0000 7fcb0d550700 1 -- 172.21.15.97:0/1342447405 shutdown_connections 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.475+0000 7fcb0d550700 1 --2- 172.21.15.97:0/1342447405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080b5d60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.475+0000 7fcb0d550700 1 -- 172.21.15.97:0/1342447405 >> 172.21.15.97:0/1342447405 conn(0x7fcb080d0950 msgr2=0x7fcb080d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.475+0000 7fcb0d550700 1 -- 172.21.15.97:0/1342447405 shutdown_connections 2023-11-24T06:10:47.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.475+0000 7fcb0d550700 1 -- 172.21.15.97:0/1342447405 wait complete. 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.476+0000 7fcb0d550700 1 Processor -- start 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.476+0000 7fcb0d550700 1 -- start start 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb0d550700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb0d550700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb080cf240 0x7fcb081f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb0d550700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb0815e1d0 0x7fcb081f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb0d550700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0815e820 con 0x7fcb080b5950 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb0d550700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb080cccf0 con 0x7fcb080cf240 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb0d550700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0815eac0 con 0x7fcb0815e1d0 2023-11-24T06:10:47.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb06ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb06ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48504/0 (socket says 172.21.15.97:48504) 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb06ffd700 1 -- 172.21.15.97:0/3020299925 learned_addr learned my addr 172.21.15.97:0/3020299925 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb067fc700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb080cf240 0x7fcb081f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb077fe700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb0815e1d0 0x7fcb081f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb06ffd700 1 -- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb0815e1d0 msgr2=0x7fcb081f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb06ffd700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb0815e1d0 0x7fcb081f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb06ffd700 1 -- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb080cf240 msgr2=0x7fcb081f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb06ffd700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb080cf240 0x7fcb081f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.477+0000 7fcb06ffd700 1 -- 172.21.15.97:0/3020299925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00009b30 con 0x7fcb080b5950 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.478+0000 7fcb077fe700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb0815e1d0 0x7fcb081f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.478+0000 7fcb06ffd700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080c9740 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800b470 tx=0x7fcaf800b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.478+0000 7fcaef7fe700 1 -- 172.21.15.97:0/3020299925 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8011830 con 0x7fcb080b5950 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.478+0000 7fcaef7fe700 1 -- 172.21.15.97:0/3020299925 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf8011990 con 0x7fcb080b5950 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.478+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0815eda0 con 0x7fcb080b5950 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.478+0000 7fcaef7fe700 1 -- 172.21.15.97:0/3020299925 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8011d40 con 0x7fcb080b5950 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.478+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb080c9c80 con 0x7fcb080b5950 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.479+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0815f3f0 con 0x7fcb080b5950 2023-11-24T06:10:47.486 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.479+0000 7fcaef7fe700 1 -- 172.21.15.97:0/3020299925 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fcaf801a400 con 0x7fcb080b5950 2023-11-24T06:10:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.480+0000 7fcaef7fe700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaf406b110 0x7fcaf406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.480+0000 7fcaef7fe700 1 -- 172.21.15.97:0/3020299925 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcaf8089af0 con 0x7fcb080b5950 2023-11-24T06:10:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.480+0000 7fcb067fc700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaf406b110 0x7fcaf406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.480+0000 7fcb067fc700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaf406b110 0x7fcaf406d5d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcb080cded0 tx=0x7fcafc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.484+0000 7fcaef7fe700 1 -- 172.21.15.97:0/3020299925 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaf8054800 con 0x7fcb080b5950 2023-11-24T06:10:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.676+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb08068fb0 con 0x7fcb080b5950 2023-11-24T06:10:47.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.676+0000 7fcaef7fe700 1 -- 172.21.15.97:0/3020299925 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcaf8015020 con 0x7fcb080b5950 2023-11-24T06:10:47.679 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:10:47.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaf406b110 msgr2=0x7fcaf406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaf406b110 0x7fcaf406d5d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcb080cded0 tx=0x7fcafc009040 comp rx=0 tx=0).stop 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 msgr2=0x7fcb080c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080c9740 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800b470 tx=0x7fcaf800b830 comp rx=0 tx=0).stop 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 shutdown_connections 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcaf406b110 0x7fcaf406d5d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcb0815e1d0 0x7fcb081f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb080b5950 0x7fcb080c9740 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 --2- 172.21.15.97:0/3020299925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb080cf240 0x7fcb081f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 >> 172.21.15.97:0/3020299925 conn(0x7fcb080d0950 msgr2=0x7fcb080b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 shutdown_connections 2023-11-24T06:10:47.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.678+0000 7fcb0d550700 1 -- 172.21.15.97:0/3020299925 wait complete. 2023-11-24T06:10:47.693 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:10:47.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.814+0000 7fbf74943700 1 Processor -- start 2023-11-24T06:10:47.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.814+0000 7fbf74943700 1 -- start start 2023-11-24T06:10:47.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.815+0000 7fbf74943700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf700adf00 0x7fbf7006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.815+0000 7fbf74943700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbf7006c540 0x7fbf700b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.815+0000 7fbf74943700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf700af950 con 0x7fbf701446a0 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.815+0000 7fbf74943700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7006c9f0 con 0x7fbf700adf00 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.815+0000 7fbf74943700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf70144a80 con 0x7fbf7006c540 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.815+0000 7fbf6dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf700adf00 0x7fbf7006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.815+0000 7fbf6dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbf700adf00 0x7fbf7006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48512/0 (socket says 172.21.15.97:48512) 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.815+0000 7fbf6dd9b700 1 -- 172.21.15.97:0/1702650808 learned_addr learned my addr 172.21.15.97:0/1702650808 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6ed9d700 1 --2- 172.21.15.97:0/1702650808 >> v2:172.21.15.183:3300/0 conn(0x7fbf7006c540 0x7fbf700b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6dd9b700 1 -- 172.21.15.97:0/1702650808 >> v2:172.21.15.183:3300/0 conn(0x7fbf7006c540 msgr2=0x7fbf700b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6dd9b700 1 --2- 172.21.15.97:0/1702650808 >> v2:172.21.15.183:3300/0 conn(0x7fbf7006c540 0x7fbf700b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6dd9b700 1 -- 172.21.15.97:0/1702650808 >> v1:172.21.15.97:6789/0 conn(0x7fbf701446a0 legacy=0x7fbf700b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6dd9b700 1 -- 172.21.15.97:0/1702650808 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf700afc30 con 0x7fbf700adf00 2023-11-24T06:10:47.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6dd9b700 1 --2- 172.21.15.97:0/1702650808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf700adf00 0x7fbf7006c000 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbf700aedf0 tx=0x7fbf60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910ae7de01d0007e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6d59a700 1 -- 172.21.15.97:0/1702650808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6000f8a0 con 0x7fbf700adf00 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6d59a700 1 -- 172.21.15.97:0/1702650808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6000fa00 con 0x7fbf700adf00 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf6d59a700 1 -- 172.21.15.97:0/1702650808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6000fd90 con 0x7fbf700adf00 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf74943700 1 -- 172.21.15.97:0/1702650808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf700adf00 msgr2=0x7fbf7006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.816+0000 7fbf74943700 1 --2- 172.21.15.97:0/1702650808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf700adf00 0x7fbf7006c000 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbf700aedf0 tx=0x7fbf60009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.817+0000 7fbf74943700 1 -- 172.21.15.97:0/1702650808 shutdown_connections 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.817+0000 7fbf74943700 1 --2- 172.21.15.97:0/1702650808 >> v2:172.21.15.183:3300/0 conn(0x7fbf7006c540 0x7fbf700b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.817+0000 7fbf74943700 1 --2- 172.21.15.97:0/1702650808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf700adf00 0x7fbf7006c000 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.817+0000 7fbf74943700 1 -- 172.21.15.97:0/1702650808 >> 172.21.15.97:0/1702650808 conn(0x7fbf70071b90 msgr2=0x7fbf70071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.817+0000 7fbf74943700 1 -- 172.21.15.97:0/1702650808 shutdown_connections 2023-11-24T06:10:47.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.817+0000 7fbf74943700 1 -- 172.21.15.97:0/1702650808 wait complete. 2023-11-24T06:10:47.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.818+0000 7fbf74943700 1 Processor -- start 2023-11-24T06:10:47.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.818+0000 7fbf74943700 1 -- start start 2023-11-24T06:10:47.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf74943700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf7006c540 0x7fbf701e5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf74943700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf700adf00 0x7fbf70159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf74943700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf701446a0 0x7fbf70140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf74943700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf70159ff0 con 0x7fbf701446a0 2023-11-24T06:10:47.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf74943700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700cd600 con 0x7fbf7006c540 2023-11-24T06:10:47.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf74943700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf700cd8e0 con 0x7fbf700adf00 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf701446a0 0x7fbf70140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf701446a0 0x7fbf70140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48522/0 (socket says 172.21.15.97:48522) 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 -- 172.21.15.97:0/1784219019 learned_addr learned my addr 172.21.15.97:0/1784219019 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6e59c700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf7006c540 0x7fbf701e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6dd9b700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf700adf00 0x7fbf70159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 -- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf700adf00 msgr2=0x7fbf70159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf700adf00 0x7fbf70159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 -- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf7006c540 msgr2=0x7fbf701e5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf7006c540 0x7fbf701e5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 -- 172.21.15.97:0/1784219019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf60009550 con 0x7fbf701446a0 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6e59c700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf7006c540 0x7fbf701e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6dd9b700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf700adf00 0x7fbf70159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:47.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf6ed9d700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf701446a0 0x7fbf70140af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800cd70 tx=0x7fbf6800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf5effd700 1 -- 172.21.15.97:0/1784219019 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68003a20 con 0x7fbf701446a0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf5effd700 1 -- 172.21.15.97:0/1784219019 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68003b80 con 0x7fbf701446a0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf700cdbc0 con 0x7fbf701446a0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf5effd700 1 -- 172.21.15.97:0/1784219019 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68004020 con 0x7fbf701446a0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.819+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf700ce180 con 0x7fbf701446a0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.820+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf700753f0 con 0x7fbf701446a0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.821+0000 7fbf5effd700 1 -- 172.21.15.97:0/1784219019 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fbf68004180 con 0x7fbf701446a0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.821+0000 7fbf5effd700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf5006b110 0x7fbf5006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.821+0000 7fbf5effd700 1 -- 172.21.15.97:0/1784219019 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fbf68012070 con 0x7fbf701446a0 2023-11-24T06:10:47.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.821+0000 7fbf6e59c700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf5006b110 0x7fbf5006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:47.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.822+0000 7fbf6e59c700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf5006b110 0x7fbf5006d5d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf64000f60 tx=0x7fbf64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:47.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:47.826+0000 7fbf5effd700 1 -- 172.21.15.97:0/1784219019 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf68054fd0 con 0x7fbf701446a0 2023-11-24T06:10:48.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.032+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbf70067f60 con 0x7fbf701446a0 2023-11-24T06:10:48.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.032+0000 7fbf5effd700 1 -- 172.21.15.97:0/1784219019 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbf680585f0 con 0x7fbf701446a0 2023-11-24T06:10:48.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf5006b110 msgr2=0x7fbf5006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf5006b110 0x7fbf5006d5d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbf64000f60 tx=0x7fbf64009040 comp rx=0 tx=0).stop 2023-11-24T06:10:48.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf701446a0 msgr2=0x7fbf70140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf701446a0 0x7fbf70140af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800cd70 tx=0x7fbf6800a570 comp rx=0 tx=0).stop 2023-11-24T06:10:48.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 shutdown_connections 2023-11-24T06:10:48.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf701446a0 0x7fbf70140af0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf7006c540 0x7fbf701e5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf5006b110 0x7fbf5006d5d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 --2- 172.21.15.97:0/1784219019 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf700adf00 0x7fbf70159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 >> 172.21.15.97:0/1784219019 conn(0x7fbf70071b90 msgr2=0x7fbf7006e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:48.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 shutdown_connections 2023-11-24T06:10:48.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.034+0000 7fbf74943700 1 -- 172.21.15.97:0/1784219019 wait complete. 2023-11-24T06:10:48.038 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:10:48.048 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:10:48.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.049 DEBUG:teuthology.orchestra.run.smithi183:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T06:10:48.098 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:10:48.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:10:48.098 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:10:48.116 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:48.116 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:48.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:10:48.135 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:48.135 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:48.176 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:10:48.176 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:48.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.294+0000 7f2182eb4700 1 Processor -- start 2023-11-24T06:10:48.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.294+0000 7f2182eb4700 1 -- start start 2023-11-24T06:10:48.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2182eb4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f217c15e080 0x7f217c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2182eb4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c06c710 0x7f217c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2182eb4700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c0ae430 con 0x7f217c0af4d0 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2182eb4700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c0af0b0 con 0x7f217c15e080 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2182eb4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b05a0 con 0x7f217c06c710 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2180c50700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f217c15e080 0x7f217c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2180c50700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f217c15e080 0x7f217c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48526/0 (socket says 172.21.15.97:48526) 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2180c50700 1 -- 172.21.15.97:0/2290519062 learned_addr learned my addr 172.21.15.97:0/2290519062 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2181451700 1 --2- 172.21.15.97:0/2290519062 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c06c710 0x7f217c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2180c50700 1 -- 172.21.15.97:0/2290519062 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c06c710 msgr2=0x7f217c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2180c50700 1 --2- 172.21.15.97:0/2290519062 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c06c710 0x7f217c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2180c50700 1 -- 172.21.15.97:0/2290519062 >> v1:172.21.15.183:6789/0 conn(0x7f217c0af4d0 legacy=0x7f217c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2180c50700 1 -- 172.21.15.97:0/2290519062 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0ae710 con 0x7f217c15e080 2023-11-24T06:10:48.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.295+0000 7f2180c50700 1 --2- 172.21.15.97:0/2290519062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c15e080 0x7f217c0b5050 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2170009920 tx=0x7f2170009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b365195a975c21d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f217b7fe700 1 -- 172.21.15.97:0/2290519062 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217000ba20 con 0x7f217c15e080 2023-11-24T06:10:48.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f217b7fe700 1 -- 172.21.15.97:0/2290519062 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217000bb80 con 0x7f217c15e080 2023-11-24T06:10:48.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f217b7fe700 1 -- 172.21.15.97:0/2290519062 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170009e70 con 0x7f217c15e080 2023-11-24T06:10:48.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f2182eb4700 1 -- 172.21.15.97:0/2290519062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c15e080 msgr2=0x7f217c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f2182eb4700 1 --2- 172.21.15.97:0/2290519062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c15e080 0x7f217c0b5050 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2170009920 tx=0x7f2170009c30 comp rx=0 tx=0).stop 2023-11-24T06:10:48.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f2182eb4700 1 -- 172.21.15.97:0/2290519062 shutdown_connections 2023-11-24T06:10:48.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f2182eb4700 1 --2- 172.21.15.97:0/2290519062 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c06c710 0x7f217c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f2182eb4700 1 --2- 172.21.15.97:0/2290519062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c15e080 0x7f217c0b5050 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f2182eb4700 1 -- 172.21.15.97:0/2290519062 >> 172.21.15.97:0/2290519062 conn(0x7f217c071b90 msgr2=0x7f217c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f2182eb4700 1 -- 172.21.15.97:0/2290519062 shutdown_connections 2023-11-24T06:10:48.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.296+0000 7f2182eb4700 1 -- 172.21.15.97:0/2290519062 wait complete. 2023-11-24T06:10:48.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.297+0000 7f2182eb4700 1 Processor -- start 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.297+0000 7f2182eb4700 1 -- start start 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2182eb4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c06c710 0x7f217c1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2182eb4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f217c0af4d0 0x7f217c155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2182eb4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c15e080 0x7f217c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2182eb4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0c5de0 con 0x7f217c06c710 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2182eb4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c153910 con 0x7f217c0af4d0 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2182eb4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c156420 con 0x7f217c15e080 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2180c50700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c06c710 0x7f217c1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2180c50700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c06c710 0x7f217c1533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48532/0 (socket says 172.21.15.97:48532) 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.298+0000 7f2180c50700 1 -- 172.21.15.97:0/3226154936 learned_addr learned my addr 172.21.15.97:0/3226154936 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:48.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f2180c50700 1 -- 172.21.15.97:0/3226154936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c15e080 msgr2=0x7f217c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f2180c50700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c15e080 0x7f217c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f2180c50700 1 -- 172.21.15.97:0/3226154936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f217c0af4d0 msgr2=0x7f217c155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f217bfff700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f217c0af4d0 0x7f217c155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f2180c50700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f217c0af4d0 0x7f217c155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f2180c50700 1 -- 172.21.15.97:0/3226154936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170009550 con 0x7f217c06c710 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f2180c50700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c06c710 0x7f217c1533d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2170009130 tx=0x7f2170004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f21797fa700 1 -- 172.21.15.97:0/3226154936 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170004550 con 0x7f217c06c710 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f21797fa700 1 -- 172.21.15.97:0/3226154936 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217001e070 con 0x7f217c06c710 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c156700 con 0x7f217c06c710 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f21797fa700 1 -- 172.21.15.97:0/3226154936 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21700038a0 con 0x7f217c06c710 2023-11-24T06:10:48.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.299+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c157660 con 0x7f217c06c710 2023-11-24T06:10:48.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.300+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c0753f0 con 0x7f217c06c710 2023-11-24T06:10:48.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.300+0000 7f21797fa700 1 -- 172.21.15.97:0/3226154936 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2170003a00 con 0x7f217c06c710 2023-11-24T06:10:48.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.301+0000 7f21797fa700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f216006b110 0x7f216006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.301+0000 7f21797fa700 1 -- 172.21.15.97:0/3226154936 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f2170020070 con 0x7f217c06c710 2023-11-24T06:10:48.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.301+0000 7f217bfff700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f216006b110 0x7f216006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.301+0000 7f217bfff700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f216006b110 0x7f216006d5d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f217c0cf7e0 tx=0x7f2168009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.305+0000 7f21797fa700 1 -- 172.21.15.97:0/3226154936 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2170055c50 con 0x7f217c06c710 2023-11-24T06:10:48.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.543+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f217c067f60 con 0x7f217c06c710 2023-11-24T06:10:48.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.543+0000 7f21797fa700 1 -- 172.21.15.97:0/3226154936 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f217002a020 con 0x7f217c06c710 2023-11-24T06:10:48.546 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:48.547 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/163703518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":163703518},{"type":"v1","addr":"172.21.15.97:6837","nonce":163703518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/1688123009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1688123009},{"type":"v1","addr":"172.21.15.183:6834","nonce":1688123009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/310487911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":310487911},{"type":"v1","addr":"172.21.15.183:6835","nonce":310487911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:27.576219+0000","modified":"2023-11-24T06:10:30.898770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/19331625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":19331625},{"type":"v1","addr":"172.21.15.97:6835","nonce":19331625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.545+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f216006b110 msgr2=0x7f216006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.545+0000 7f2182eb4700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f216006b110 0x7f216006d5d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f217c0cf7e0 tx=0x7f2168009040 comp rx=0 tx=0).stop 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.545+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c06c710 msgr2=0x7f217c1533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.545+0000 7f2182eb4700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c06c710 0x7f217c1533d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2170009130 tx=0x7f2170004750 comp rx=0 tx=0).stop 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.546+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 shutdown_connections 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.546+0000 7f2182eb4700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f216006b110 0x7f216006d5d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.546+0000 7f2182eb4700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f217c15e080 0x7f217c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.546+0000 7f2182eb4700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f217c06c710 0x7f217c1533d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.546+0000 7f2182eb4700 1 --2- 172.21.15.97:0/3226154936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f217c0af4d0 0x7f217c155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.546+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 >> 172.21.15.97:0/3226154936 conn(0x7f217c071b90 msgr2=0x7f217c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:48.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.546+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 shutdown_connections 2023-11-24T06:10:48.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.546+0000 7f2182eb4700 1 -- 172.21.15.97:0/3226154936 wait complete. 2023-11-24T06:10:48.550 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2023-11-24T06:10:48.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:48.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.680+0000 7f00a3bff700 1 Processor -- start 2023-11-24T06:10:48.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.681+0000 7f00a3bff700 1 -- start start 2023-11-24T06:10:48.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.681+0000 7f00a3bff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f009c0b59f0 0x7f009c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.681+0000 7f00a3bff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f009c0b4350 0x7f009c0af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.681+0000 7f00a3bff700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009c0b0290 con 0x7f009c06e0b0 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.681+0000 7f00a3bff700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c0b4840 con 0x7f009c0b59f0 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.681+0000 7f00a3bff700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0b4b00 con 0x7f009c0b4350 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a119a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f009c06e0b0 0x7f009c0b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:40992/0 (socket says 172.21.15.97:40992) 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a119a700 1 -- 172.21.15.97:0/3543213204 learned_addr learned my addr 172.21.15.97:0/3543213204 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a219c700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f009c0b4350 0x7f009c0af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a199b700 1 --2- 172.21.15.97:0/3543213204 >> v2:172.21.15.183:3300/0 conn(0x7f009c0b59f0 0x7f009c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158639116 0 0) 0x7f009c0b0290 con 0x7f009c06e0b0 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00880035b0 con 0x7f009c06e0b0 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 103989670 0 0) 0x7f00880035b0 con 0x7f009c06e0b0 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f009c0b4350 msgr2=0x7f009c0af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:48.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f009c0b4350 0x7f009c0af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 >> v2:172.21.15.183:3300/0 conn(0x7f009c0b59f0 msgr2=0x7f009c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 --2- 172.21.15.97:0/3543213204 >> v2:172.21.15.183:3300/0 conn(0x7f009c0b59f0 0x7f009c06db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0ae9b0 con 0x7f009c06e0b0 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f0098001670 con 0x7f009c06e0b0 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 >> v1:172.21.15.97:6789/0 conn(0x7f009c06e0b0 legacy=0x7f009c0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f009c0b4350 0x7f0088003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0088003f80 0x7f0088004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0088004d90 0x7f0088005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0b4840 con 0x7f0088003f80 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0b4b00 con 0x7f0088004d90 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008800ba10 con 0x7f009c0b4350 2023-11-24T06:10:48.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a119a700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0088003f80 0x7f0088004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a199b700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f009c0b4350 0x7f0088003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.682+0000 7f00a219c700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0088004d90 0x7f0088005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a119a700 1 -- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f009c0b4350 msgr2=0x7f0088003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a119a700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f009c0b4350 0x7f0088003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a119a700 1 -- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0088004d90 msgr2=0x7f0088005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a119a700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0088004d90 0x7f0088005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a119a700 1 -- 172.21.15.97:0/3543213204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880035b0 con 0x7f0088003f80 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a119a700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0088003f80 0x7f0088004850 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f009800c210 tx=0x7f0088005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d460c8b0eb3f9c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098007c10 con 0x7f0088003f80 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009800cd70 con 0x7f0088003f80 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a0999700 1 -- 172.21.15.97:0/3543213204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098005ba0 con 0x7f0088003f80 2023-11-24T06:10:48.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a3bff700 1 -- 172.21.15.97:0/3543213204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0088003f80 msgr2=0x7f0088004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a3bff700 1 --2- 172.21.15.97:0/3543213204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0088003f80 0x7f0088004850 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f009800c210 tx=0x7f0088005880 comp rx=0 tx=0).stop 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a219c700 1 -- 172.21.15.97:0/3543213204 reap_dead start 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a3bff700 1 -- 172.21.15.97:0/3543213204 shutdown_connections 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a3bff700 1 -- 172.21.15.97:0/3543213204 >> 172.21.15.97:0/3543213204 conn(0x7f009c072990 msgr2=0x7f009c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a3bff700 1 -- 172.21.15.97:0/3543213204 shutdown_connections 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.683+0000 7f00a3bff700 1 -- 172.21.15.97:0/3543213204 wait complete. 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a3bff700 1 Processor -- start 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a3bff700 1 -- start start 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a3bff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0088003f80 0x7f009c1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a3bff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f009c14f2e0 0x7f009c14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a3bff700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f009c0ce920 0x7f009c0ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a3bff700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0cf910 con 0x7f009c14f2e0 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a3bff700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c14fca0 con 0x7f009c0ce920 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a3bff700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c15adc0 con 0x7f0088003f80 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.685+0000 7f00a119a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f009c14f2e0 0x7f009c14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a119a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f009c14f2e0 0x7f009c14f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48552/0 (socket says 172.21.15.97:48552) 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a119a700 1 -- 172.21.15.97:0/1174356003 learned_addr learned my addr 172.21.15.97:0/1174356003 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a219c700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f009c0ce920 0x7f009c0ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a199b700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0088003f80 0x7f009c1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a119a700 1 -- 172.21.15.97:0/1174356003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0088003f80 msgr2=0x7f009c1445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a119a700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0088003f80 0x7f009c1445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a119a700 1 -- 172.21.15.97:0/1174356003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f009c0ce920 msgr2=0x7f009c0ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a119a700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f009c0ce920 0x7f009c0ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a119a700 1 -- 172.21.15.97:0/1174356003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0cf630 con 0x7f009c14f2e0 2023-11-24T06:10:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a119a700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f009c14f2e0 0x7f009c14f760 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f00880046f0 tx=0x7f0098001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00927fc700 1 -- 172.21.15.97:0/1174356003 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800cee0 con 0x7f009c14f2e0 2023-11-24T06:10:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00927fc700 1 -- 172.21.15.97:0/1174356003 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0098001650 con 0x7f009c14f2e0 2023-11-24T06:10:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009c15a770 con 0x7f009c14f2e0 2023-11-24T06:10:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00927fc700 1 -- 172.21.15.97:0/1174356003 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098007310 con 0x7f009c14f2e0 2023-11-24T06:10:48.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.686+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c0cf2e0 con 0x7f009c14f2e0 2023-11-24T06:10:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.687+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009c15a5d0 con 0x7f009c14f2e0 2023-11-24T06:10:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.688+0000 7f00927fc700 1 -- 172.21.15.97:0/1174356003 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0098019030 con 0x7f009c14f2e0 2023-11-24T06:10:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.688+0000 7f00927fc700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f008006b110 0x7f008006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.688+0000 7f00927fc700 1 -- 172.21.15.97:0/1174356003 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f009808ae80 con 0x7f009c14f2e0 2023-11-24T06:10:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.688+0000 7f00a199b700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f008006b110 0x7f008006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:48.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.688+0000 7f00a199b700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f008006b110 0x7f008006d5d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f008c009c60 tx=0x7f008c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:48.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.692+0000 7f00927fc700 1 -- 172.21.15.97:0/1174356003 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f009c15a5d0 con 0x7f009c14f2e0 2023-11-24T06:10:48.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.884+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f009c15a5d0 con 0x7f009c14f2e0 2023-11-24T06:10:48.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.885+0000 7f00927fc700 1 -- 172.21.15.97:0/1174356003 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16785 (secure 0 0 0) 0x7f009c15a5d0 con 0x7f009c14f2e0 2023-11-24T06:10:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:48.889 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":48,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:47.603703+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:25.335173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:26.547561+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.37","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f008006b110 msgr2=0x7f008006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f008006b110 0x7f008006d5d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f008c009c60 tx=0x7f008c009210 comp rx=0 tx=0).stop 2023-11-24T06:10:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f009c14f2e0 msgr2=0x7f009c14f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f009c14f2e0 0x7f009c14f760 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f00880046f0 tx=0x7f0098001a00 comp rx=0 tx=0).stop 2023-11-24T06:10:48.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 shutdown_connections 2023-11-24T06:10:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f009c0ce920 0x7f009c0ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f008006b110 0x7f008006d5d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0088003f80 0x7f009c1445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 --2- 172.21.15.97:0/1174356003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f009c14f2e0 0x7f009c14f760 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:48.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 >> 172.21.15.97:0/1174356003 conn(0x7f009c072990 msgr2=0x7f009c158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:48.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 shutdown_connections 2023-11-24T06:10:48.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:48.887+0000 7f00a3bff700 1 -- 172.21.15.97:0/1174356003 wait complete. 2023-11-24T06:10:48.903 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:10:25.335173+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:10:26.547561+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 48, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:48.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.023+0000 7f762e7db700 1 Processor -- start 2023-11-24T06:10:49.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.023+0000 7f762e7db700 1 -- start start 2023-11-24T06:10:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f762e7db700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f762815bf50 0x7f76280ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f762e7db700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76280c6c30 0x7f76280cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f762e7db700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76280d0140 con 0x7f76280ce830 2023-11-24T06:10:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f762e7db700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762815c330 con 0x7f762815bf50 2023-11-24T06:10:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f762e7db700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76280cf3c0 con 0x7f76280c6c30 2023-11-24T06:10:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f762cd78700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76280c6c30 0x7f76280cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f7627fff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f762815bf50 0x7f76280ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f762cd78700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76280c6c30 0x7f76280cc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:35598/0 (socket says 172.21.15.97:35598) 2023-11-24T06:10:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f762cd78700 1 -- 172.21.15.97:0/2125707890 learned_addr learned my addr 172.21.15.97:0/2125707890 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:49.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f7627fff700 1 -- 172.21.15.97:0/2125707890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76280c6c30 msgr2=0x7f76280cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f7627fff700 1 --2- 172.21.15.97:0/2125707890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76280c6c30 0x7f76280cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.024+0000 7f7627fff700 1 -- 172.21.15.97:0/2125707890 >> v1:172.21.15.183:6789/0 conn(0x7f76280ce830 legacy=0x7f76280cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f7627fff700 1 -- 172.21.15.97:0/2125707890 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76280d03e0 con 0x7f762815bf50 2023-11-24T06:10:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f7626ffd700 1 -- 172.21.15.97:0/2125707890 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318809130 0 0) 0x7f76280d0140 con 0x7f76280ce830 2023-11-24T06:10:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f7627fff700 1 --2- 172.21.15.97:0/2125707890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762815bf50 0x7f76280ce2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f761c009840 tx=0x7f761c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=755caf43da7c3649 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f7626ffd700 1 -- 172.21.15.97:0/2125707890 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c011830 con 0x7f762815bf50 2023-11-24T06:10:49.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f7626ffd700 1 -- 172.21.15.97:0/2125707890 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c011990 con 0x7f762815bf50 2023-11-24T06:10:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f7626ffd700 1 -- 172.21.15.97:0/2125707890 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c011d20 con 0x7f762815bf50 2023-11-24T06:10:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f762e7db700 1 -- 172.21.15.97:0/2125707890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762815bf50 msgr2=0x7f76280ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f762e7db700 1 --2- 172.21.15.97:0/2125707890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762815bf50 0x7f76280ce2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f761c009840 tx=0x7f761c009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f762e7db700 1 -- 172.21.15.97:0/2125707890 shutdown_connections 2023-11-24T06:10:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f762e7db700 1 --2- 172.21.15.97:0/2125707890 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76280c6c30 0x7f76280cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f762e7db700 1 --2- 172.21.15.97:0/2125707890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f762815bf50 0x7f76280ce2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f762e7db700 1 -- 172.21.15.97:0/2125707890 >> 172.21.15.97:0/2125707890 conn(0x7f76280b3780 msgr2=0x7f76280b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.025+0000 7f762e7db700 1 -- 172.21.15.97:0/2125707890 shutdown_connections 2023-11-24T06:10:49.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.026+0000 7f762e7db700 1 -- 172.21.15.97:0/2125707890 wait complete. 2023-11-24T06:10:49.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.027+0000 7f762e7db700 1 Processor -- start 2023-11-24T06:10:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.027+0000 7f762e7db700 1 -- start start 2023-11-24T06:10:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762e7db700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280c6c30 0x7f76280b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762e7db700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76280ce830 0x7f7628144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762e7db700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f762815bf50 0x7f762806f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762e7db700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281e56c0 con 0x7f76280c6c30 2023-11-24T06:10:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762e7db700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7628145110 con 0x7f76280ce830 2023-11-24T06:10:49.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762e7db700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76281453f0 con 0x7f762815bf50 2023-11-24T06:10:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280c6c30 0x7f76280b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280c6c30 0x7f76280b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48564/0 (socket says 172.21.15.97:48564) 2023-11-24T06:10:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 -- 172.21.15.97:0/3826589124 learned_addr learned my addr 172.21.15.97:0/3826589124 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762cd78700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f762815bf50 0x7f762806f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f76277fe700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76280ce830 0x7f7628144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 -- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f762815bf50 msgr2=0x7f762806f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f762815bf50 0x7f762806f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 -- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76280ce830 msgr2=0x7f7628144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76280ce830 0x7f7628144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 -- 172.21.15.97:0/3826589124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761c0094a0 con 0x7f76280c6c30 2023-11-24T06:10:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762cd78700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f762815bf50 0x7f762806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f76277fe700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76280ce830 0x7f7628144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7627fff700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280c6c30 0x7f76280b7300 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f761c009b90 tx=0x7f761c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7624ff9700 1 -- 172.21.15.97:0/3826589124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c0038a0 con 0x7f76280c6c30 2023-11-24T06:10:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7624ff9700 1 -- 172.21.15.97:0/3826589124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c01e070 con 0x7f76280c6c30 2023-11-24T06:10:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f7624ff9700 1 -- 172.21.15.97:0/3826589124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c003c40 con 0x7f76280c6c30 2023-11-24T06:10:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76281456d0 con 0x7f76280c6c30 2023-11-24T06:10:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.028+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76280b7840 con 0x7f76280c6c30 2023-11-24T06:10:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.029+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7628068fb0 con 0x7f76280c6c30 2023-11-24T06:10:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.030+0000 7f7624ff9700 1 -- 172.21.15.97:0/3826589124 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f761c02d450 con 0x7f76280c6c30 2023-11-24T06:10:49.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.030+0000 7f7624ff9700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f761406b110 0x7f761406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.030+0000 7f7624ff9700 1 -- 172.21.15.97:0/3826589124 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f761c08aa10 con 0x7f76280c6c30 2023-11-24T06:10:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.030+0000 7f76277fe700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f761406b110 0x7f761406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.031+0000 7f76277fe700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f761406b110 0x7f761406d5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7618000f60 tx=0x7f7618009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.035+0000 7f7624ff9700 1 -- 172.21.15.97:0/3826589124 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f761c055780 con 0x7f76280c6c30 2023-11-24T06:10:49.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.223+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76280cee20 con 0x7f76280c6c30 2023-11-24T06:10:49.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.224+0000 7f7624ff9700 1 -- 172.21.15.97:0/3826589124 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f761c0150a0 con 0x7f76280c6c30 2023-11-24T06:10:49.227 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f761406b110 msgr2=0x7f761406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f761406b110 0x7f761406d5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7618000f60 tx=0x7f7618009040 comp rx=0 tx=0).stop 2023-11-24T06:10:49.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280c6c30 msgr2=0x7f76280b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280c6c30 0x7f76280b7300 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f761c009b90 tx=0x7f761c004750 comp rx=0 tx=0).stop 2023-11-24T06:10:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 shutdown_connections 2023-11-24T06:10:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f761406b110 0x7f761406d5d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f762815bf50 0x7f762806f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76280c6c30 0x7f76280b7300 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 --2- 172.21.15.97:0/3826589124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76280ce830 0x7f7628144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 >> 172.21.15.97:0/3826589124 conn(0x7f76280b3780 msgr2=0x7f76280b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:49.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 shutdown_connections 2023-11-24T06:10:49.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.226+0000 7f762e7db700 1 -- 172.21.15.97:0/3826589124 wait complete. 2023-11-24T06:10:49.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:10:49.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.363+0000 7f8ba5fa6700 1 Processor -- start 2023-11-24T06:10:49.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.363+0000 7f8ba5fa6700 1 -- start start 2023-11-24T06:10:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8ba5fa6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8ba01587f0 0x7f8ba006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8ba5fa6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ba015b370 0x7f8ba015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8ba5fa6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba006b530 con 0x7f8ba00b3c60 2023-11-24T06:10:49.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8ba5fa6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba00ae5e0 con 0x7f8ba015b370 2023-11-24T06:10:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8ba5fa6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba015be30 con 0x7f8ba01587f0 2023-11-24T06:10:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b96ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ba015b370 0x7f8ba015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b96ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8ba015b370 0x7f8ba015b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48574/0 (socket says 172.21.15.97:48574) 2023-11-24T06:10:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b96ffd700 1 -- 172.21.15.97:0/4125171495 learned_addr learned my addr 172.21.15.97:0/4125171495 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b96ffd700 1 -- 172.21.15.97:0/4125171495 >> v2:172.21.15.183:3300/0 conn(0x7f8ba01587f0 msgr2=0x7f8ba006aff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:10:49.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b96ffd700 1 --2- 172.21.15.97:0/4125171495 >> v2:172.21.15.183:3300/0 conn(0x7f8ba01587f0 0x7f8ba006aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b96ffd700 1 -- 172.21.15.97:0/4125171495 >> v1:172.21.15.97:6789/0 conn(0x7f8ba00b3c60 legacy=0x7f8ba00af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b96ffd700 1 -- 172.21.15.97:0/4125171495 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba015c110 con 0x7f8ba015b370 2023-11-24T06:10:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b96ffd700 1 --2- 172.21.15.97:0/4125171495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba015b370 0x7f8ba015b800 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8b880098f0 tx=0x7f8b88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df91868417ef884 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b9effd700 1 -- 172.21.15.97:0/4125171495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800b860 con 0x7f8ba015b370 2023-11-24T06:10:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.364+0000 7f8b9effd700 1 -- 172.21.15.97:0/4125171495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8800b9c0 con 0x7f8ba015b370 2023-11-24T06:10:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8b9effd700 1 -- 172.21.15.97:0/4125171495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800bd50 con 0x7f8ba015b370 2023-11-24T06:10:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/4125171495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba015b370 msgr2=0x7f8ba015b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/4125171495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba015b370 0x7f8ba015b800 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8b880098f0 tx=0x7f8b88009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/4125171495 shutdown_connections 2023-11-24T06:10:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/4125171495 >> v2:172.21.15.183:3300/0 conn(0x7f8ba01587f0 0x7f8ba006aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/4125171495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba015b370 0x7f8ba015b800 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/4125171495 >> 172.21.15.97:0/4125171495 conn(0x7f8ba00d0bf0 msgr2=0x7f8ba00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/4125171495 shutdown_connections 2023-11-24T06:10:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.365+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/4125171495 wait complete. 2023-11-24T06:10:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.367+0000 7f8ba5fa6700 1 Processor -- start 2023-11-24T06:10:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.367+0000 7f8ba5fa6700 1 -- start start 2023-11-24T06:10:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8ba5fa6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba00b3c60 0x7f8ba014a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8ba5fa6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba01587f0 0x7f8ba014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8ba5fa6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba015b370 0x7f8ba006f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8ba5fa6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba0150f50 con 0x7f8ba00b3c60 2023-11-24T06:10:49.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8ba5fa6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba0150340 con 0x7f8ba015b370 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8ba5fa6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba0150620 con 0x7f8ba01587f0 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba00b3c60 0x7f8ba014a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba00b3c60 0x7f8ba014a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48578/0 (socket says 172.21.15.97:48578) 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 -- 172.21.15.97:0/1120939300 learned_addr learned my addr 172.21.15.97:0/1120939300 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 -- 172.21.15.97:0/1120939300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba01587f0 msgr2=0x7f8ba014f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba01587f0 0x7f8ba014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 -- 172.21.15.97:0/1120939300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba015b370 msgr2=0x7f8ba006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba015b370 0x7f8ba006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.378 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 -- 172.21.15.97:0/1120939300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88009550 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9f7fe700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba00b3c60 0x7f8ba014a540 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000b550 tx=0x7f8b9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9cff9700 1 -- 172.21.15.97:0/1120939300 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90011830 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9cff9700 1 -- 172.21.15.97:0/1120939300 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b900119b0 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba0150900 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8b9cff9700 1 -- 172.21.15.97:0/1120939300 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90011d60 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.368+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba014aa80 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.369+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba0068fb0 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.370+0000 7f8b9cff9700 1 -- 172.21.15.97:0/1120939300 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8b9001a400 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.370+0000 7f8b9cff9700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b8006b110 0x7f8b8006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.370+0000 7f8b9cff9700 1 -- 172.21.15.97:0/1120939300 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8b90089b50 con 0x7f8ba00b3c60 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.370+0000 7f8b96ffd700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b8006b110 0x7f8b8006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.371+0000 7f8b96ffd700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b8006b110 0x7f8b8006d5d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00b05d0 tx=0x7f8b88010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.375+0000 7f8b9cff9700 1 -- 172.21.15.97:0/1120939300 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b90054830 con 0x7f8ba00b3c60 2023-11-24T06:10:49.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.565+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ba015b8d0 con 0x7f8ba00b3c60 2023-11-24T06:10:49.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.565+0000 7f8b9cff9700 1 -- 172.21.15.97:0/1120939300 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b900150a0 con 0x7f8ba00b3c60 2023-11-24T06:10:49.568 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:10:49.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b8006b110 msgr2=0x7f8b8006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b8006b110 0x7f8b8006d5d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00b05d0 tx=0x7f8b88010000 comp rx=0 tx=0).stop 2023-11-24T06:10:49.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba00b3c60 msgr2=0x7f8ba014a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba00b3c60 0x7f8ba014a540 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000b550 tx=0x7f8b9000b910 comp rx=0 tx=0).stop 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 shutdown_connections 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ba015b370 0x7f8ba006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ba00b3c60 0x7f8ba014a540 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8b8006b110 0x7f8b8006d5d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 --2- 172.21.15.97:0/1120939300 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ba01587f0 0x7f8ba014f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 >> 172.21.15.97:0/1120939300 conn(0x7f8ba00d0bf0 msgr2=0x7f8ba00745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 shutdown_connections 2023-11-24T06:10:49.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.568+0000 7f8ba5fa6700 1 -- 172.21.15.97:0/1120939300 wait complete. 2023-11-24T06:10:49.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:10:49.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.704+0000 7f460891a700 1 Processor -- start 2023-11-24T06:10:49.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.704+0000 7f460891a700 1 -- start start 2023-11-24T06:10:49.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f460891a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f46040add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f460891a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4604073f90 0x7f4604074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f460891a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460406b080 con 0x7f460415f9e0 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f460891a700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4604074960 con 0x7f4604073f90 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f460891a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460415c290 con 0x7f46040cf610 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f4602d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4604073f90 0x7f4604074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f4602d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4604073f90 0x7f4604074420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48590/0 (socket says 172.21.15.97:48590) 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f4602d9d700 1 -- 172.21.15.97:0/3265908392 learned_addr learned my addr 172.21.15.97:0/3265908392 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f4602d9d700 1 -- 172.21.15.97:0/3265908392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 msgr2=0x7f46040add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f460259c700 1 --2- 172.21.15.97:0/3265908392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f46040add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f4602d9d700 1 --2- 172.21.15.97:0/3265908392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f46040add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f4602d9d700 1 -- 172.21.15.97:0/3265908392 >> v1:172.21.15.97:6789/0 conn(0x7f460415f9e0 legacy=0x7f46040b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f4602d9d700 1 -- 172.21.15.97:0/3265908392 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46040cf9f0 con 0x7f4604073f90 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.705+0000 7f4602d9d700 1 --2- 172.21.15.97:0/3265908392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4604073f90 0x7f4604074420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009e50 tx=0x7f45fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb6fd4dee87ea92 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460159a700 1 -- 172.21.15.97:0/3265908392 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00e070 con 0x7f4604073f90 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460159a700 1 -- 172.21.15.97:0/3265908392 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc00b9c0 con 0x7f4604073f90 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460159a700 1 -- 172.21.15.97:0/3265908392 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc00bd70 con 0x7f4604073f90 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460891a700 1 -- 172.21.15.97:0/3265908392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4604073f90 msgr2=0x7f4604074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460891a700 1 --2- 172.21.15.97:0/3265908392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4604073f90 0x7f4604074420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009e50 tx=0x7f45fc00b330 comp rx=0 tx=0).stop 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460891a700 1 -- 172.21.15.97:0/3265908392 shutdown_connections 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460891a700 1 --2- 172.21.15.97:0/3265908392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4604073f90 0x7f4604074420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460891a700 1 --2- 172.21.15.97:0/3265908392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f46040add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460891a700 1 -- 172.21.15.97:0/3265908392 >> 172.21.15.97:0/3265908392 conn(0x7f46040d0bf0 msgr2=0x7f46040d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460891a700 1 -- 172.21.15.97:0/3265908392 shutdown_connections 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.706+0000 7f460891a700 1 -- 172.21.15.97:0/3265908392 wait complete. 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f460891a700 1 Processor -- start 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f460891a700 1 -- start start 2023-11-24T06:10:49.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f460891a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604073f90 0x7f4604142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f460891a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f4604155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f460891a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f460415f9e0 0x7f46041564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f460891a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4604142bd0 con 0x7f460415f9e0 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f460891a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4604154c60 con 0x7f4604073f90 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f460891a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4604154f40 con 0x7f46040cf610 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f4602d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f460415f9e0 0x7f46041564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f4602d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f460415f9e0 0x7f46041564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48596/0 (socket says 172.21.15.97:48596) 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.708+0000 7f4602d9d700 1 -- 172.21.15.97:0/1614435910 learned_addr learned my addr 172.21.15.97:0/1614435910 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f460259c700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604073f90 0x7f4604142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f4601d9b700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f4604155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f4602d9d700 1 -- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 msgr2=0x7f4604155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f4602d9d700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f4604155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.717 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f4602d9d700 1 -- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604073f90 msgr2=0x7f4604142690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f4602d9d700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604073f90 0x7f4604142690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f4602d9d700 1 -- 172.21.15.97:0/1614435910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc009b30 con 0x7f460415f9e0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f460259c700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604073f90 0x7f4604142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f4601d9b700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f4604155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f4602d9d700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f460415f9e0 0x7f46041564a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4604073030 tx=0x7f45fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f45f2ffd700 1 -- 172.21.15.97:0/1614435910 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc0092e0 con 0x7f460415f9e0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f45f2ffd700 1 -- 172.21.15.97:0/1614435910 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc004960 con 0x7f460415f9e0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4604155220 con 0x7f460415f9e0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f45f2ffd700 1 -- 172.21.15.97:0/1614435910 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc01c680 con 0x7f460415f9e0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.709+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46041557e0 con 0x7f460415f9e0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.710+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4604155970 con 0x7f460415f9e0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.710+0000 7f45f2ffd700 1 -- 172.21.15.97:0/1614435910 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f45fc01c7e0 con 0x7f460415f9e0 2023-11-24T06:10:49.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.711+0000 7f45f2ffd700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f45e406b110 0x7f45e406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:49.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.711+0000 7f45f2ffd700 1 -- 172.21.15.97:0/1614435910 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f45fc08b970 con 0x7f460415f9e0 2023-11-24T06:10:49.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.711+0000 7f460259c700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f45e406b110 0x7f45e406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:49.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.711+0000 7f460259c700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f45e406b110 0x7f45e406d5d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f45f4006fd0 tx=0x7f45f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:49.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.715+0000 7f45f2ffd700 1 -- 172.21.15.97:0/1614435910 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45fc0565d0 con 0x7f460415f9e0 2023-11-24T06:10:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.904+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f460406c240 con 0x7f460415f9e0 2023-11-24T06:10:49.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.904+0000 7f45f2ffd700 1 -- 172.21.15.97:0/1614435910 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45fc059bf0 con 0x7f460415f9e0 2023-11-24T06:10:49.907 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:10:49.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f45e406b110 msgr2=0x7f45e406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f45e406b110 0x7f45e406d5d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f45f4006fd0 tx=0x7f45f4008040 comp rx=0 tx=0).stop 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f460415f9e0 msgr2=0x7f46041564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f460415f9e0 0x7f46041564a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4604073030 tx=0x7f45fc004ad0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 shutdown_connections 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f460415f9e0 0x7f46041564a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4604073f90 0x7f4604142690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f45e406b110 0x7f45e406d5d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 --2- 172.21.15.97:0/1614435910 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46040cf610 0x7f4604155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.906+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 >> 172.21.15.97:0/1614435910 conn(0x7f46040d0bf0 msgr2=0x7f46040b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.907+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 shutdown_connections 2023-11-24T06:10:49.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:49.907+0000 7f460891a700 1 -- 172.21.15.97:0/1614435910 wait complete. 2023-11-24T06:10:49.921 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.038+0000 7f1f9551a700 1 Processor -- start 2023-11-24T06:10:50.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.038+0000 7f1f9551a700 1 -- start start 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f9551a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1f900b0f80 0x7f1f9015a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f9551a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1f900cd770 0x7f1f9015a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f9551a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900b2bd0 con 0x7f1f900b0ba0 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f9551a700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900b1fc0 con 0x7f1f900b0f80 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f9551a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900b22a0 con 0x7f1f900cd770 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f8e7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1f900b0f80 0x7f1f9015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f8e7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1f900b0f80 0x7f1f9015a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48602/0 (socket says 172.21.15.97:48602) 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f8e7fc700 1 -- 172.21.15.97:0/516068591 learned_addr learned my addr 172.21.15.97:0/516068591 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.039+0000 7f1f8f7fe700 1 --2- 172.21.15.97:0/516068591 >> v2:172.21.15.183:3300/0 conn(0x7f1f900cd770 0x7f1f9015a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8e7fc700 1 -- 172.21.15.97:0/516068591 >> v2:172.21.15.183:3300/0 conn(0x7f1f900cd770 msgr2=0x7f1f9015a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8e7fc700 1 --2- 172.21.15.97:0/516068591 >> v2:172.21.15.183:3300/0 conn(0x7f1f900cd770 0x7f1f9015a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8e7fc700 1 -- 172.21.15.97:0/516068591 >> v1:172.21.15.183:6789/0 conn(0x7f1f900b0ba0 legacy=0x7f1f900af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8e7fc700 1 -- 172.21.15.97:0/516068591 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900b2580 con 0x7f1f900b0f80 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8f7fe700 1 --2- 172.21.15.97:0/516068591 >> v2:172.21.15.183:3300/0 conn(0x7f1f900cd770 0x7f1f9015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8e7fc700 1 --2- 172.21.15.97:0/516068591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0f80 0x7f1f9015a440 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1f900b2b40 tx=0x7f1f80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61edd9d10538961c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8dffb700 1 -- 172.21.15.97:0/516068591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000b870 con 0x7f1f900b0f80 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8dffb700 1 -- 172.21.15.97:0/516068591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8000b9d0 con 0x7f1f900b0f80 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f8dffb700 1 -- 172.21.15.97:0/516068591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8000bd60 con 0x7f1f900b0f80 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f9551a700 1 -- 172.21.15.97:0/516068591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0f80 msgr2=0x7f1f9015a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.040+0000 7f1f9551a700 1 --2- 172.21.15.97:0/516068591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0f80 0x7f1f9015a440 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1f900b2b40 tx=0x7f1f80009bd0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.041+0000 7f1f9551a700 1 -- 172.21.15.97:0/516068591 shutdown_connections 2023-11-24T06:10:50.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.041+0000 7f1f9551a700 1 --2- 172.21.15.97:0/516068591 >> v2:172.21.15.183:3300/0 conn(0x7f1f900cd770 0x7f1f9015a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.041+0000 7f1f9551a700 1 --2- 172.21.15.97:0/516068591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0f80 0x7f1f9015a440 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.041+0000 7f1f9551a700 1 -- 172.21.15.97:0/516068591 >> 172.21.15.97:0/516068591 conn(0x7f1f900b57b0 msgr2=0x7f1f900b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:50.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.041+0000 7f1f9551a700 1 -- 172.21.15.97:0/516068591 shutdown_connections 2023-11-24T06:10:50.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.041+0000 7f1f9551a700 1 -- 172.21.15.97:0/516068591 wait complete. 2023-11-24T06:10:50.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.042+0000 7f1f9551a700 1 Processor -- start 2023-11-24T06:10:50.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.042+0000 7f1f9551a700 1 -- start start 2023-11-24T06:10:50.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f9551a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0ba0 0x7f1f9015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f9551a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f900b0f80 0x7f1f900ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f9551a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f900cd770 0x7f1f9015ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f9551a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015c320 con 0x7f1f900b0ba0 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f9551a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015b0d0 con 0x7f1f900cd770 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f9551a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900c96b0 con 0x7f1f900b0f80 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0ba0 0x7f1f9015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0ba0 0x7f1f9015a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48606/0 (socket says 172.21.15.97:48606) 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 -- 172.21.15.97:0/179356405 learned_addr learned my addr 172.21.15.97:0/179356405 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8e7fc700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f900b0f80 0x7f1f900ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8f7fe700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f900cd770 0x7f1f9015ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 -- 172.21.15.97:0/179356405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f900b0f80 msgr2=0x7f1f900ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f900b0f80 0x7f1f900ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 -- 172.21.15.97:0/179356405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f900cd770 msgr2=0x7f1f9015ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f900cd770 0x7f1f9015ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 -- 172.21.15.97:0/179356405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009550 con 0x7f1f900b0ba0 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f8effd700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0ba0 0x7f1f9015a440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400c4e0 tx=0x7f1f8400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f777fe700 1 -- 172.21.15.97:0/179356405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840052d0 con 0x7f1f900b0ba0 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f777fe700 1 -- 172.21.15.97:0/179356405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f84005430 con 0x7f1f900b0ba0 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900c9990 con 0x7f1f900b0ba0 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f777fe700 1 -- 172.21.15.97:0/179356405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840038c0 con 0x7f1f900b0ba0 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.043+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f900c9f50 con 0x7f1f900b0ba0 2023-11-24T06:10:50.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.044+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f900ca0e0 con 0x7f1f900b0ba0 2023-11-24T06:10:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.045+0000 7f1f777fe700 1 -- 172.21.15.97:0/179356405 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f1f84003a20 con 0x7f1f900b0ba0 2023-11-24T06:10:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.045+0000 7f1f777fe700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1f7c06b110 0x7f1f7c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.045+0000 7f1f777fe700 1 -- 172.21.15.97:0/179356405 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f1f84088e80 con 0x7f1f900b0ba0 2023-11-24T06:10:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.045+0000 7f1f8e7fc700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1f7c06b110 0x7f1f7c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.046+0000 7f1f8e7fc700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1f7c06b110 0x7f1f7c06d5d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000c00 tx=0x7f1f80008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.050+0000 7f1f777fe700 1 -- 172.21.15.97:0/179356405 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f84053c30 con 0x7f1f900b0ba0 2023-11-24T06:10:50.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.281+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f90067f60 con 0x7f1f900b0ba0 2023-11-24T06:10:50.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.281+0000 7f1f777fe700 1 -- 172.21.15.97:0/179356405 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f1f84017090 con 0x7f1f900b0ba0 2023-11-24T06:10:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:50.284 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/163703518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":163703518},{"type":"v1","addr":"172.21.15.97:6837","nonce":163703518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/1688123009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1688123009},{"type":"v1","addr":"172.21.15.183:6834","nonce":1688123009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/310487911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":310487911},{"type":"v1","addr":"172.21.15.183:6835","nonce":310487911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:27.576219+0000","modified":"2023-11-24T06:10:30.898770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/19331625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":19331625},{"type":"v1","addr":"172.21.15.97:6835","nonce":19331625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1f7c06b110 msgr2=0x7f1f7c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1f7c06b110 0x7f1f7c06d5d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000c00 tx=0x7f1f80008030 comp rx=0 tx=0).stop 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0ba0 msgr2=0x7f1f9015a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0ba0 0x7f1f9015a440 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400c4e0 tx=0x7f1f8400c8a0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 shutdown_connections 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f900cd770 0x7f1f9015ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f900b0ba0 0x7f1f9015a440 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1f7c06b110 0x7f1f7c06d5d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 --2- 172.21.15.97:0/179356405 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1f900b0f80 0x7f1f900ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 >> 172.21.15.97:0/179356405 conn(0x7f1f900b57b0 msgr2=0x7f1f90159810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:50.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 shutdown_connections 2023-11-24T06:10:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.284+0000 7f1f9551a700 1 -- 172.21.15.97:0/179356405 wait complete. 2023-11-24T06:10:50.288 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2023-11-24T06:10:50.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.415+0000 7fad470de700 1 Processor -- start 2023-11-24T06:10:50.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.415+0000 7fad470de700 1 -- start start 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad470de700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fad401473a0 0x7fad4015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad470de700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad4006dc20 0x7fad40073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad470de700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4015eb00 con 0x7fad40147780 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad470de700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4015ede0 con 0x7fad401473a0 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad470de700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4015f0c0 con 0x7fad4006dc20 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad44e7a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fad401473a0 0x7fad4015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad44e7a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fad401473a0 0x7fad4015e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:42774/0 (socket says 172.21.15.97:42774) 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad44e7a700 1 -- 172.21.15.97:0/1621281996 learned_addr learned my addr 172.21.15.97:0/1621281996 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad3ffff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fad40147780 0x7fad4006c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:50390/0 (socket says 172.21.15.97:50390) 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.416+0000 7fad4567b700 1 --2- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad4006dc20 0x7fad40073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad44e7a700 1 -- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad4006dc20 msgr2=0x7fad40073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad44e7a700 1 --2- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad4006dc20 0x7fad40073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad44e7a700 1 -- 172.21.15.97:0/1621281996 >> v1:172.21.15.183:6789/0 conn(0x7fad40147780 legacy=0x7fad4006c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad44e7a700 1 -- 172.21.15.97:0/1621281996 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4015f3a0 con 0x7fad401473a0 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad3f7fe700 1 -- 172.21.15.97:0/1621281996 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991354468 0 0) 0x7fad4015eb00 con 0x7fad40147780 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad4567b700 1 --2- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad4006dc20 0x7fad40073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad44e7a700 1 --2- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad401473a0 0x7fad4015e4b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad340098f0 tx=0x7fad34009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=138c5d04fa2cc826 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad3f7fe700 1 -- 172.21.15.97:0/1621281996 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34011830 con 0x7fad401473a0 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad3f7fe700 1 -- 172.21.15.97:0/1621281996 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad34011990 con 0x7fad401473a0 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.417+0000 7fad3f7fe700 1 -- 172.21.15.97:0/1621281996 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34011d20 con 0x7fad401473a0 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.418+0000 7fad470de700 1 -- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad401473a0 msgr2=0x7fad4015e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.418+0000 7fad470de700 1 --2- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad401473a0 0x7fad4015e4b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad340098f0 tx=0x7fad34009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.418+0000 7fad470de700 1 -- 172.21.15.97:0/1621281996 shutdown_connections 2023-11-24T06:10:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.418+0000 7fad470de700 1 --2- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad4006dc20 0x7fad40073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.418+0000 7fad470de700 1 --2- 172.21.15.97:0/1621281996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad401473a0 0x7fad4015e4b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.418+0000 7fad470de700 1 -- 172.21.15.97:0/1621281996 >> 172.21.15.97:0/1621281996 conn(0x7fad400b3130 msgr2=0x7fad400d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.418+0000 7fad470de700 1 -- 172.21.15.97:0/1621281996 shutdown_connections 2023-11-24T06:10:50.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.418+0000 7fad470de700 1 -- 172.21.15.97:0/1621281996 wait complete. 2023-11-24T06:10:50.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.419+0000 7fad470de700 1 Processor -- start 2023-11-24T06:10:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.419+0000 7fad470de700 1 -- start start 2023-11-24T06:10:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad470de700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad4006dc20 0x7fad4013c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad470de700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad401473a0 0x7fad40140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad470de700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad40147780 0x7fad4013dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad470de700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad400ce680 con 0x7fad4006dc20 2023-11-24T06:10:50.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad470de700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4013e130 con 0x7fad40147780 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad470de700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad40140760 con 0x7fad401473a0 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad4006dc20 0x7fad4013c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad4006dc20 0x7fad4013c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48620/0 (socket says 172.21.15.97:48620) 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 -- 172.21.15.97:0/3628754750 learned_addr learned my addr 172.21.15.97:0/3628754750 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad3ffff700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad401473a0 0x7fad40140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad4567b700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad40147780 0x7fad4013dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 -- 172.21.15.97:0/3628754750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad401473a0 msgr2=0x7fad40140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad401473a0 0x7fad40140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 -- 172.21.15.97:0/3628754750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad40147780 msgr2=0x7fad4013dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad40147780 0x7fad4013dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 -- 172.21.15.97:0/3628754750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad38009a70 con 0x7fad4006dc20 2023-11-24T06:10:50.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.420+0000 7fad44e7a700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad4006dc20 0x7fad4013c0d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fad34009c40 tx=0x7fad34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.421+0000 7fad3d7fa700 1 -- 172.21.15.97:0/3628754750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34004340 con 0x7fad4006dc20 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.421+0000 7fad3d7fa700 1 -- 172.21.15.97:0/3628754750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3401f070 con 0x7fad4006dc20 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.421+0000 7fad3d7fa700 1 -- 172.21.15.97:0/3628754750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad340036a0 con 0x7fad4006dc20 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.421+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad34009550 con 0x7fad4006dc20 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.421+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad400ce030 con 0x7fad4006dc20 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.421+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad40067f60 con 0x7fad4006dc20 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.422+0000 7fad3d7fa700 1 -- 172.21.15.97:0/3628754750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fad34021030 con 0x7fad4006dc20 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.422+0000 7fad3d7fa700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad2406b110 0x7fad2406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.422+0000 7fad3d7fa700 1 -- 172.21.15.97:0/3628754750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fad3408acb0 con 0x7fad4006dc20 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.423+0000 7fad3ffff700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad2406b110 0x7fad2406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.423+0000 7fad3ffff700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad2406b110 0x7fad2406d5d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad400cdc40 tx=0x7fad2c007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.427+0000 7fad3d7fa700 1 -- 172.21.15.97:0/3628754750 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad34055b40 con 0x7fad4006dc20 2023-11-24T06:10:50.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.616+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad4006c240 con 0x7fad4006dc20 2023-11-24T06:10:50.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.618+0000 7fad3d7fa700 1 -- 172.21.15.97:0/3628754750 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16785 (secure 0 0 0) 0x7fad34015020 con 0x7fad4006dc20 2023-11-24T06:10:50.621 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:50.621 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":48,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:47.603703+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:25.335173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:26.547561+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.37","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:50.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad2406b110 msgr2=0x7fad2406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad2406b110 0x7fad2406d5d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fad400cdc40 tx=0x7fad2c007e70 comp rx=0 tx=0).stop 2023-11-24T06:10:50.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad4006dc20 msgr2=0x7fad4013c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad4006dc20 0x7fad4013c0d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fad34009c40 tx=0x7fad34004750 comp rx=0 tx=0).stop 2023-11-24T06:10:50.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 shutdown_connections 2023-11-24T06:10:50.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad40147780 0x7fad4013dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad4006dc20 0x7fad4013c0d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fad2406b110 0x7fad2406d5d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 --2- 172.21.15.97:0/3628754750 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fad401473a0 0x7fad40140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 >> 172.21.15.97:0/3628754750 conn(0x7fad400b3130 msgr2=0x7fad4006b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:50.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 shutdown_connections 2023-11-24T06:10:50.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.620+0000 7fad470de700 1 -- 172.21.15.97:0/3628754750 wait complete. 2023-11-24T06:10:50.636 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:10:50.636 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:50.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.754+0000 7fc4c6c75700 1 Processor -- start 2023-11-24T06:10:50.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.754+0000 7fc4c6c75700 1 -- start start 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c6c75700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4c015e080 0x7fc4c00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c6c75700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c00af4d0 0x7fc4c00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c6c75700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc4c006c000 0x7fc4c006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c6c75700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c00af0b0 con 0x7fc4c015e080 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c6c75700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c006c9f0 con 0x7fc4c006c000 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c6c75700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00b05a0 con 0x7fc4c00af4d0 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4c015e080 0x7fc4c00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4c015e080 0x7fc4c00b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48628/0 (socket says 172.21.15.97:48628) 2023-11-24T06:10:50.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 -- 172.21.15.97:0/2107629427 learned_addr learned my addr 172.21.15.97:0/2107629427 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4bffff700 1 --2- 172.21.15.97:0/2107629427 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c00af4d0 0x7fc4c00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c5212700 1 --2- 172.21.15.97:0/2107629427 >> v2:172.21.15.183:3300/0 conn(0x7fc4c006c000 0x7fc4c006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 -- 172.21.15.97:0/2107629427 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c00af4d0 msgr2=0x7fc4c00afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 --2- 172.21.15.97:0/2107629427 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c00af4d0 0x7fc4c00afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 -- 172.21.15.97:0/2107629427 >> v2:172.21.15.183:3300/0 conn(0x7fc4c006c000 msgr2=0x7fc4c006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 --2- 172.21.15.97:0/2107629427 >> v2:172.21.15.183:3300/0 conn(0x7fc4c006c000 0x7fc4c006c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 -- 172.21.15.97:0/2107629427 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00ae430 con 0x7fc4c015e080 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c5212700 1 --2- 172.21.15.97:0/2107629427 >> v2:172.21.15.183:3300/0 conn(0x7fc4c006c000 0x7fc4c006c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4bffff700 1 --2- 172.21.15.97:0/2107629427 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c00af4d0 0x7fc4c00afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4c4a11700 1 --2- 172.21.15.97:0/2107629427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c015e080 0x7fc4c00b5050 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc4c009eea0 tx=0x7fc4b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec58c18dbcb71ff8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4bf7fe700 1 -- 172.21.15.97:0/2107629427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400b860 con 0x7fc4c015e080 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.755+0000 7fc4bf7fe700 1 -- 172.21.15.97:0/2107629427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b400b9c0 con 0x7fc4c015e080 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4bf7fe700 1 -- 172.21.15.97:0/2107629427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b400bd50 con 0x7fc4c015e080 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 -- 172.21.15.97:0/2107629427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c015e080 msgr2=0x7fc4c00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/2107629427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c015e080 0x7fc4c00b5050 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc4c009eea0 tx=0x7fc4b4009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 -- 172.21.15.97:0/2107629427 shutdown_connections 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/2107629427 >> v2:172.21.15.183:3300/0 conn(0x7fc4c006c000 0x7fc4c006c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/2107629427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c015e080 0x7fc4c00b5050 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/2107629427 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c00af4d0 0x7fc4c00afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 -- 172.21.15.97:0/2107629427 >> 172.21.15.97:0/2107629427 conn(0x7fc4c0071b90 msgr2=0x7fc4c0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 -- 172.21.15.97:0/2107629427 shutdown_connections 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.756+0000 7fc4c6c75700 1 -- 172.21.15.97:0/2107629427 wait complete. 2023-11-24T06:10:50.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.757+0000 7fc4c6c75700 1 Processor -- start 2023-11-24T06:10:50.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.757+0000 7fc4c6c75700 1 -- start start 2023-11-24T06:10:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c6c75700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c006c000 0x7fc4c01483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c6c75700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c00af4d0 0x7fc4c0159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c6c75700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c015e080 0x7fc4c00cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c6c75700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00cf570 con 0x7fc4c006c000 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c6c75700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015fcf0 con 0x7fc4c00af4d0 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c6c75700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015ffd0 con 0x7fc4c015e080 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c4a11700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c006c000 0x7fc4c01483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c4a11700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c006c000 0x7fc4c01483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48634/0 (socket says 172.21.15.97:48634) 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c4a11700 1 -- 172.21.15.97:0/3698903854 learned_addr learned my addr 172.21.15.97:0/3698903854 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4bffff700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c00af4d0 0x7fc4c0159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.758+0000 7fc4c5212700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c015e080 0x7fc4c00cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4c4a11700 1 -- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c015e080 msgr2=0x7fc4c00cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4c4a11700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c015e080 0x7fc4c00cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4c4a11700 1 -- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c00af4d0 msgr2=0x7fc4c0159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4c4a11700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c00af4d0 0x7fc4c0159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:50.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4c4a11700 1 -- 172.21.15.97:0/3698903854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b4009550 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4bffff700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c00af4d0 0x7fc4c0159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4c4a11700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c006c000 0x7fc4c01483a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4014040 tx=0x7fc4b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4bd7fa700 1 -- 172.21.15.97:0/3698903854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40036a0 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4bd7fa700 1 -- 172.21.15.97:0/3698903854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b401e070 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c01602b0 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4bd7fa700 1 -- 172.21.15.97:0/3698903854 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4003a60 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.759+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c015f9a0 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.760+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c00753f0 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.760+0000 7fc4bd7fa700 1 -- 172.21.15.97:0/3698903854 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc4b4003bc0 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.761+0000 7fc4bd7fa700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc4a406b110 0x7fc4a406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.761+0000 7fc4bd7fa700 1 -- 172.21.15.97:0/3698903854 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fc4b4020070 con 0x7fc4c006c000 2023-11-24T06:10:50.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.761+0000 7fc4bffff700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc4a406b110 0x7fc4a406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:50.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.761+0000 7fc4bffff700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc4a406b110 0x7fc4a406d5d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac006fd0 tx=0x7fc4ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:50.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.765+0000 7fc4bd7fa700 1 -- 172.21.15.97:0/3698903854 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4b4055d80 con 0x7fc4c006c000 2023-11-24T06:10:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.997+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4c0068670 con 0x7fc4c006c000 2023-11-24T06:10:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.997+0000 7fc4bd7fa700 1 -- 172.21.15.97:0/3698903854 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fc4b402a090 con 0x7fc4c006c000 2023-11-24T06:10:51.000 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:51.000 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/163703518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":163703518},{"type":"v1","addr":"172.21.15.97:6837","nonce":163703518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/1688123009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1688123009},{"type":"v1","addr":"172.21.15.183:6834","nonce":1688123009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/310487911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":310487911},{"type":"v1","addr":"172.21.15.183:6835","nonce":310487911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:27.576219+0000","modified":"2023-11-24T06:10:30.898770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/19331625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":19331625},{"type":"v1","addr":"172.21.15.97:6835","nonce":19331625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:10:51.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc4a406b110 msgr2=0x7fc4a406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc4a406b110 0x7fc4a406d5d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac006fd0 tx=0x7fc4ac008040 comp rx=0 tx=0).stop 2023-11-24T06:10:51.002 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c006c000 msgr2=0x7fc4c01483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c006c000 0x7fc4c01483a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4014040 tx=0x7fc4b4004750 comp rx=0 tx=0).stop 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 shutdown_connections 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc4a406b110 0x7fc4a406d5d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc4c015e080 0x7fc4c00cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4c006c000 0x7fc4c01483a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 --2- 172.21.15.97:0/3698903854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc4c00af4d0 0x7fc4c0159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 >> 172.21.15.97:0/3698903854 conn(0x7fc4c0071b90 msgr2=0x7fc4c015d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 shutdown_connections 2023-11-24T06:10:51.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:50.999+0000 7fc4c6c75700 1 -- 172.21.15.97:0/3698903854 wait complete. 2023-11-24T06:10:51.004 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2023-11-24T06:10:51.014 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.132+0000 7f6462b33700 1 Processor -- start 2023-11-24T06:10:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.132+0000 7f6462b33700 1 -- start start 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.132+0000 7f6462b33700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f645c06c710 0x7f645c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.132+0000 7f6462b33700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f645c0af4d0 0x7f645c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.132+0000 7f6462b33700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c0ae430 con 0x7f645c15e080 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.132+0000 7f6462b33700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0af0b0 con 0x7f645c06c710 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.132+0000 7f6462b33700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0b05a0 con 0x7f645c0af4d0 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f645c06c710 0x7f645c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f645c06c710 0x7f645c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48644/0 (socket says 172.21.15.97:48644) 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645bfff700 1 -- 172.21.15.97:0/927799803 learned_addr learned my addr 172.21.15.97:0/927799803 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f64610d0700 1 --2- 172.21.15.97:0/927799803 >> v2:172.21.15.183:3300/0 conn(0x7f645c0af4d0 0x7f645c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645bfff700 1 -- 172.21.15.97:0/927799803 >> v2:172.21.15.183:3300/0 conn(0x7f645c0af4d0 msgr2=0x7f645c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645bfff700 1 --2- 172.21.15.97:0/927799803 >> v2:172.21.15.183:3300/0 conn(0x7f645c0af4d0 0x7f645c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645bfff700 1 -- 172.21.15.97:0/927799803 >> v1:172.21.15.183:6789/0 conn(0x7f645c15e080 legacy=0x7f645c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645bfff700 1 -- 172.21.15.97:0/927799803 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0ae710 con 0x7f645c06c710 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645bfff700 1 --2- 172.21.15.97:0/927799803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c0b5050 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f645c09eea0 tx=0x7f6448009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b092376b5ca2c9e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645b7fe700 1 -- 172.21.15.97:0/927799803 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448004230 con 0x7f645c06c710 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645b7fe700 1 -- 172.21.15.97:0/927799803 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6448004390 con 0x7f645c06c710 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.133+0000 7f645b7fe700 1 -- 172.21.15.97:0/927799803 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480036a0 con 0x7f645c06c710 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.134+0000 7f6462b33700 1 -- 172.21.15.97:0/927799803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 msgr2=0x7f645c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.134+0000 7f6462b33700 1 --2- 172.21.15.97:0/927799803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c0b5050 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f645c09eea0 tx=0x7f6448009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.134+0000 7f6462b33700 1 -- 172.21.15.97:0/927799803 shutdown_connections 2023-11-24T06:10:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.134+0000 7f6462b33700 1 --2- 172.21.15.97:0/927799803 >> v2:172.21.15.183:3300/0 conn(0x7f645c0af4d0 0x7f645c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.134+0000 7f6462b33700 1 --2- 172.21.15.97:0/927799803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c0b5050 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.134+0000 7f6462b33700 1 -- 172.21.15.97:0/927799803 >> 172.21.15.97:0/927799803 conn(0x7f645c071b90 msgr2=0x7f645c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:51.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.134+0000 7f6462b33700 1 -- 172.21.15.97:0/927799803 shutdown_connections 2023-11-24T06:10:51.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.134+0000 7f6462b33700 1 -- 172.21.15.97:0/927799803 wait complete. 2023-11-24T06:10:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.135+0000 7f6462b33700 1 Processor -- start 2023-11-24T06:10:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.135+0000 7f6462b33700 1 -- start start 2023-11-24T06:10:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.136+0000 7f6462b33700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.136+0000 7f6462b33700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f645c0af4d0 0x7f645c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.136+0000 7f6462b33700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645c15e080 0x7f645c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.136+0000 7f6462b33700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c15f2c0 con 0x7f645c06c710 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.136+0000 7f6462b33700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c15ecd0 con 0x7f645c15e080 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.136+0000 7f6462b33700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c0cfb40 con 0x7f645c0af4d0 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48648/0 (socket says 172.21.15.97:48648) 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 -- 172.21.15.97:0/1217822326 learned_addr learned my addr 172.21.15.97:0/1217822326 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f645bfff700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f645c0af4d0 0x7f645c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64610d0700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645c15e080 0x7f645c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 -- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f645c0af4d0 msgr2=0x7f645c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f645c0af4d0 0x7f645c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 -- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645c15e080 msgr2=0x7f645c157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645c15e080 0x7f645c157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 -- 172.21.15.97:0/1217822326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6448009550 con 0x7f645c06c710 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64610d0700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645c15e080 0x7f645c157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64608cf700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c145460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f645000b4a0 tx=0x7f645000b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64597fa700 1 -- 172.21.15.97:0/1217822326 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450011830 con 0x7f645c06c710 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64597fa700 1 -- 172.21.15.97:0/1217822326 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6450011990 con 0x7f645c06c710 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f64597fa700 1 -- 172.21.15.97:0/1217822326 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450011d40 con 0x7f645c06c710 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c0cfe20 con 0x7f645c06c710 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.137+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c1459a0 con 0x7f645c06c710 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.138+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c15f530 con 0x7f645c06c710 2023-11-24T06:10:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.139+0000 7f64597fa700 1 -- 172.21.15.97:0/1217822326 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f645001a400 con 0x7f645c06c710 2023-11-24T06:10:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.139+0000 7f64597fa700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f644006b110 0x7f644006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.139+0000 7f64597fa700 1 -- 172.21.15.97:0/1217822326 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f6450089710 con 0x7f645c06c710 2023-11-24T06:10:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.139+0000 7f645bfff700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f644006b110 0x7f644006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.139+0000 7f645bfff700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f644006b110 0x7f644006d5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6448009130 tx=0x7f6448008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.143+0000 7f64597fa700 1 -- 172.21.15.97:0/1217822326 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64500543f0 con 0x7f645c06c710 2023-11-24T06:10:51.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.380+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f645c0753f0 con 0x7f645c06c710 2023-11-24T06:10:51.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.380+0000 7f64597fa700 1 -- 172.21.15.97:0/1217822326 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f6450057a10 con 0x7f645c06c710 2023-11-24T06:10:51.383 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:51.383 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/163703518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":163703518},{"type":"v1","addr":"172.21.15.97:6837","nonce":163703518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/1688123009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1688123009},{"type":"v1","addr":"172.21.15.183:6834","nonce":1688123009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/310487911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":310487911},{"type":"v1","addr":"172.21.15.183:6835","nonce":310487911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:10:27.576219+0000","modified":"2023-11-24T06:10:30.898770+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5532},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5532":{"gid":5532,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6835/19331625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":19331625},{"type":"v1","addr":"172.21.15.97:6835","nonce":19331625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T06:10:51.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.382+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f644006b110 msgr2=0x7f644006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.382+0000 7f6462b33700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f644006b110 0x7f644006d5d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6448009130 tx=0x7f6448008030 comp rx=0 tx=0).stop 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.382+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 msgr2=0x7f645c145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.382+0000 7f6462b33700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c145460 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f645000b4a0 tx=0x7f645000b7b0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.383+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 shutdown_connections 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.383+0000 7f6462b33700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f645c15e080 0x7f645c157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.383+0000 7f6462b33700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c06c710 0x7f645c145460 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.383+0000 7f6462b33700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f644006b110 0x7f644006d5d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.383+0000 7f6462b33700 1 --2- 172.21.15.97:0/1217822326 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f645c0af4d0 0x7f645c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.383+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 >> 172.21.15.97:0/1217822326 conn(0x7f645c071b90 msgr2=0x7f645c0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.383+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 shutdown_connections 2023-11-24T06:10:51.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.383+0000 7f6462b33700 1 -- 172.21.15.97:0/1217822326 wait complete. 2023-11-24T06:10:51.387 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 27 2023-11-24T06:10:51.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.515+0000 7f7256c26700 1 Processor -- start 2023-11-24T06:10:51.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.515+0000 7f7256c26700 1 -- start start 2023-11-24T06:10:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f7256c26700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72500d2240 0x7f72500b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f7256c26700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f725015e1b0 0x7f72500cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f7256c26700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72500b1d40 con 0x7f725015ddd0 2023-11-24T06:10:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f7256c26700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72500b28c0 con 0x7f72500d2240 2023-11-24T06:10:51.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f7256c26700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725006c000 con 0x7f725015e1b0 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72500d2240 0x7f72500b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72500d2240 0x7f72500b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48658/0 (socket says 172.21.15.97:48658) 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724ffff700 1 -- 172.21.15.97:0/508631540 learned_addr learned my addr 172.21.15.97:0/508631540 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f72551c3700 1 --2- 172.21.15.97:0/508631540 >> v2:172.21.15.183:3300/0 conn(0x7f725015e1b0 0x7f72500cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724ffff700 1 -- 172.21.15.97:0/508631540 >> v2:172.21.15.183:3300/0 conn(0x7f725015e1b0 msgr2=0x7f72500cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724ffff700 1 --2- 172.21.15.97:0/508631540 >> v2:172.21.15.183:3300/0 conn(0x7f725015e1b0 0x7f72500cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724ffff700 1 -- 172.21.15.97:0/508631540 >> v1:172.21.15.97:6789/0 conn(0x7f725015ddd0 legacy=0x7f72500d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724ffff700 1 -- 172.21.15.97:0/508631540 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72500b1f90 con 0x7f72500d2240 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724f7fe700 1 -- 172.21.15.97:0/508631540 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950328213 0 0) 0x7f72500b1d40 con 0x7f725015ddd0 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724ffff700 1 --2- 172.21.15.97:0/508631540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72500d2240 0x7f72500b1680 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f723c0098f0 tx=0x7f723c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6683dc5c1be277fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.516+0000 7f724f7fe700 1 -- 172.21.15.97:0/508631540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00b860 con 0x7f72500d2240 2023-11-24T06:10:51.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f724f7fe700 1 -- 172.21.15.97:0/508631540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c00b9c0 con 0x7f72500d2240 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f724f7fe700 1 -- 172.21.15.97:0/508631540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c00bd50 con 0x7f72500d2240 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f7256c26700 1 -- 172.21.15.97:0/508631540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72500d2240 msgr2=0x7f72500b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f7256c26700 1 --2- 172.21.15.97:0/508631540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72500d2240 0x7f72500b1680 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f723c0098f0 tx=0x7f723c009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f7256c26700 1 -- 172.21.15.97:0/508631540 shutdown_connections 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f7256c26700 1 --2- 172.21.15.97:0/508631540 >> v2:172.21.15.183:3300/0 conn(0x7f725015e1b0 0x7f72500cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f7256c26700 1 --2- 172.21.15.97:0/508631540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72500d2240 0x7f72500b1680 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f7256c26700 1 -- 172.21.15.97:0/508631540 >> 172.21.15.97:0/508631540 conn(0x7f72500731a0 msgr2=0x7f725009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f7256c26700 1 -- 172.21.15.97:0/508631540 shutdown_connections 2023-11-24T06:10:51.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.517+0000 7f7256c26700 1 -- 172.21.15.97:0/508631540 wait complete. 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.519+0000 7f7256c26700 1 Processor -- start 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.519+0000 7f7256c26700 1 -- start start 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f7256c26700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72500d2240 0x7f72500b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f7256c26700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725015ddd0 0x7f72500cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f7256c26700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f725015e1b0 0x7f725015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f7256c26700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72500b45c0 con 0x7f725015ddd0 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f7256c26700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72500b48a0 con 0x7f72500d2240 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f7256c26700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72500b4b80 con 0x7f725015e1b0 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725015ddd0 0x7f72500cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725015ddd0 0x7f72500cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48664/0 (socket says 172.21.15.97:48664) 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 -- 172.21.15.97:0/2323107911 learned_addr learned my addr 172.21.15.97:0/2323107911 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 -- 172.21.15.97:0/2323107911 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f725015e1b0 msgr2=0x7f725015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f725015e1b0 0x7f725015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 -- 172.21.15.97:0/2323107911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72500d2240 msgr2=0x7f72500b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72500d2240 0x7f72500b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 -- 172.21.15.97:0/2323107911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c009550 con 0x7f725015ddd0 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724ffff700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725015ddd0 0x7f72500cdad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f723c009c40 tx=0x7f723c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724d7fa700 1 -- 172.21.15.97:0/2323107911 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c004400 con 0x7f725015ddd0 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724d7fa700 1 -- 172.21.15.97:0/2323107911 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c01f070 con 0x7f725015ddd0 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72500b2ed0 con 0x7f725015ddd0 2023-11-24T06:10:51.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f724d7fa700 1 -- 172.21.15.97:0/2323107911 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c0038c0 con 0x7f725015ddd0 2023-11-24T06:10:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.520+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72500b5140 con 0x7f725015ddd0 2023-11-24T06:10:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.521+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72500b5300 con 0x7f725015ddd0 2023-11-24T06:10:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.522+0000 7f724d7fa700 1 -- 172.21.15.97:0/2323107911 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f723c021030 con 0x7f725015ddd0 2023-11-24T06:10:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.522+0000 7f724d7fa700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f723406b110 0x7f723406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.522+0000 7f724d7fa700 1 -- 172.21.15.97:0/2323107911 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f723c08adb0 con 0x7f725015ddd0 2023-11-24T06:10:51.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.522+0000 7f72549c2700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f723406b110 0x7f723406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.522+0000 7f72549c2700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f723406b110 0x7f723406d5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7244007fd0 tx=0x7f72440079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.527+0000 7f724d7fa700 1 -- 172.21.15.97:0/2323107911 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f723c055bd0 con 0x7f725015ddd0 2023-11-24T06:10:51.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.719+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7250067f60 con 0x7f725015ddd0 2023-11-24T06:10:51.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.721+0000 7f724d7fa700 1 -- 172.21.15.97:0/2323107911 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16785 (secure 0 0 0) 0x7f723c015020 con 0x7f725015ddd0 2023-11-24T06:10:51.724 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:51.724 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":48,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:47.603703+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:10:25.335173+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T06:10:26.547561+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.37","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f723406b110 msgr2=0x7f723406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f723406b110 0x7f723406d5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7244007fd0 tx=0x7f72440079a0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725015ddd0 msgr2=0x7f72500cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725015ddd0 0x7f72500cdad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f723c009c40 tx=0x7f723c004750 comp rx=0 tx=0).stop 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 shutdown_connections 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f723406b110 0x7f723406d5d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f725015e1b0 0x7f725015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72500d2240 0x7f72500b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 --2- 172.21.15.97:0/2323107911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f725015ddd0 0x7f72500cdad0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 >> 172.21.15.97:0/2323107911 conn(0x7f72500731a0 msgr2=0x7f72501606b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 shutdown_connections 2023-11-24T06:10:51.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.723+0000 7f7256c26700 1 -- 172.21.15.97:0/2323107911 wait complete. 2023-11-24T06:10:51.738 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:10:51.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.855+0000 7f88457c6700 1 Processor -- start 2023-11-24T06:10:51.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.855+0000 7f88457c6700 1 -- start start 2023-11-24T06:10:51.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f88457c6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f88400adf00 0x7f884006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f88457c6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f884006c540 0x7f88400b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f88457c6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88400af950 con 0x7f88401446a0 2023-11-24T06:10:51.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f88457c6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884006c9f0 con 0x7f884006c540 2023-11-24T06:10:51.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f88457c6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8840144a80 con 0x7f88400adf00 2023-11-24T06:10:51.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883effd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f88401446a0 0x7f88400b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41122/0 (socket says 172.21.15.97:41122) 2023-11-24T06:10:51.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883effd700 1 -- 172.21.15.97:0/1958823636 learned_addr learned my addr 172.21.15.97:0/1958823636 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:51.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883f7fe700 1 --2- 172.21.15.97:0/1958823636 >> v2:172.21.15.97:3300/0 conn(0x7f884006c540 0x7f88400b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883e7fc700 1 --2- 172.21.15.97:0/1958823636 >> v2:172.21.15.183:3300/0 conn(0x7f88400adf00 0x7f884006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883f7fe700 1 -- 172.21.15.97:0/1958823636 >> v2:172.21.15.183:3300/0 conn(0x7f88400adf00 msgr2=0x7f884006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883f7fe700 1 --2- 172.21.15.97:0/1958823636 >> v2:172.21.15.183:3300/0 conn(0x7f88400adf00 0x7f884006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883f7fe700 1 -- 172.21.15.97:0/1958823636 >> v1:172.21.15.97:6789/0 conn(0x7f88401446a0 legacy=0x7f88400b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883f7fe700 1 -- 172.21.15.97:0/1958823636 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400afc30 con 0x7f884006c540 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883e7fc700 1 --2- 172.21.15.97:0/1958823636 >> v2:172.21.15.183:3300/0 conn(0x7f88400adf00 0x7f884006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883f7fe700 1 --2- 172.21.15.97:0/1958823636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f884006c540 0x7f88400b3d40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f884009f030 tx=0x7f883800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c722fc6df74c9e8d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883dffb700 1 -- 172.21.15.97:0/1958823636 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883800e070 con 0x7f884006c540 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883dffb700 1 -- 172.21.15.97:0/1958823636 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883800bb90 con 0x7f884006c540 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.857+0000 7f883dffb700 1 -- 172.21.15.97:0/1958823636 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838012430 con 0x7f884006c540 2023-11-24T06:10:51.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.858+0000 7f88457c6700 1 -- 172.21.15.97:0/1958823636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f884006c540 msgr2=0x7f88400b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.858+0000 7f88457c6700 1 --2- 172.21.15.97:0/1958823636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f884006c540 0x7f88400b3d40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f884009f030 tx=0x7f883800b330 comp rx=0 tx=0).stop 2023-11-24T06:10:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.858+0000 7f88457c6700 1 -- 172.21.15.97:0/1958823636 shutdown_connections 2023-11-24T06:10:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.858+0000 7f88457c6700 1 --2- 172.21.15.97:0/1958823636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f884006c540 0x7f88400b3d40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.858+0000 7f88457c6700 1 --2- 172.21.15.97:0/1958823636 >> v2:172.21.15.183:3300/0 conn(0x7f88400adf00 0x7f884006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.858+0000 7f88457c6700 1 -- 172.21.15.97:0/1958823636 >> 172.21.15.97:0/1958823636 conn(0x7f8840071b90 msgr2=0x7f8840071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.858+0000 7f88457c6700 1 -- 172.21.15.97:0/1958823636 shutdown_connections 2023-11-24T06:10:51.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.858+0000 7f88457c6700 1 -- 172.21.15.97:0/1958823636 wait complete. 2023-11-24T06:10:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.859+0000 7f88457c6700 1 Processor -- start 2023-11-24T06:10:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.859+0000 7f88457c6700 1 -- start start 2023-11-24T06:10:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f88457c6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f884006c540 0x7f88400cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f88457c6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88400adf00 0x7f8840073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f88457c6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88401446a0 0x7f88400744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f88457c6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840137cb0 con 0x7f88401446a0 2023-11-24T06:10:51.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f88457c6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8840137f20 con 0x7f88400adf00 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f88457c6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8840138200 con 0x7f884006c540 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88401446a0 0x7f88400744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88401446a0 0x7f88400744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48680/0 (socket says 172.21.15.97:48680) 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883f7fe700 1 -- 172.21.15.97:0/3006498341 learned_addr learned my addr 172.21.15.97:0/3006498341 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883e7fc700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88400adf00 0x7f8840073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883effd700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f884006c540 0x7f88400cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883f7fe700 1 -- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f884006c540 msgr2=0x7f88400cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883f7fe700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f884006c540 0x7f88400cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883f7fe700 1 -- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88400adf00 msgr2=0x7f8840073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883f7fe700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88400adf00 0x7f8840073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883f7fe700 1 -- 172.21.15.97:0/3006498341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8838009a70 con 0x7f88401446a0 2023-11-24T06:10:51.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.860+0000 7f883e7fc700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88400adf00 0x7f8840073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.861+0000 7f883f7fe700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88401446a0 0x7f88400744d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f883800b370 tx=0x7f8838004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.861+0000 7f88277fe700 1 -- 172.21.15.97:0/3006498341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838012430 con 0x7f88401446a0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.861+0000 7f88277fe700 1 -- 172.21.15.97:0/3006498341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8838020070 con 0x7f88401446a0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.861+0000 7f88277fe700 1 -- 172.21.15.97:0/3006498341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838004280 con 0x7f88401446a0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.861+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88401384e0 con 0x7f88401446a0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.861+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88400d03b0 con 0x7f88401446a0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.862+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88400753f0 con 0x7f88401446a0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.862+0000 7f88277fe700 1 -- 172.21.15.97:0/3006498341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f88380043e0 con 0x7f88401446a0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.862+0000 7f88277fe700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f882c06b110 0x7f882c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.863+0000 7f88277fe700 1 -- 172.21.15.97:0/3006498341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8838022070 con 0x7f88401446a0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.863+0000 7f883effd700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f882c06b110 0x7f882c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.863+0000 7f883effd700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f882c06b110 0x7f882c06d5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8834005910 tx=0x7f8834009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:51.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:51.867+0000 7f88277fe700 1 -- 172.21.15.97:0/3006498341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f883808de60 con 0x7f88401446a0 2023-11-24T06:10:52.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.099+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f88401389a0 con 0x7f88401446a0 2023-11-24T06:10:52.103 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-11-24T06:10:52.100+0000 7fbb14a83700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:10:52.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.101+0000 7f88277fe700 1 -- 172.21.15.97:0/3006498341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7f883802c090 con 0x7f88401446a0 2023-11-24T06:10:52.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f882c06b110 msgr2=0x7f882c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:52.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f882c06b110 0x7f882c06d5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8834005910 tx=0x7f8834009040 comp rx=0 tx=0).stop 2023-11-24T06:10:52.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88401446a0 msgr2=0x7f88400744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:52.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88401446a0 0x7f88400744d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f883800b370 tx=0x7f8838004ad0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 shutdown_connections 2023-11-24T06:10:52.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88401446a0 0x7f88400744d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f882c06b110 0x7f882c06d5d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f884006c540 0x7f88400cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 --2- 172.21.15.97:0/3006498341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88400adf00 0x7f8840073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 >> 172.21.15.97:0/3006498341 conn(0x7f8840071b90 msgr2=0x7f88400b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:52.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 shutdown_connections 2023-11-24T06:10:52.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.105+0000 7f88457c6700 1 -- 172.21.15.97:0/3006498341 wait complete. 2023-11-24T06:10:52.109 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:10:52.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:10:52.147 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-11-24T06:10:52.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.245+0000 7f6b8ec8a700 1 Processor -- start 2023-11-24T06:10:52.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.245+0000 7f6b8ec8a700 1 -- start start 2023-11-24T06:10:52.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8ec8a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6b881587f0 0x7f6b8806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:52.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8ec8a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b8815b370 0x7f6b8815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:52.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8ec8a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b880b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:52.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8ec8a700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b880b4630 con 0x7f6b8815b370 2023-11-24T06:10:52.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8ec8a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b8806b530 con 0x7f6b881587f0 2023-11-24T06:10:52.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8ec8a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b880ae5e0 con 0x7f6b880b3c60 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b8815b370 0x7f6b8815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b8815b370 0x7f6b8815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48698/0 (socket says 172.21.15.97:48698) 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 -- 172.21.15.97:0/2349603362 learned_addr learned my addr 172.21.15.97:0/2349603362 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8ca26700 1 --2- 172.21.15.97:0/2349603362 >> v2:172.21.15.183:3300/0 conn(0x7f6b881587f0 0x7f6b8806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8d227700 1 --2- 172.21.15.97:0/2349603362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b880b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 -- 172.21.15.97:0/2349603362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 msgr2=0x7f6b880b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 --2- 172.21.15.97:0/2349603362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b880b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 -- 172.21.15.97:0/2349603362 >> v2:172.21.15.183:3300/0 conn(0x7f6b881587f0 msgr2=0x7f6b8806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 --2- 172.21.15.97:0/2349603362 >> v2:172.21.15.183:3300/0 conn(0x7f6b881587f0 0x7f6b8806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 -- 172.21.15.97:0/2349603362 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8815be30 con 0x7f6b8815b370 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b8ca26700 1 --2- 172.21.15.97:0/2349603362 >> v2:172.21.15.183:3300/0 conn(0x7f6b881587f0 0x7f6b8806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:52.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.246+0000 7f6b87fff700 1 --2- 172.21.15.97:0/2349603362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b8815b800 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6b700098f0 tx=0x7f6b70009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf71ae70834674c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b877fe700 1 -- 172.21.15.97:0/2349603362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7000b860 con 0x7f6b8815b370 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b877fe700 1 -- 172.21.15.97:0/2349603362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7000b9c0 con 0x7f6b8815b370 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b877fe700 1 -- 172.21.15.97:0/2349603362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7000bd50 con 0x7f6b8815b370 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/2349603362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 msgr2=0x7f6b8815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/2349603362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b8815b800 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6b700098f0 tx=0x7f6b70009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/2349603362 shutdown_connections 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/2349603362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b880b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/2349603362 >> v2:172.21.15.183:3300/0 conn(0x7f6b881587f0 0x7f6b8806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/2349603362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b8815b800 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/2349603362 >> 172.21.15.97:0/2349603362 conn(0x7f6b880d0bf0 msgr2=0x7f6b880d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:52.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/2349603362 shutdown_connections 2023-11-24T06:10:52.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.247+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/2349603362 wait complete. 2023-11-24T06:10:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.248+0000 7f6b8ec8a700 1 Processor -- start 2023-11-24T06:10:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.248+0000 7f6b8ec8a700 1 -- start start 2023-11-24T06:10:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8ec8a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b8814e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:52.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8ec8a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b881587f0 0x7f6b88152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8ec8a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b88152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8ec8a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b881521e0 con 0x7f6b8815b370 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8ec8a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b881530d0 con 0x7f6b881587f0 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8ec8a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b881533b0 con 0x7f6b880b3c60 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8d227700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b88152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8d227700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b88152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48706/0 (socket says 172.21.15.97:48706) 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.249+0000 7f6b8d227700 1 -- 172.21.15.97:0/231342675 learned_addr learned my addr 172.21.15.97:0/231342675 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b87fff700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b881587f0 0x7f6b88152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8ca26700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b8814e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8d227700 1 -- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 msgr2=0x7f6b8814e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:52.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8d227700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b8814e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8d227700 1 -- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b881587f0 msgr2=0x7f6b88152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8d227700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b881587f0 0x7f6b88152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8d227700 1 -- 172.21.15.97:0/231342675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b70009550 con 0x7f6b8815b370 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8ca26700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b8814e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8d227700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b88152a50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6b80007e60 tx=0x7f6b8000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b857fa700 1 -- 172.21.15.97:0/231342675 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000ede0 con 0x7f6b8815b370 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b857fa700 1 -- 172.21.15.97:0/231342675 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b80014420 con 0x7f6b8815b370 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b857fa700 1 -- 172.21.15.97:0/231342675 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800147d0 con 0x7f6b8815b370 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b88153ea0 con 0x7f6b8815b370 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.250+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8814ed20 con 0x7f6b8815b370 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.251+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b88068fb0 con 0x7f6b8815b370 2023-11-24T06:10:52.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.252+0000 7f6b857fa700 1 -- 172.21.15.97:0/231342675 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6b80014930 con 0x7f6b8815b370 2023-11-24T06:10:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.252+0000 7f6b857fa700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6b6806b110 0x7f6b6806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.252+0000 7f6b857fa700 1 -- 172.21.15.97:0/231342675 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f6b8008a1a0 con 0x7f6b8815b370 2023-11-24T06:10:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.252+0000 7f6b8ca26700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6b6806b110 0x7f6b6806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:52.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.253+0000 7f6b8ca26700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6b6806b110 0x7f6b6806d5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009a50 tx=0x7f6b78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:52.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.257+0000 7f6b857fa700 1 -- 172.21.15.97:0/231342675 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b80054e70 con 0x7f6b8815b370 2023-11-24T06:10:52.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:52.492+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6b881543c0 con 0x7f6b8815b370 2023-11-24T06:10:53.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.103+0000 7f6b857fa700 1 -- 172.21.15.97:0/231342675 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6b881543c0 con 0x7f6b8815b370 2023-11-24T06:10:53.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6b6806b110 msgr2=0x7f6b6806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:53.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6b6806b110 0x7f6b6806d5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009a50 tx=0x7f6b78008040 comp rx=0 tx=0).stop 2023-11-24T06:10:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 msgr2=0x7f6b88152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b88152a50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6b80007e60 tx=0x7f6b8000e3c0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 shutdown_connections 2023-11-24T06:10:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b8815b370 0x7f6b88152a50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6b6806b110 0x7f6b6806d5d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6b880b3c60 0x7f6b8814e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.109 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 --2- 172.21.15.97:0/231342675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b881587f0 0x7f6b88152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 >> 172.21.15.97:0/231342675 conn(0x7f6b880d0bf0 msgr2=0x7f6b880d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 shutdown_connections 2023-11-24T06:10:53.110 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.105+0000 7f6b8ec8a700 1 -- 172.21.15.97:0/231342675 wait complete. 2023-11-24T06:10:53.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:10:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.285+0000 7f3c5a6f7700 1 Processor -- start 2023-11-24T06:10:53.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.285+0000 7f3c5a6f7700 1 -- start start 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c5a6f7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c540b01e0 0x7f3c5406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c5a6f7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3c540b59c0 0x7f3c540b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c5a6f7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540b5e40 con 0x7f3c54073f90 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c5a6f7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c541470a0 con 0x7f3c540b59c0 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c5a6f7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540b05c0 con 0x7f3c540b01e0 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c537fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3c540b59c0 0x7f3c540b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c53fff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c540b01e0 0x7f3c5406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c58c94700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f3c54073f90 0x7f3c5406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:50490/0 (socket says 172.21.15.97:50490) 2023-11-24T06:10:53.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c58c94700 1 -- 172.21.15.97:0/3228936437 learned_addr learned my addr 172.21.15.97:0/3228936437 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c537fe700 1 -- 172.21.15.97:0/3228936437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c540b01e0 msgr2=0x7f3c5406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c537fe700 1 --2- 172.21.15.97:0/3228936437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c540b01e0 0x7f3c5406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.286+0000 7f3c537fe700 1 -- 172.21.15.97:0/3228936437 >> v1:172.21.15.183:6789/0 conn(0x7f3c54073f90 legacy=0x7f3c5406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c537fe700 1 -- 172.21.15.97:0/3228936437 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c540b4250 con 0x7f3c540b59c0 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c53fff700 1 --2- 172.21.15.97:0/3228936437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c540b01e0 0x7f3c5406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c537fe700 1 --2- 172.21.15.97:0/3228936437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b59c0 0x7f3c540b3b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009840 tx=0x7f3c44009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a21e7cf0f6c1e280 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c52ffd700 1 -- 172.21.15.97:0/3228936437 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44011830 con 0x7f3c540b59c0 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c52ffd700 1 -- 172.21.15.97:0/3228936437 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c44011990 con 0x7f3c540b59c0 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c52ffd700 1 -- 172.21.15.97:0/3228936437 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44011d20 con 0x7f3c540b59c0 2023-11-24T06:10:53.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/3228936437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b59c0 msgr2=0x7f3c540b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/3228936437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b59c0 0x7f3c540b3b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009840 tx=0x7f3c44009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/3228936437 shutdown_connections 2023-11-24T06:10:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/3228936437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c540b01e0 0x7f3c5406e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/3228936437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b59c0 0x7f3c540b3b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/3228936437 >> 172.21.15.97:0/3228936437 conn(0x7f3c540d0b40 msgr2=0x7f3c540d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/3228936437 shutdown_connections 2023-11-24T06:10:53.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.287+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/3228936437 wait complete. 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.289+0000 7f3c5a6f7700 1 Processor -- start 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.289+0000 7f3c5a6f7700 1 -- start start 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.289+0000 7f3c5a6f7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c54073f90 0x7f3c54160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c5a6f7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b01e0 0x7f3c541e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c5a6f7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c540b59c0 0x7f3c5415e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c5a6f7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541e7e30 con 0x7f3c540b59c0 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c5a6f7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541e8060 con 0x7f3c540b01e0 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c5a6f7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c541e8340 con 0x7f3c54073f90 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c540b59c0 0x7f3c5415e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c540b59c0 0x7f3c5415e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48722/0 (socket says 172.21.15.97:48722) 2023-11-24T06:10:53.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 -- 172.21.15.97:0/1630807121 learned_addr learned my addr 172.21.15.97:0/1630807121 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c53fff700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c54073f90 0x7f3c54160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c537fe700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b01e0 0x7f3c541e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 -- 172.21.15.97:0/1630807121 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c54073f90 msgr2=0x7f3c54160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c54073f90 0x7f3c54160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 -- 172.21.15.97:0/1630807121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b01e0 msgr2=0x7f3c541e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b01e0 0x7f3c541e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 -- 172.21.15.97:0/1630807121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c480094a0 con 0x7f3c540b59c0 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c58c94700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c540b59c0 0x7f3c5415e320 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00ca80 tx=0x7f3c4c00cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c50ff9700 1 -- 172.21.15.97:0/1630807121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00a9e0 con 0x7f3c540b59c0 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c50ff9700 1 -- 172.21.15.97:0/1630807121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c00ab40 con 0x7f3c540b59c0 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c440094a0 con 0x7f3c540b59c0 2023-11-24T06:10:53.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c50ff9700 1 -- 172.21.15.97:0/1630807121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c012400 con 0x7f3c540b59c0 2023-11-24T06:10:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.290+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c541e8900 con 0x7f3c540b59c0 2023-11-24T06:10:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.291+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54067f60 con 0x7f3c540b59c0 2023-11-24T06:10:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.292+0000 7f3c50ff9700 1 -- 172.21.15.97:0/1630807121 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3c4c01b3b0 con 0x7f3c540b59c0 2023-11-24T06:10:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.292+0000 7f3c50ff9700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3c4006b110 0x7f3c4006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.292+0000 7f3c50ff9700 1 -- 172.21.15.97:0/1630807121 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f3c4c089540 con 0x7f3c540b59c0 2023-11-24T06:10:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.292+0000 7f3c53fff700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3c4006b110 0x7f3c4006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:53.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.292+0000 7f3c53fff700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3c4006b110 0x7f3c4006d5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009d70 tx=0x7f3c48009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:53.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.297+0000 7f3c50ff9700 1 -- 172.21.15.97:0/1630807121 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c4c054210 con 0x7f3c540b59c0 2023-11-24T06:10:53.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:53.488+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3c540d0190 con 0x7f3c540b59c0 2023-11-24T06:10:54.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.110+0000 7f3c50ff9700 1 -- 172.21.15.97:0/1630807121 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3c4c057830 con 0x7f3c540b59c0 2023-11-24T06:10:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3c4006b110 msgr2=0x7f3c4006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3c4006b110 0x7f3c4006d5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009d70 tx=0x7f3c48009210 comp rx=0 tx=0).stop 2023-11-24T06:10:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c540b59c0 msgr2=0x7f3c5415e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c540b59c0 0x7f3c5415e320 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00ca80 tx=0x7f3c4c00cd90 comp rx=0 tx=0).stop 2023-11-24T06:10:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 shutdown_connections 2023-11-24T06:10:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c540b59c0 0x7f3c5415e320 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3c4006b110 0x7f3c4006d5d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3c54073f90 0x7f3c54160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 --2- 172.21.15.97:0/1630807121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c540b01e0 0x7f3c541e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 >> 172.21.15.97:0/1630807121 conn(0x7f3c540d0b40 msgr2=0x7f3c540b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 shutdown_connections 2023-11-24T06:10:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.113+0000 7f3c5a6f7700 1 -- 172.21.15.97:0/1630807121 wait complete. 2023-11-24T06:10:54.117 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:10:54.132 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:10:54.251 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:10:54.253 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:10:54.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.260+0000 7fdf1ff6d700 1 Processor -- start 2023-11-24T06:10:54.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.261+0000 7fdf1ff6d700 1 -- start start 2023-11-24T06:10:54.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1ff6d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf18145620 0x7fdf1806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1ff6d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdf180cd810 0x7fdf1806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1ff6d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf180cf530 con 0x7fdf18073f90 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1ff6d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1806b8f0 con 0x7fdf18145620 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1ff6d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1806bbd0 con 0x7fdf180cd810 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1dd09700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf18145620 0x7fdf1806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1dd09700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf18145620 0x7fdf1806dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48728/0 (socket says 172.21.15.97:48728) 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1dd09700 1 -- 172.21.15.97:0/261323126 learned_addr learned my addr 172.21.15.97:0/261323126 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1d508700 1 --2- 172.21.15.97:0/261323126 >> v2:172.21.15.183:3300/0 conn(0x7fdf180cd810 0x7fdf1806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1dd09700 1 -- 172.21.15.97:0/261323126 >> v2:172.21.15.183:3300/0 conn(0x7fdf180cd810 msgr2=0x7fdf1806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1dd09700 1 --2- 172.21.15.97:0/261323126 >> v2:172.21.15.183:3300/0 conn(0x7fdf180cd810 0x7fdf1806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1dd09700 1 -- 172.21.15.97:0/261323126 >> v1:172.21.15.183:6789/0 conn(0x7fdf18073f90 legacy=0x7fdf18074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1dd09700 1 -- 172.21.15.97:0/261323126 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180cf1c0 con 0x7fdf18145620 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1d508700 1 --2- 172.21.15.97:0/261323126 >> v2:172.21.15.183:3300/0 conn(0x7fdf180cd810 0x7fdf1806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:54.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.262+0000 7fdf1dd09700 1 --2- 172.21.15.97:0/261323126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18145620 0x7fdf1806dc00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdf180cce30 tx=0x7fdf14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef880571d844e67 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.263+0000 7fdf1cd07700 1 -- 172.21.15.97:0/261323126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400b870 con 0x7fdf18145620 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.263+0000 7fdf1cd07700 1 -- 172.21.15.97:0/261323126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1400b9d0 con 0x7fdf18145620 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.263+0000 7fdf1cd07700 1 -- 172.21.15.97:0/261323126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400bd60 con 0x7fdf18145620 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.263+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/261323126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18145620 msgr2=0x7fdf1806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.263+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/261323126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18145620 0x7fdf1806dc00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdf180cce30 tx=0x7fdf14009bd0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.264+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/261323126 shutdown_connections 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.264+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/261323126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18145620 0x7fdf1806dc00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.264+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/261323126 >> v2:172.21.15.183:3300/0 conn(0x7fdf180cd810 0x7fdf1806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.264+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/261323126 >> 172.21.15.97:0/261323126 conn(0x7fdf180d0b40 msgr2=0x7fdf180d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.264+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/261323126 shutdown_connections 2023-11-24T06:10:54.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.264+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/261323126 wait complete. 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.265+0000 7fdf1ff6d700 1 Processor -- start 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.265+0000 7fdf1ff6d700 1 -- start start 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1ff6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18073f90 0x7fdf1815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1ff6d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf180cd810 0x7fdf1815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1ff6d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf18145620 0x7fdf1815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1ff6d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1815bb10 con 0x7fdf18073f90 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1ff6d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1813bf60 con 0x7fdf180cd810 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1ff6d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1813c240 con 0x7fdf18145620 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1dd09700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18073f90 0x7fdf1815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1dd09700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18073f90 0x7fdf1815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48734/0 (socket says 172.21.15.97:48734) 2023-11-24T06:10:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1dd09700 1 -- 172.21.15.97:0/1815477824 learned_addr learned my addr 172.21.15.97:0/1815477824 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1d508700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf180cd810 0x7fdf1815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.266+0000 7fdf1e50a700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf18145620 0x7fdf1815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1dd09700 1 -- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf18145620 msgr2=0x7fdf1815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1dd09700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf18145620 0x7fdf1815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1dd09700 1 -- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf180cd810 msgr2=0x7fdf1815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1dd09700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf180cd810 0x7fdf1815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1dd09700 1 -- 172.21.15.97:0/1815477824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf14009550 con 0x7fdf18073f90 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1e50a700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf18145620 0x7fdf1815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1dd09700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18073f90 0x7fdf1815c3c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdf14000c00 tx=0x7fdf14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf0e7fc700 1 -- 172.21.15.97:0/1815477824 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14004520 con 0x7fdf18073f90 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf0e7fc700 1 -- 172.21.15.97:0/1815477824 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1401e070 con 0x7fdf18073f90 2023-11-24T06:10:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1813c520 con 0x7fdf18073f90 2023-11-24T06:10:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf0e7fc700 1 -- 172.21.15.97:0/1815477824 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140038a0 con 0x7fdf18073f90 2023-11-24T06:10:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.267+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1813cae0 con 0x7fdf18073f90 2023-11-24T06:10:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.268+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1813ce50 con 0x7fdf18073f90 2023-11-24T06:10:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.269+0000 7fdf0e7fc700 1 -- 172.21.15.97:0/1815477824 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fdf14003a00 con 0x7fdf18073f90 2023-11-24T06:10:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.269+0000 7fdf0e7fc700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdefc06b110 0x7fdefc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.269+0000 7fdf0e7fc700 1 -- 172.21.15.97:0/1815477824 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fdf14020070 con 0x7fdf18073f90 2023-11-24T06:10:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.269+0000 7fdf1d508700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdefc06b110 0x7fdefc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.270+0000 7fdf1d508700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdefc06b110 0x7fdefc06d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdf1806e240 tx=0x7fdf08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:54.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.274+0000 7fdf0e7fc700 1 -- 172.21.15.97:0/1815477824 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf14055c80 con 0x7fdf18073f90 2023-11-24T06:10:54.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:54.468+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdf18067f60 con 0x7fdf18073f90 2023-11-24T06:10:55.118 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.115+0000 7fdf0e7fc700 1 -- 172.21.15.97:0/1815477824 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf14015020 con 0x7fdf18073f90 2023-11-24T06:10:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdefc06b110 msgr2=0x7fdefc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdefc06b110 0x7fdefc06d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdf1806e240 tx=0x7fdf08008040 comp rx=0 tx=0).stop 2023-11-24T06:10:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18073f90 msgr2=0x7fdf1815c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18073f90 0x7fdf1815c3c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdf14000c00 tx=0x7fdf14004750 comp rx=0 tx=0).stop 2023-11-24T06:10:55.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 shutdown_connections 2023-11-24T06:10:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdefc06b110 0x7fdefc06d5d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdf18145620 0x7fdf1815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf18073f90 0x7fdf1815c3c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 --2- 172.21.15.97:0/1815477824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf180cd810 0x7fdf1815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 >> 172.21.15.97:0/1815477824 conn(0x7fdf180d0b40 msgr2=0x7fdf1806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 shutdown_connections 2023-11-24T06:10:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.119+0000 7fdf1ff6d700 1 -- 172.21.15.97:0/1815477824 wait complete. 2023-11-24T06:10:55.123 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-11-24T06:10:55.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:10:55.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.262+0000 7f3e0051e700 1 Processor -- start 2023-11-24T06:10:55.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.262+0000 7f3e0051e700 1 -- start start 2023-11-24T06:10:55.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3e0051e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3df80aedc0 0x7f3df80ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3e0051e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df815e290 0x7f3df80af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3e0051e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df806e400 con 0x7f3df80ad870 2023-11-24T06:10:55.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3e0051e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df806fde0 con 0x7f3df80aedc0 2023-11-24T06:10:55.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3e0051e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80af6e0 con 0x7f3df815e290 2023-11-24T06:10:55.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3dfe2ba700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3df80aedc0 0x7f3df80ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3dfdab9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df815e290 0x7f3df80af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3dfe2ba700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3df80aedc0 0x7f3df80ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:42902/0 (socket says 172.21.15.97:42902) 2023-11-24T06:10:55.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3dfdab9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df815e290 0x7f3df80af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:35780/0 (socket says 172.21.15.97:35780) 2023-11-24T06:10:55.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3dfe2ba700 1 -- 172.21.15.97:0/3644119976 learned_addr learned my addr 172.21.15.97:0/3644119976 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:55.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.263+0000 7f3dfeabb700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f3df80ad870 0x7f3df806f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:50520/0 (socket says 172.21.15.97:50520) 2023-11-24T06:10:55.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfdab9700 1 -- 172.21.15.97:0/3644119976 >> v2:172.21.15.183:3300/0 conn(0x7f3df80aedc0 msgr2=0x7f3df80ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:55.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfdab9700 1 --2- 172.21.15.97:0/3644119976 >> v2:172.21.15.183:3300/0 conn(0x7f3df80aedc0 0x7f3df80ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfdab9700 1 -- 172.21.15.97:0/3644119976 >> v1:172.21.15.183:6789/0 conn(0x7f3df80ad870 legacy=0x7f3df806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfdab9700 1 -- 172.21.15.97:0/3644119976 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df806e630 con 0x7f3df815e290 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfe2ba700 1 --2- 172.21.15.97:0/3644119976 >> v2:172.21.15.183:3300/0 conn(0x7f3df80aedc0 0x7f3df80ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfdab9700 1 --2- 172.21.15.97:0/3644119976 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df815e290 0x7f3df80af1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009840 tx=0x7f3df4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70676f596c1c293 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfd2b8700 1 -- 172.21.15.97:0/3644119976 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4011830 con 0x7f3df815e290 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfd2b8700 1 -- 172.21.15.97:0/3644119976 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df4011990 con 0x7f3df815e290 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.264+0000 7f3dfd2b8700 1 -- 172.21.15.97:0/3644119976 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4011d20 con 0x7f3df815e290 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.265+0000 7f3e0051e700 1 -- 172.21.15.97:0/3644119976 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df815e290 msgr2=0x7f3df80af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.265+0000 7f3e0051e700 1 --2- 172.21.15.97:0/3644119976 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df815e290 0x7f3df80af1a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009840 tx=0x7f3df4009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.265+0000 7f3e0051e700 1 -- 172.21.15.97:0/3644119976 shutdown_connections 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.265+0000 7f3e0051e700 1 --2- 172.21.15.97:0/3644119976 >> v2:172.21.15.183:3300/0 conn(0x7f3df80aedc0 0x7f3df80ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.265+0000 7f3e0051e700 1 --2- 172.21.15.97:0/3644119976 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df815e290 0x7f3df80af1a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.265+0000 7f3e0051e700 1 -- 172.21.15.97:0/3644119976 >> 172.21.15.97:0/3644119976 conn(0x7f3df80b2050 msgr2=0x7f3df80b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:55.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.265+0000 7f3e0051e700 1 -- 172.21.15.97:0/3644119976 shutdown_connections 2023-11-24T06:10:55.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.265+0000 7f3e0051e700 1 -- 172.21.15.97:0/3644119976 wait complete. 2023-11-24T06:10:55.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.266+0000 7f3e0051e700 1 Processor -- start 2023-11-24T06:10:55.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.266+0000 7f3e0051e700 1 -- start start 2023-11-24T06:10:55.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3e0051e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df80ad870 0x7f3df8160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3e0051e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df80aedc0 0x7f3df814b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3e0051e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3df815e290 0x7f3df8155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3e0051e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df814b970 con 0x7f3df80aedc0 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3e0051e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80cd710 con 0x7f3df815e290 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3e0051e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df8156af0 con 0x7f3df80ad870 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfdab9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df80aedc0 0x7f3df814b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfdab9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df80aedc0 0x7f3df814b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48750/0 (socket says 172.21.15.97:48750) 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfdab9700 1 -- 172.21.15.97:0/9367854 learned_addr learned my addr 172.21.15.97:0/9367854 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfe2ba700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df80ad870 0x7f3df8160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfeabb700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3df815e290 0x7f3df8155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfdab9700 1 -- 172.21.15.97:0/9367854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df80ad870 msgr2=0x7f3df8160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfdab9700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df80ad870 0x7f3df8160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfdab9700 1 -- 172.21.15.97:0/9367854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3df815e290 msgr2=0x7f3df8155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfdab9700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3df815e290 0x7f3df8155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.267+0000 7f3dfdab9700 1 -- 172.21.15.97:0/9367854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8009550 con 0x7f3df80aedc0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.268+0000 7f3dfdab9700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df80aedc0 0x7f3df814b430 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3df40090b0 tx=0x7f3df4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.268+0000 7f3deeffd700 1 -- 172.21.15.97:0/9367854 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df40036a0 con 0x7f3df80aedc0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.268+0000 7f3deeffd700 1 -- 172.21.15.97:0/9367854 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df401f070 con 0x7f3df80aedc0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.268+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df40094a0 con 0x7f3df80aedc0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.268+0000 7f3deeffd700 1 -- 172.21.15.97:0/9367854 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df4003a00 con 0x7f3df80aedc0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.268+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df8155f70 con 0x7f3df80aedc0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.269+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df8068d90 con 0x7f3df80aedc0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.269+0000 7f3deeffd700 1 -- 172.21.15.97:0/9367854 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3df4003b60 con 0x7f3df80aedc0 2023-11-24T06:10:55.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.270+0000 7f3deeffd700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ddc06b110 0x7f3ddc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.270+0000 7f3dfe2ba700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ddc06b110 0x7f3ddc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.270+0000 7f3deeffd700 1 -- 172.21.15.97:0/9367854 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3df4021070 con 0x7f3df80aedc0 2023-11-24T06:10:55.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.270+0000 7f3dfe2ba700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ddc06b110 0x7f3ddc06d5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009cf0 tx=0x7f3de8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:55.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.274+0000 7f3deeffd700 1 -- 172.21.15.97:0/9367854 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3df4056110 con 0x7f3df80aedc0 2023-11-24T06:10:55.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.468+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3df8068fb0 con 0x7f3df80aedc0 2023-11-24T06:10:55.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.471+0000 7f3deeffd700 1 -- 172.21.15.97:0/9367854 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3df4059730 con 0x7f3df80aedc0 2023-11-24T06:10:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ddc06b110 msgr2=0x7f3ddc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ddc06b110 0x7f3ddc06d5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009cf0 tx=0x7f3de8008040 comp rx=0 tx=0).stop 2023-11-24T06:10:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df80aedc0 msgr2=0x7f3df814b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df80aedc0 0x7f3df814b430 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3df40090b0 tx=0x7f3df4004750 comp rx=0 tx=0).stop 2023-11-24T06:10:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 shutdown_connections 2023-11-24T06:10:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3df815e290 0x7f3df8155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ddc06b110 0x7f3ddc06d5d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3df80ad870 0x7f3df8160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 --2- 172.21.15.97:0/9367854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df80aedc0 0x7f3df814b430 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 >> 172.21.15.97:0/9367854 conn(0x7f3df80b2050 msgr2=0x7f3df806f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 shutdown_connections 2023-11-24T06:10:55.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.473+0000 7f3e0051e700 1 -- 172.21.15.97:0/9367854 wait complete. 2023-11-24T06:10:55.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T06:10:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.604+0000 7f5fe2b90700 1 Processor -- start 2023-11-24T06:10:55.607 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.604+0000 7f5fe2b90700 1 -- start start 2023-11-24T06:10:55.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe2b90700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fdc06c710 0x7f5fdc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe2b90700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe2b90700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fdc0ae430 con 0x7f5fdc15e080 2023-11-24T06:10:55.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe2b90700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc0af0b0 con 0x7f5fdc06c710 2023-11-24T06:10:55.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe2b90700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0b05a0 con 0x7f5fdc0af4d0 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fdbfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fdc06c710 0x7f5fdc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe092c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5fdc15e080 0x7f5fdc06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41208/0 (socket says 172.21.15.97:41208) 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe092c700 1 -- 172.21.15.97:0/1319819046 learned_addr learned my addr 172.21.15.97:0/1319819046 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe112d700 1 --2- 172.21.15.97:0/1319819046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fdbfff700 1 -- 172.21.15.97:0/1319819046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 msgr2=0x7f5fdc0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fdbfff700 1 --2- 172.21.15.97:0/1319819046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fdbfff700 1 -- 172.21.15.97:0/1319819046 >> v1:172.21.15.97:6789/0 conn(0x7f5fdc15e080 legacy=0x7f5fdc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fdbfff700 1 -- 172.21.15.97:0/1319819046 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0ae710 con 0x7f5fdc06c710 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fdb7fe700 1 -- 172.21.15.97:0/1319819046 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817594278 0 0) 0x7f5fdc0ae430 con 0x7f5fdc15e080 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.605+0000 7f5fe112d700 1 --2- 172.21.15.97:0/1319819046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:55.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fdbfff700 1 --2- 172.21.15.97:0/1319819046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc0b5050 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009840 tx=0x7f5fc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc45115ce3dcfde7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fdb7fe700 1 -- 172.21.15.97:0/1319819046 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8011830 con 0x7f5fdc06c710 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fdb7fe700 1 -- 172.21.15.97:0/1319819046 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc8011990 con 0x7f5fdc06c710 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fdb7fe700 1 -- 172.21.15.97:0/1319819046 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8011d20 con 0x7f5fdc06c710 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fe2b90700 1 -- 172.21.15.97:0/1319819046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 msgr2=0x7f5fdc0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/1319819046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc0b5050 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009840 tx=0x7f5fc8009b50 comp rx=0 tx=0).stop 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fe2b90700 1 -- 172.21.15.97:0/1319819046 shutdown_connections 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/1319819046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/1319819046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc0b5050 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fe2b90700 1 -- 172.21.15.97:0/1319819046 >> 172.21.15.97:0/1319819046 conn(0x7f5fdc071b90 msgr2=0x7f5fdc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fe2b90700 1 -- 172.21.15.97:0/1319819046 shutdown_connections 2023-11-24T06:10:55.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.606+0000 7f5fe2b90700 1 -- 172.21.15.97:0/1319819046 wait complete. 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.607+0000 7f5fe2b90700 1 Processor -- start 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe2b90700 1 -- start start 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe2b90700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe2b90700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe2b90700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fdc15e080 0x7f5fdc14a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe2b90700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0d0620 con 0x7f5fdc06c710 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe2b90700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc1586b0 con 0x7f5fdc15e080 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe2b90700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc158990 con 0x7f5fdc0af4d0 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe092c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe092c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48766/0 (socket says 172.21.15.97:48766) 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.608+0000 7f5fe092c700 1 -- 172.21.15.97:0/2106167643 learned_addr learned my addr 172.21.15.97:0/2106167643 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fdbfff700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe112d700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fdc15e080 0x7f5fdc14a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe092c700 1 -- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 msgr2=0x7f5fdc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe092c700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe092c700 1 -- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fdc15e080 msgr2=0x7f5fdc14a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe092c700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fdc15e080 0x7f5fdc14a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe092c700 1 -- 172.21.15.97:0/2106167643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc80094a0 con 0x7f5fdc06c710 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe112d700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fdc15e080 0x7f5fdc14a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe092c700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc158020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c550 tx=0x7f5fd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fd97fa700 1 -- 172.21.15.97:0/2106167643 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0004c80 con 0x7f5fdc06c710 2023-11-24T06:10:55.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fd97fa700 1 -- 172.21.15.97:0/2106167643 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0004e00 con 0x7f5fdc06c710 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc158c70 con 0x7f5fdc06c710 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fd97fa700 1 -- 172.21.15.97:0/2106167643 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0007310 con 0x7f5fdc06c710 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.609+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc0cd900 con 0x7f5fdc06c710 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.610+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc0cdc70 con 0x7f5fdc06c710 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.611+0000 7f5fd97fa700 1 -- 172.21.15.97:0/2106167643 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5fd0007470 con 0x7f5fdc06c710 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.611+0000 7f5fd97fa700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fc006b110 0x7f5fc006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.611+0000 7f5fd97fa700 1 -- 172.21.15.97:0/2106167643 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5fd0089400 con 0x7f5fdc06c710 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.611+0000 7f5fdbfff700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fc006b110 0x7f5fc006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:55.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.611+0000 7f5fdbfff700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fc006b110 0x7f5fc006d5d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80090b0 tx=0x7f5fc8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:55.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.615+0000 7f5fd97fa700 1 -- 172.21.15.97:0/2106167643 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5fd0054450 con 0x7f5fdc06c710 2023-11-24T06:10:55.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:55.884+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f5fdc0753f0 con 0x7f5fdc06c710 2023-11-24T06:10:56.119 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.116+0000 7f5fd97fa700 1 -- 172.21.15.97:0/2106167643 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v134) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5fd0057a70 con 0x7f5fdc06c710 2023-11-24T06:10:56.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.118+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fc006b110 msgr2=0x7f5fc006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.118+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fc006b110 0x7f5fc006d5d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80090b0 tx=0x7f5fc8008030 comp rx=0 tx=0).stop 2023-11-24T06:10:56.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.118+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 msgr2=0x7f5fdc158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.118+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc158020 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c550 tx=0x7f5fd000c910 comp rx=0 tx=0).stop 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.119+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 shutdown_connections 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.119+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fdc15e080 0x7f5fdc14a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.119+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fdc06c710 0x7f5fdc158020 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.119+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fc006b110 0x7f5fc006d5d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.119+0000 7f5fe2b90700 1 --2- 172.21.15.97:0/2106167643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fdc0af4d0 0x7f5fdc0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.119+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 >> 172.21.15.97:0/2106167643 conn(0x7f5fdc071b90 msgr2=0x7f5fdc0b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.119+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 shutdown_connections 2023-11-24T06:10:56.122 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.119+0000 7f5fe2b90700 1 -- 172.21.15.97:0/2106167643 wait complete. 2023-11-24T06:10:56.134 INFO:tasks.cephfs_test_runner:Validate cleaning of stale subvolume snapshot metadata. ... ok 2023-11-24T06:10:56.135 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:10:56.135 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:10:56.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.252+0000 7fd103db0700 1 Processor -- start 2023-11-24T06:10:56.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.252+0000 7fd103db0700 1 -- start start 2023-11-24T06:10:56.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd103db0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0fc0cff10 0x7fd0fc165bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd103db0700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc0b4290 con 0x7fd0fc15f180 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd103db0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc0d0380 con 0x7fd0fc13e000 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd103db0700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc13e570 con 0x7fd0fc0cff10 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd10234d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0fc0cff10 0x7fd0fc165bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd10134b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd0fc15f180 0x7fd0fc0cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41222/0 (socket says 172.21.15.97:41222) 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd10134b700 1 -- 172.21.15.97:0/700840437 learned_addr learned my addr 172.21.15.97:0/700840437 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd10234d700 1 -- 172.21.15.97:0/700840437 >> v1:172.21.15.183:6789/0 conn(0x7fd0fc13e000 legacy=0x7fd0fc15ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd10234d700 1 -- 172.21.15.97:0/700840437 >> v1:172.21.15.97:6789/0 conn(0x7fd0fc15f180 legacy=0x7fd0fc0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd10234d700 1 -- 172.21.15.97:0/700840437 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc0d05d0 con 0x7fd0fc0cff10 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.253+0000 7fd100b4a700 1 -- 172.21.15.97:0/700840437 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464308698 0 0) 0x7fd0fc0b4290 con 0x7fd0fc15f180 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd10234d700 1 --2- 172.21.15.97:0/700840437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc0cff10 0x7fd0fc165bb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009d90 tx=0x7fd0f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fcf0190bfbe002 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd100b4a700 1 -- 172.21.15.97:0/700840437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400e070 con 0x7fd0fc0cff10 2023-11-24T06:10:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd100b4a700 1 -- 172.21.15.97:0/700840437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f400bbd0 con 0x7fd0fc0cff10 2023-11-24T06:10:56.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd100b4a700 1 -- 172.21.15.97:0/700840437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4012430 con 0x7fd0fc0cff10 2023-11-24T06:10:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd103db0700 1 -- 172.21.15.97:0/700840437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc0cff10 msgr2=0x7fd0fc165bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd103db0700 1 --2- 172.21.15.97:0/700840437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc0cff10 0x7fd0fc165bb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009d90 tx=0x7fd0f400b330 comp rx=0 tx=0).stop 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd103db0700 1 -- 172.21.15.97:0/700840437 shutdown_connections 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd103db0700 1 --2- 172.21.15.97:0/700840437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc0cff10 0x7fd0fc165bb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd103db0700 1 -- 172.21.15.97:0/700840437 >> 172.21.15.97:0/700840437 conn(0x7fd0fc072990 msgr2=0x7fd0fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd103db0700 1 -- 172.21.15.97:0/700840437 shutdown_connections 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.254+0000 7fd103db0700 1 -- 172.21.15.97:0/700840437 wait complete. 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd103db0700 1 Processor -- start 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd103db0700 1 -- start start 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd103db0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc0cff10 0x7fd0fc13d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd103db0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0fc13e000 0x7fd0fc13f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd103db0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc15f180 0x7fd0fc1437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd103db0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc13da80 con 0x7fd0fc15f180 2023-11-24T06:10:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd103db0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc153250 con 0x7fd0fc0cff10 2023-11-24T06:10:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd103db0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc06b170 con 0x7fd0fc13e000 2023-11-24T06:10:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd10234d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc15f180 0x7fd0fc1437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd10234d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc15f180 0x7fd0fc1437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48780/0 (socket says 172.21.15.97:48780) 2023-11-24T06:10:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.256+0000 7fd10234d700 1 -- 172.21.15.97:0/3068037478 learned_addr learned my addr 172.21.15.97:0/3068037478 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd10134b700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0fc13e000 0x7fd0fc13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd101b4c700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc0cff10 0x7fd0fc13d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd10234d700 1 -- 172.21.15.97:0/3068037478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0fc13e000 msgr2=0x7fd0fc13f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd10234d700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0fc13e000 0x7fd0fc13f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd10234d700 1 -- 172.21.15.97:0/3068037478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc0cff10 msgr2=0x7fd0fc13d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd10234d700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc0cff10 0x7fd0fc13d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd10234d700 1 -- 172.21.15.97:0/3068037478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009a70 con 0x7fd0fc15f180 2023-11-24T06:10:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd10234d700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc15f180 0x7fd0fc1437e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4014040 tx=0x7fd0f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd0f27fc700 1 -- 172.21.15.97:0/3068037478 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f401b810 con 0x7fd0fc15f180 2023-11-24T06:10:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd0f27fc700 1 -- 172.21.15.97:0/3068037478 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f4020070 con 0x7fd0fc15f180 2023-11-24T06:10:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0fc06b450 con 0x7fd0fc15f180 2023-11-24T06:10:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd0f27fc700 1 -- 172.21.15.97:0/3068037478 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f401bbb0 con 0x7fd0fc15f180 2023-11-24T06:10:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.257+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0fc06bc50 con 0x7fd0fc15f180 2023-11-24T06:10:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.258+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0fc067f60 con 0x7fd0fc15f180 2023-11-24T06:10:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.258+0000 7fd0f27fc700 1 -- 172.21.15.97:0/3068037478 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd0f401bd10 con 0x7fd0fc15f180 2023-11-24T06:10:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.259+0000 7fd0f27fc700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0e006b140 0x7fd0e006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.259+0000 7fd0f27fc700 1 -- 172.21.15.97:0/3068037478 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd0f4022070 con 0x7fd0fc15f180 2023-11-24T06:10:56.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.259+0000 7fd101b4c700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0e006b140 0x7fd0e006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.259+0000 7fd101b4c700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0e006b140 0x7fd0e006d600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec000f60 tx=0x7fd0ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.263+0000 7fd0f27fc700 1 -- 172.21.15.97:0/3068037478 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0f4059ea0 con 0x7fd0fc15f180 2023-11-24T06:10:56.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.454+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0fc15eca0 con 0x7fd0fc15f180 2023-11-24T06:10:56.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.455+0000 7fd0f27fc700 1 -- 172.21.15.97:0/3068037478 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+12994 (secure 0 0 0) 0x7fd0f4059a30 con 0x7fd0fc15f180 2023-11-24T06:10:56.458 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:56.458 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":52,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:10:55.111834+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/19331625":"2023-11-25T06:10:52.101011+0000","172.21.15.97:6834/19331625":"2023-11-25T06:10:52.101011+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:10:56.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0e006b140 msgr2=0x7fd0e006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0e006b140 0x7fd0e006d600 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec000f60 tx=0x7fd0ec009040 comp rx=0 tx=0).stop 2023-11-24T06:10:56.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc15f180 msgr2=0x7fd0fc1437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc15f180 0x7fd0fc1437e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4014040 tx=0x7fd0f4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 shutdown_connections 2023-11-24T06:10:56.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0fc15f180 0x7fd0fc1437e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0fc0cff10 0x7fd0fc13d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0e006b140 0x7fd0e006d600 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 --2- 172.21.15.97:0/3068037478 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0fc13e000 0x7fd0fc13f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 >> 172.21.15.97:0/3068037478 conn(0x7fd0fc072990 msgr2=0x7fd0fc15df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 shutdown_connections 2023-11-24T06:10:56.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.457+0000 7fd103db0700 1 -- 172.21.15.97:0/3068037478 wait complete. 2023-11-24T06:10:56.471 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:10:56.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:10:56.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.592+0000 7fe9ce856700 1 Processor -- start 2023-11-24T06:10:56.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.592+0000 7fe9ce856700 1 -- start start 2023-11-24T06:10:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.592+0000 7fe9ce856700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c815e320 0x7fe9c80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.592+0000 7fe9ce856700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe9c80b93c0 0x7fe9c806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.592+0000 7fe9ce856700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c80b6fb0 con 0x7fe9c80b3250 2023-11-24T06:10:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9ce856700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c80d4da0 con 0x7fe9c80b93c0 2023-11-24T06:10:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9ce856700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80d2940 con 0x7fe9c815e320 2023-11-24T06:10:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9ccdf3700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe9c80b3250 0x7fe9c80ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41240/0 (socket says 172.21.15.97:41240) 2023-11-24T06:10:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9ccdf3700 1 -- 172.21.15.97:0/1199127316 learned_addr learned my addr 172.21.15.97:0/1199127316 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:56.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81571899 0 0) 0x7fe9c80b6fb0 con 0x7fe9c80b3250 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b00035b0 con 0x7fe9c80b3250 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3070895547 0 0) 0x7fe9b00035b0 con 0x7fe9c80b3250 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c815e320 msgr2=0x7fe9c80d2400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c815e320 0x7fe9c80d2400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 >> v2:172.21.15.183:3300/0 conn(0x7fe9c80b93c0 msgr2=0x7fe9c806aff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 --2- 172.21.15.97:0/1199127316 >> v2:172.21.15.183:3300/0 conn(0x7fe9c80b93c0 0x7fe9c806aff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80d2c10 con 0x7fe9c80b3250 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fe9c00019f0 con 0x7fe9c80b3250 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 >> v1:172.21.15.97:6789/0 conn(0x7fe9c80b3250 legacy=0x7fe9c80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b0003a00 0x7fe9b0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c815e320 0x7fe9b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9b0004e00 0x7fe9b0005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80d4da0 con 0x7fe9b0003a00 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80d2940 con 0x7fe9b0004e00 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b000bd30 con 0x7fe9c815e320 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c7fff700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b0003a00 0x7fe9b0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9ccdf3700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9b0004e00 0x7fe9b0005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.593+0000 7fe9c77fe700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c815e320 0x7fe9b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c7fff700 1 -- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c815e320 msgr2=0x7fe9b0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c7fff700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c815e320 0x7fe9b0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c7fff700 1 -- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9b0004e00 msgr2=0x7fe9b0005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c7fff700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9b0004e00 0x7fe9b0005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c7fff700 1 -- 172.21.15.97:0/1199127316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b00035b0 con 0x7fe9b0003a00 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c7fff700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b0003a00 0x7fe9b0003e20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009830 tx=0x7fe9b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd157460a2f6dca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80092d0 con 0x7fe9b0003a00 2023-11-24T06:10:56.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8011620 con 0x7fe9b0003a00 2023-11-24T06:10:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9c6ffd700 1 -- 172.21.15.97:0/1199127316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80119d0 con 0x7fe9b0003a00 2023-11-24T06:10:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9ce856700 1 -- 172.21.15.97:0/1199127316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b0003a00 msgr2=0x7fe9b0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9ce856700 1 --2- 172.21.15.97:0/1199127316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b0003a00 0x7fe9b0003e20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009830 tx=0x7fe9b0005880 comp rx=0 tx=0).stop 2023-11-24T06:10:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9ccdf3700 1 -- 172.21.15.97:0/1199127316 reap_dead start 2023-11-24T06:10:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9ce856700 1 -- 172.21.15.97:0/1199127316 shutdown_connections 2023-11-24T06:10:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.594+0000 7fe9ce856700 1 -- 172.21.15.97:0/1199127316 >> 172.21.15.97:0/1199127316 conn(0x7fe9c80d3e50 msgr2=0x7fe9c80d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.595+0000 7fe9ce856700 1 -- 172.21.15.97:0/1199127316 shutdown_connections 2023-11-24T06:10:56.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.595+0000 7fe9ce856700 1 -- 172.21.15.97:0/1199127316 wait complete. 2023-11-24T06:10:56.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.596+0000 7fe9ce856700 1 Processor -- start 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.596+0000 7fe9ce856700 1 -- start start 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ce856700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9b0003a00 0x7fe9c80cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ce856700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c80cc320 0x7fe9c8150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ce856700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c80c9b50 0x7fe9c80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ce856700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80ca520 con 0x7fe9c80c9b50 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ce856700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80ccae0 con 0x7fe9c80cc320 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ce856700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80ccdc0 con 0x7fe9b0003a00 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c80c9b50 0x7fe9c80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c80c9b50 0x7fe9c80c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48800/0 (socket says 172.21.15.97:48800) 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 -- 172.21.15.97:0/2488069247 learned_addr learned my addr 172.21.15.97:0/2488069247 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9c77fe700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c80cc320 0x7fe9c8150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 -- 172.21.15.97:0/2488069247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9b0003a00 msgr2=0x7fe9c80cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9b0003a00 0x7fe9c80cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 -- 172.21.15.97:0/2488069247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c80cc320 msgr2=0x7fe9c8150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c80cc320 0x7fe9c8150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 -- 172.21.15.97:0/2488069247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80cd0a0 con 0x7fe9c80c9b50 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ccdf3700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c80c9b50 0x7fe9c80c9fe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8068c60 tx=0x7fe9c000e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9c4ff9700 1 -- 172.21.15.97:0/2488069247 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000ee00 con 0x7fe9c80c9b50 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9c4ff9700 1 -- 172.21.15.97:0/2488069247 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c0013420 con 0x7fe9c80c9b50 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c80cd380 con 0x7fe9c80c9b50 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9c4ff9700 1 -- 172.21.15.97:0/2488069247 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00137d0 con 0x7fe9c80c9b50 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.597+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c80cc7a0 con 0x7fe9c80c9b50 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.598+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c8068480 con 0x7fe9c80c9b50 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.599+0000 7fe9c4ff9700 1 -- 172.21.15.97:0/2488069247 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe9c0013930 con 0x7fe9c80c9b50 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.599+0000 7fe9c4ff9700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9b406b110 0x7fe9b406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.599+0000 7fe9c7fff700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9b406b110 0x7fe9b406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.599+0000 7fe9c4ff9700 1 -- 172.21.15.97:0/2488069247 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe9c0089d60 con 0x7fe9c80c9b50 2023-11-24T06:10:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.600+0000 7fe9c7fff700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9b406b110 0x7fe9b406d5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009fd0 tx=0x7fe9b8010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.606 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.604+0000 7fe9c4ff9700 1 -- 172.21.15.97:0/2488069247 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9c0054f10 con 0x7fe9c80c9b50 2023-11-24T06:10:56.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.798+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9c80cd840 con 0x7fe9c80c9b50 2023-11-24T06:10:56.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.798+0000 7fe9c4ff9700 1 -- 172.21.15.97:0/2488069247 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9c0058530 con 0x7fe9c80c9b50 2023-11-24T06:10:56.801 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:10:56.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9b406b110 msgr2=0x7fe9b406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9b406b110 0x7fe9b406d5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009fd0 tx=0x7fe9b8010000 comp rx=0 tx=0).stop 2023-11-24T06:10:56.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c80c9b50 msgr2=0x7fe9c80c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c80c9b50 0x7fe9c80c9fe0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8068c60 tx=0x7fe9c000e4f0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 shutdown_connections 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c80c9b50 0x7fe9c80c9fe0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9b406b110 0x7fe9b406d5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9b0003a00 0x7fe9c80cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 --2- 172.21.15.97:0/2488069247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c80cc320 0x7fe9c8150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 >> 172.21.15.97:0/2488069247 conn(0x7fe9c80d3e50 msgr2=0x7fe9c80d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 shutdown_connections 2023-11-24T06:10:56.804 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.800+0000 7fe9ce856700 1 -- 172.21.15.97:0/2488069247 wait complete. 2023-11-24T06:10:56.815 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T06:10:56.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.933+0000 7f3ded8ea700 1 Processor -- start 2023-11-24T06:10:56.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.933+0000 7f3ded8ea700 1 -- start start 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.933+0000 7f3ded8ea700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de815e0f0 0x7f3de80b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.933+0000 7f3ded8ea700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3de80b2b60 0x7f3de80ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.933+0000 7f3ded8ea700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de80cf530 con 0x7f3de80cf0d0 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.933+0000 7f3ded8ea700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de806bc00 con 0x7f3de80b2b60 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.933+0000 7f3ded8ea700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de815e4d0 con 0x7f3de815e0f0 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.933+0000 7f3de67fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3de80b2b60 0x7f3de80ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de67fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3de80b2b60 0x7f3de80ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48804/0 (socket says 172.21.15.97:48804) 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de67fc700 1 -- 172.21.15.97:0/2942276012 learned_addr learned my addr 172.21.15.97:0/2942276012 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de6ffd700 1 --2- 172.21.15.97:0/2942276012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de815e0f0 0x7f3de80b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de67fc700 1 -- 172.21.15.97:0/2942276012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de815e0f0 msgr2=0x7f3de80b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de67fc700 1 --2- 172.21.15.97:0/2942276012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de815e0f0 0x7f3de80b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de67fc700 1 -- 172.21.15.97:0/2942276012 >> v1:172.21.15.183:6789/0 conn(0x7f3de80cf0d0 legacy=0x7f3de8161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de67fc700 1 -- 172.21.15.97:0/2942276012 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de806aff0 con 0x7f3de80b2b60 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de67fc700 1 --2- 172.21.15.97:0/2942276012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de80ceb90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80098f0 tx=0x7f3dd8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44cd2bc0e0e7830 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de5ffb700 1 -- 172.21.15.97:0/2942276012 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800b8a0 con 0x7f3de80b2b60 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de5ffb700 1 -- 172.21.15.97:0/2942276012 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd800ba00 con 0x7f3de80b2b60 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.934+0000 7f3de5ffb700 1 -- 172.21.15.97:0/2942276012 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800bd90 con 0x7f3de80b2b60 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.935+0000 7f3ded8ea700 1 -- 172.21.15.97:0/2942276012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 msgr2=0x7f3de80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.935+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/2942276012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de80ceb90 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3dd80098f0 tx=0x7f3dd8009c00 comp rx=0 tx=0).stop 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.935+0000 7f3ded8ea700 1 -- 172.21.15.97:0/2942276012 shutdown_connections 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.935+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/2942276012 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de815e0f0 0x7f3de80b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.935+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/2942276012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de80ceb90 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.935+0000 7f3ded8ea700 1 -- 172.21.15.97:0/2942276012 >> 172.21.15.97:0/2942276012 conn(0x7f3de80b5ce0 msgr2=0x7f3de80b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.935+0000 7f3ded8ea700 1 -- 172.21.15.97:0/2942276012 shutdown_connections 2023-11-24T06:10:56.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.935+0000 7f3ded8ea700 1 -- 172.21.15.97:0/2942276012 wait complete. 2023-11-24T06:10:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.936+0000 7f3ded8ea700 1 Processor -- start 2023-11-24T06:10:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.936+0000 7f3ded8ea700 1 -- start start 2023-11-24T06:10:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3ded8ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3ded8ea700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de80cf0d0 0x7f3de81601d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3ded8ea700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3de815e0f0 0x7f3de81f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3ded8ea700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de815f6f0 con 0x7f3de80b2b60 2023-11-24T06:10:56.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3ded8ea700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de815eae0 con 0x7f3de815e0f0 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3ded8ea700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de815edc0 con 0x7f3de80cf0d0 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de815fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48810/0 (socket says 172.21.15.97:48810) 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 -- 172.21.15.97:0/1122633181 learned_addr learned my addr 172.21.15.97:0/1122633181 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de67fc700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de80cf0d0 0x7f3de81601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de77fe700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3de815e0f0 0x7f3de81f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 -- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de80cf0d0 msgr2=0x7f3de81601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de80cf0d0 0x7f3de81601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 -- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3de815e0f0 msgr2=0x7f3de81f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3de815e0f0 0x7f3de81f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:56.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 -- 172.21.15.97:0/1122633181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8009550 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de77fe700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3de815e0f0 0x7f3de81f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3de6ffd700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de815fc90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00e810 tx=0x7f3ddc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3dcffff700 1 -- 172.21.15.97:0/1122633181 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc013610 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3dcffff700 1 -- 172.21.15.97:0/1122633181 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0137b0 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de815f0a0 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.937+0000 7f3dcffff700 1 -- 172.21.15.97:0/1122633181 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc013b60 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.938+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de8158410 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.938+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de80751e0 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.939+0000 7f3dcffff700 1 -- 172.21.15.97:0/1122633181 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3ddc013cc0 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.939+0000 7f3dcffff700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3dd406b110 0x7f3dd406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.939+0000 7f3dcffff700 1 -- 172.21.15.97:0/1122633181 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3ddc0894c0 con 0x7f3de80b2b60 2023-11-24T06:10:56.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.939+0000 7f3de67fc700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3dd406b110 0x7f3dd406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:56.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.940+0000 7f3de67fc700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3dd406b110 0x7f3dd406d5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8014040 tx=0x7f3dd8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:56.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:56.944+0000 7f3dcffff700 1 -- 172.21.15.97:0/1122633181 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ddc054400 con 0x7f3de80b2b60 2023-11-24T06:10:57.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:57.215+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7f3de80683f0 con 0x7f3de80b2b60 2023-11-24T06:10:58.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.127+0000 7f3dcffff700 1 -- 172.21.15.97:0/1122633181 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v136) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3ddc057a20 con 0x7f3de80b2b60 2023-11-24T06:10:58.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.129+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3dd406b110 msgr2=0x7f3dd406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.129+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3dd406b110 0x7f3dd406d5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8014040 tx=0x7f3dd8008030 comp rx=0 tx=0).stop 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 msgr2=0x7f3de815fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de815fc90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00e810 tx=0x7f3ddc00eb20 comp rx=0 tx=0).stop 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 shutdown_connections 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3de815e0f0 0x7f3de81f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3de80b2b60 0x7f3de815fc90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3dd406b110 0x7f3dd406d5d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 --2- 172.21.15.97:0/1122633181 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3de80cf0d0 0x7f3de81601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 >> 172.21.15.97:0/1122633181 conn(0x7f3de80b5ce0 msgr2=0x7f3de80d1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 shutdown_connections 2023-11-24T06:10:58.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.130+0000 7f3ded8ea700 1 -- 172.21.15.97:0/1122633181 wait complete. 2023-11-24T06:10:58.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:10:58.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.273+0000 7f8aef872700 1 Processor -- start 2023-11-24T06:10:58.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.273+0000 7f8aef872700 1 -- start start 2023-11-24T06:10:58.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8aef872700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae806c710 0x7f8ae815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8aef872700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae815f280 con 0x7f8ae815ba30 2023-11-24T06:10:58.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8aef872700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae8161d20 con 0x7f8ae8162800 2023-11-24T06:10:58.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8aef872700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae815f560 con 0x7f8ae806c710 2023-11-24T06:10:58.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8aed60e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8ae815ba30 0x7f8ae806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41270/0 (socket says 172.21.15.97:41270) 2023-11-24T06:10:58.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8aed60e700 1 -- 172.21.15.97:0/3005169354 learned_addr learned my addr 172.21.15.97:0/3005169354 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:58.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332234483 0 0) 0x7f8ae815f280 con 0x7f8ae815ba30 2023-11-24T06:10:58.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad40035b0 con 0x7f8ae815ba30 2023-11-24T06:10:58.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.274+0000 7f8aece0d700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae806c710 0x7f8ae815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2747906793 0 0) 0x7f8ad40035b0 con 0x7f8ae815ba30 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae806c710 msgr2=0x7f8ae815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae806c710 0x7f8ae815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 >> v1:172.21.15.183:6789/0 conn(0x7f8ae8162800 legacy=0x7f8ae815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8162000 con 0x7f8ae815ba30 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f8ae4001670 con 0x7f8ae815ba30 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 >> v1:172.21.15.97:6789/0 conn(0x7f8ae815ba30 legacy=0x7f8ae806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae806c710 0x7f8ad4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ad4004060 0x7f8ad4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4004d90 0x7f8ad4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae815f560 con 0x7f8ad4004d90 2023-11-24T06:10:58.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad400bbd0 con 0x7f8ad4004060 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad400bd90 con 0x7f8ae806c710 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aede0f700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4004d90 0x7f8ad4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aece0d700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ad4004060 0x7f8ad4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aed60e700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae806c710 0x7f8ad4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aede0f700 1 -- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae806c710 msgr2=0x7f8ad4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aede0f700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae806c710 0x7f8ad4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aede0f700 1 -- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ad4004060 msgr2=0x7f8ad4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aede0f700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ad4004060 0x7f8ad4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aede0f700 1 -- 172.21.15.97:0/3005169354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40035b0 con 0x7f8ad4004d90 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8aede0f700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4004d90 0x7f8ad4005210 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000c190 tx=0x7f8ad4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05f7b84f7d1d79d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0001db0 con 0x7f8ad4004d90 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000ccd0 con 0x7f8ad4004d90 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.275+0000 7f8adffff700 1 -- 172.21.15.97:0/3005169354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0003c40 con 0x7f8ad4004d90 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.276+0000 7f8aef872700 1 -- 172.21.15.97:0/3005169354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4004d90 msgr2=0x7f8ad4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.276+0000 7f8aef872700 1 --2- 172.21.15.97:0/3005169354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ad4004d90 0x7f8ad4005210 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000c190 tx=0x7f8ad4005880 comp rx=0 tx=0).stop 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.276+0000 7f8aede0f700 1 -- 172.21.15.97:0/3005169354 reap_dead start 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.276+0000 7f8aef872700 1 -- 172.21.15.97:0/3005169354 shutdown_connections 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.276+0000 7f8aef872700 1 -- 172.21.15.97:0/3005169354 >> 172.21.15.97:0/3005169354 conn(0x7f8ae8071a70 msgr2=0x7f8ae8071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.276+0000 7f8aef872700 1 -- 172.21.15.97:0/3005169354 shutdown_connections 2023-11-24T06:10:58.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.276+0000 7f8aef872700 1 -- 172.21.15.97:0/3005169354 wait complete. 2023-11-24T06:10:58.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.277+0000 7f8aef872700 1 Processor -- start 2023-11-24T06:10:58.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.277+0000 7f8aef872700 1 -- start start 2023-11-24T06:10:58.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aef872700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ae81401d0 0x7f8ae81409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aef872700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ae8140f30 0x7f8ae8159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aef872700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae81597c0 0x7f8ae8157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aef872700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8158250 con 0x7f8ae8140f30 2023-11-24T06:10:58.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aef872700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae81584c0 con 0x7f8ae81401d0 2023-11-24T06:10:58.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aef872700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae81587a0 con 0x7f8ae81597c0 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aece0d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ae8140f30 0x7f8ae8159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aece0d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ae8140f30 0x7f8ae8159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48832/0 (socket says 172.21.15.97:48832) 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aece0d700 1 -- 172.21.15.97:0/861861124 learned_addr learned my addr 172.21.15.97:0/861861124 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.278+0000 7f8aed60e700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ae81401d0 0x7f8ae81409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aede0f700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae81597c0 0x7f8ae8157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aed60e700 1 -- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae81597c0 msgr2=0x7f8ae8157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aed60e700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae81597c0 0x7f8ae8157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aed60e700 1 -- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ae8140f30 msgr2=0x7f8ae8159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aed60e700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ae8140f30 0x7f8ae8159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aed60e700 1 -- 172.21.15.97:0/861861124 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae815bf90 con 0x7f8ae81401d0 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aece0d700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ae8140f30 0x7f8ae8159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:10:58.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aed60e700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ae81401d0 0x7f8ae81409f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4007290 tx=0x7f8ae40075a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8addffb700 1 -- 172.21.15.97:0/861861124 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4007780 con 0x7f8ae81401d0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8addffb700 1 -- 172.21.15.97:0/861861124 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae400f850 con 0x7f8ae81401d0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae8158a80 con 0x7f8ae81401d0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8addffb700 1 -- 172.21.15.97:0/861861124 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400fc00 con 0x7f8ae81401d0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.279+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae8146700 con 0x7f8ae81401d0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.281+0000 7f8addffb700 1 -- 172.21.15.97:0/861861124 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8ae400fd60 con 0x7f8ae81401d0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.281+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae8068fb0 con 0x7f8ae81401d0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.281+0000 7f8addffb700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8acc06b110 0x7f8acc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.282+0000 7f8addffb700 1 -- 172.21.15.97:0/861861124 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8ae4089770 con 0x7f8ae81401d0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.282+0000 7f8aece0d700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8acc06b110 0x7f8acc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.282+0000 7f8aece0d700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8acc06b110 0x7f8acc06d5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006c90 tx=0x7f8ad8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.288+0000 7f8addffb700 1 -- 172.21.15.97:0/861861124 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ae4054740 con 0x7f8ae81401d0 2023-11-24T06:10:58.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.476+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8ae80d07b0 con 0x7f8ae81401d0 2023-11-24T06:10:58.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.484+0000 7f8addffb700 1 -- 172.21.15.97:0/861861124 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ae4057d60 con 0x7f8ae81401d0 2023-11-24T06:10:58.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.486+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8acc06b110 msgr2=0x7f8acc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.486+0000 7f8aef872700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8acc06b110 0x7f8acc06d5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8006c90 tx=0x7f8ad8008040 comp rx=0 tx=0).stop 2023-11-24T06:10:58.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.486+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ae81401d0 msgr2=0x7f8ae81409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.486+0000 7f8aef872700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ae81401d0 0x7f8ae81409f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4007290 tx=0x7f8ae40075a0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.486+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 shutdown_connections 2023-11-24T06:10:58.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.486+0000 7f8aef872700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8acc06b110 0x7f8acc06d5d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.486+0000 7f8aef872700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8ae81597c0 0x7f8ae8157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.486+0000 7f8aef872700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ae81401d0 0x7f8ae81409f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.487+0000 7f8aef872700 1 --2- 172.21.15.97:0/861861124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ae8140f30 0x7f8ae8159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.487+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 >> 172.21.15.97:0/861861124 conn(0x7f8ae8071a70 msgr2=0x7f8ae815f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.487+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 shutdown_connections 2023-11-24T06:10:58.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.487+0000 7f8aef872700 1 -- 172.21.15.97:0/861861124 wait complete. 2023-11-24T06:10:58.501 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:10:58.502 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:10:58.539 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:10:58.589 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:10:58.589 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:58.589 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:58.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:10:58.607 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:58.607 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:58.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:10:58.627 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:10:58.627 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:58.664 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:10:58.664 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:10:58.664 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:10:58.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:10:58.686 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:10:58.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:10:58.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.821+0000 7f932dbf5700 1 Processor -- start 2023-11-24T06:10:58.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.821+0000 7f932dbf5700 1 -- start start 2023-11-24T06:10:58.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f932dbf5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f932815ae20 0x7f93280b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f932dbf5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f932815b200 0x7f93280b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f932dbf5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280cfd40 con 0x7f93280cf780 2023-11-24T06:10:58.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f932dbf5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93280d0020 con 0x7f932815ae20 2023-11-24T06:10:58.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f932dbf5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93280affa0 con 0x7f932815b200 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93277fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f932815ae20 0x7f93280b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93277fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f932815ae20 0x7f93280b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48838/0 (socket says 172.21.15.97:48838) 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93277fe700 1 -- 172.21.15.97:0/408613651 learned_addr learned my addr 172.21.15.97:0/408613651 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f9326ffd700 1 --2- 172.21.15.97:0/408613651 >> v2:172.21.15.183:3300/0 conn(0x7f932815b200 0x7f93280b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93277fe700 1 -- 172.21.15.97:0/408613651 >> v2:172.21.15.183:3300/0 conn(0x7f932815b200 msgr2=0x7f93280b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93277fe700 1 --2- 172.21.15.97:0/408613651 >> v2:172.21.15.183:3300/0 conn(0x7f932815b200 0x7f93280b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93277fe700 1 -- 172.21.15.97:0/408613651 >> v1:172.21.15.183:6789/0 conn(0x7f93280cf780 legacy=0x7f93280b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93277fe700 1 -- 172.21.15.97:0/408613651 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280b0280 con 0x7f932815ae20 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93277fe700 1 --2- 172.21.15.97:0/408613651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932815ae20 0x7f93280b29a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f93280cefe0 tx=0x7f931c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7054e77a6b005461 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.822+0000 7f93267fc700 1 -- 172.21.15.97:0/408613651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00dc70 con 0x7f932815ae20 2023-11-24T06:10:58.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f93267fc700 1 -- 172.21.15.97:0/408613651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c00ddd0 con 0x7f932815ae20 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f93267fc700 1 -- 172.21.15.97:0/408613651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c010690 con 0x7f932815ae20 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f932dbf5700 1 -- 172.21.15.97:0/408613651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932815ae20 msgr2=0x7f93280b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f932dbf5700 1 --2- 172.21.15.97:0/408613651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932815ae20 0x7f93280b29a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f93280cefe0 tx=0x7f931c00be70 comp rx=0 tx=0).stop 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f932dbf5700 1 -- 172.21.15.97:0/408613651 shutdown_connections 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f932dbf5700 1 --2- 172.21.15.97:0/408613651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f932815ae20 0x7f93280b29a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f932dbf5700 1 --2- 172.21.15.97:0/408613651 >> v2:172.21.15.183:3300/0 conn(0x7f932815b200 0x7f93280b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f932dbf5700 1 -- 172.21.15.97:0/408613651 >> 172.21.15.97:0/408613651 conn(0x7f93280b57b0 msgr2=0x7f93280b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f932dbf5700 1 -- 172.21.15.97:0/408613651 shutdown_connections 2023-11-24T06:10:58.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.823+0000 7f932dbf5700 1 -- 172.21.15.97:0/408613651 wait complete. 2023-11-24T06:10:58.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.824+0000 7f932dbf5700 1 Processor -- start 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f932dbf5700 1 -- start start 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f932dbf5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93280cf780 0x7f93280998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f932dbf5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f932815ae20 0x7f932806e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f932dbf5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f932815b200 0x7f932806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f932dbf5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9328155e00 con 0x7f93280cf780 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f932dbf5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9328156610 con 0x7f932815b200 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f932dbf5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9328156050 con 0x7f932815ae20 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f93277fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93280cf780 0x7f93280998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f93277fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93280cf780 0x7f93280998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48844/0 (socket says 172.21.15.97:48844) 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.825+0000 7f93277fe700 1 -- 172.21.15.97:0/2470310345 learned_addr learned my addr 172.21.15.97:0/2470310345 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f93277fe700 1 -- 172.21.15.97:0/2470310345 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f932815ae20 msgr2=0x7f932806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f93277fe700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f932815ae20 0x7f932806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f93277fe700 1 -- 172.21.15.97:0/2470310345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f932815b200 msgr2=0x7f932806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f93277fe700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f932815b200 0x7f932806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f93277fe700 1 -- 172.21.15.97:0/2470310345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c00b7f0 con 0x7f93280cf780 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f93277fe700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93280cf780 0x7f93280998c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f931c00beb0 tx=0x7f931c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f930ffff700 1 -- 172.21.15.97:0/2470310345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c01f070 con 0x7f93280cf780 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f930ffff700 1 -- 172.21.15.97:0/2470310345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c0039f0 con 0x7f93280cf780 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9328156330 con 0x7f93280cf780 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f930ffff700 1 -- 172.21.15.97:0/2470310345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c003dc0 con 0x7f93280cf780 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.826+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9328099e00 con 0x7f93280cf780 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.827+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9328067f60 con 0x7f93280cf780 2023-11-24T06:10:58.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.827+0000 7f930ffff700 1 -- 172.21.15.97:0/2470310345 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f931c019400 con 0x7f93280cf780 2023-11-24T06:10:58.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.828+0000 7f930ffff700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f931406b110 0x7f931406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:58.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.828+0000 7f930ffff700 1 -- 172.21.15.97:0/2470310345 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f931c08cbc0 con 0x7f93280cf780 2023-11-24T06:10:58.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.828+0000 7f9326ffd700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f931406b110 0x7f931406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:58.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.828+0000 7f9326ffd700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f931406b110 0x7f931406d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9328061dd0 tx=0x7f93180095a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:58.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:58.832+0000 7f930ffff700 1 -- 172.21.15.97:0/2470310345 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f931c057b90 con 0x7f93280cf780 2023-11-24T06:10:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.064+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93280cde80 con 0x7f93280cf780 2023-11-24T06:10:59.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.065+0000 7f930ffff700 1 -- 172.21.15.97:0/2470310345 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f931c05b1b0 con 0x7f93280cf780 2023-11-24T06:10:59.068 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:10:59.068 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4937,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/163703518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":163703518},{"type":"v1","addr":"172.21.15.97:6837","nonce":163703518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/1688123009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1688123009},{"type":"v1","addr":"172.21.15.183:6834","nonce":1688123009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/310487911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":310487911},{"type":"v1","addr":"172.21.15.183:6835","nonce":310487911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/4039097546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":4039097546},{"type":"v1","addr":"172.21.15.97:6835","nonce":4039097546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2023-11-24T06:10:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f931406b110 msgr2=0x7f931406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f931406b110 0x7f931406d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9328061dd0 tx=0x7f93180095a0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93280cf780 msgr2=0x7f93280998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93280cf780 0x7f93280998c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f931c00beb0 tx=0x7f931c004750 comp rx=0 tx=0).stop 2023-11-24T06:10:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 shutdown_connections 2023-11-24T06:10:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f932815b200 0x7f932806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93280cf780 0x7f93280998c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f931406b110 0x7f931406d5d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 --2- 172.21.15.97:0/2470310345 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f932815ae20 0x7f932806e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 >> 172.21.15.97:0/2470310345 conn(0x7f93280b57b0 msgr2=0x7f93280b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 shutdown_connections 2023-11-24T06:10:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.067+0000 7f932dbf5700 1 -- 172.21.15.97:0/2470310345 wait complete. 2023-11-24T06:10:59.071 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 29 2023-11-24T06:10:59.082 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:10:59.082 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:10:59.082 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:10:59.082 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:10:59.082 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:10:59.083 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:10:59.083 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:10:59.083 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:10:59.083 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:10:59.083 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:10:59.083 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:10:59.083 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:10:59.084 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:10:59.084 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:10:59.084 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:10:59.084 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:10:59.091 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:10:59.092 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:10:59.092 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:10:59.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:10:59.095 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:10:59.095 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:10:59.096 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:10:59.096 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:10:59.134 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:10:59.134 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:59.135 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:10:59.135 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:59.138 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:10:59.138 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:59.139 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:10:59.139 DEBUG:teuthology.parallel:result is None 2023-11-24T06:10:59.139 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:10:59.212 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-11-24T06:10:59.213 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-11-24T06:10:59.223 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-11-24T06:10:59.224 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-11-24T06:10:59.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.264+0000 7f60af4b7700 1 Processor -- start 2023-11-24T06:10:59.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.264+0000 7f60af4b7700 1 -- start start 2023-11-24T06:10:59.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f60af4b7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a8168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f60af4b7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a806bc00 con 0x7f60a815e0f0 2023-11-24T06:10:59.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f60af4b7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a815e4d0 con 0x7f60a80b2b60 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f60af4b7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a806aff0 con 0x7f60a80cf2a0 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f60ad253700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f60a815e0f0 0x7f60a80b9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41328/0 (socket says 172.21.15.97:41328) 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f60ad253700 1 -- 172.21.15.97:0/2492544235 learned_addr learned my addr 172.21.15.97:0/2492544235 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f60ada54700 1 --2- 172.21.15.97:0/2492544235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a8168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f609ffff700 1 -- 172.21.15.97:0/2492544235 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097490848 0 0) 0x7f60a806bc00 con 0x7f60a815e0f0 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.265+0000 7f609ffff700 1 -- 172.21.15.97:0/2492544235 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60940035b0 con 0x7f60a815e0f0 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f609ffff700 1 -- 172.21.15.97:0/2492544235 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664623142 0 0) 0x7f60a815e4d0 con 0x7f60a80b2b60 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f609ffff700 1 -- 172.21.15.97:0/2492544235 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60a806bc00 con 0x7f60a80b2b60 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f60ada54700 1 -- 172.21.15.97:0/2492544235 >> v1:172.21.15.183:6789/0 conn(0x7f60a80b2b60 legacy=0x7f60a80ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f60ada54700 1 -- 172.21.15.97:0/2492544235 >> v1:172.21.15.97:6789/0 conn(0x7f60a815e0f0 legacy=0x7f60a80b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f60ada54700 1 -- 172.21.15.97:0/2492544235 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a806b2d0 con 0x7f60a80cf2a0 2023-11-24T06:10:59.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f60ada54700 1 --2- 172.21.15.97:0/2492544235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a8168f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009dc0 tx=0x7f60a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fa52b90ab0a2d07 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f609ffff700 1 -- 172.21.15.97:0/2492544235 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a000e070 con 0x7f60a80cf2a0 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f609ffff700 1 -- 172.21.15.97:0/2492544235 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a000bb20 con 0x7f60a80cf2a0 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f609ffff700 1 -- 172.21.15.97:0/2492544235 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0012430 con 0x7f60a80cf2a0 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f60af4b7700 1 -- 172.21.15.97:0/2492544235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 msgr2=0x7f60a8168f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.266+0000 7f60af4b7700 1 --2- 172.21.15.97:0/2492544235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a8168f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009dc0 tx=0x7f60a000b330 comp rx=0 tx=0).stop 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.267+0000 7f60af4b7700 1 -- 172.21.15.97:0/2492544235 shutdown_connections 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.267+0000 7f60af4b7700 1 --2- 172.21.15.97:0/2492544235 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a8168f10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.267+0000 7f60af4b7700 1 -- 172.21.15.97:0/2492544235 >> 172.21.15.97:0/2492544235 conn(0x7f60a80b5ce0 msgr2=0x7f60a80b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.267+0000 7f60af4b7700 1 -- 172.21.15.97:0/2492544235 shutdown_connections 2023-11-24T06:10:59.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.267+0000 7f60af4b7700 1 -- 172.21.15.97:0/2492544235 wait complete. 2023-11-24T06:10:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.268+0000 7f60af4b7700 1 Processor -- start 2023-11-24T06:10:59.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.268+0000 7f60af4b7700 1 -- start start 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60af4b7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60a80b2b60 0x7f60a80c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60af4b7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a80cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60af4b7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60a815e0f0 0x7f60a815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60af4b7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a81f71d0 con 0x7f60a815e0f0 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60af4b7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a80c7f80 con 0x7f60a80b2b60 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60af4b7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a80c8260 con 0x7f60a80cf2a0 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60a815e0f0 0x7f60a815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60a815e0f0 0x7f60a815aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48886/0 (socket says 172.21.15.97:48886) 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 -- 172.21.15.97:0/3233534101 learned_addr learned my addr 172.21.15.97:0/3233534101 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60aca52700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a80cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ad253700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60a80b2b60 0x7f60a80c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 -- 172.21.15.97:0/3233534101 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 msgr2=0x7f60a80cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a80cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 -- 172.21.15.97:0/3233534101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60a80b2b60 msgr2=0x7f60a80c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60a80b2b60 0x7f60a80c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 -- 172.21.15.97:0/3233534101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a0009a70 con 0x7f60a815e0f0 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60ada54700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60a815e0f0 0x7f60a815aea0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b370 tx=0x7f60a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f609dffb700 1 -- 172.21.15.97:0/3233534101 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a001b810 con 0x7f60a815e0f0 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f609dffb700 1 -- 172.21.15.97:0/3233534101 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a0020070 con 0x7f60a815e0f0 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a81f7440 con 0x7f60a815e0f0 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.269+0000 7f609dffb700 1 -- 172.21.15.97:0/3233534101 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a001bbf0 con 0x7f60a815e0f0 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.270+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a815b3e0 con 0x7f60a815e0f0 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.270+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a81f75d0 con 0x7f60a815e0f0 2023-11-24T06:10:59.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.271+0000 7f609dffb700 1 -- 172.21.15.97:0/3233534101 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f60a001bd50 con 0x7f60a815e0f0 2023-11-24T06:10:59.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.271+0000 7f609dffb700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f608c06b110 0x7f608c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:10:59.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.271+0000 7f609dffb700 1 -- 172.21.15.97:0/3233534101 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f60a0022070 con 0x7f60a815e0f0 2023-11-24T06:10:59.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.271+0000 7f60ad253700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f608c06b110 0x7f608c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:10:59.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.272+0000 7f60ad253700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f608c06b110 0x7f608c06d5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f60980011e0 tx=0x7f6098009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:10:59.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.276+0000 7f609dffb700 1 -- 172.21.15.97:0/3233534101 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f60a0056bb0 con 0x7f60a815e0f0 2023-11-24T06:10:59.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:10:59.481+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f60a80751e0 con 0x7f60a815e0f0 2023-11-24T06:11:00.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.143+0000 7f609dffb700 1 -- 172.21.15.97:0/3233534101 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7f60a002c020 con 0x7f60a815e0f0 2023-11-24T06:11:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f608c06b110 msgr2=0x7f608c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f608c06b110 0x7f608c06d5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f60980011e0 tx=0x7f6098009040 comp rx=0 tx=0).stop 2023-11-24T06:11:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60a815e0f0 msgr2=0x7f60a815aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60a815e0f0 0x7f60a815aea0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b370 tx=0x7f60a0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 shutdown_connections 2023-11-24T06:11:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60a815e0f0 0x7f60a815aea0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f60a80b2b60 0x7f60a80c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f608c06b110 0x7f608c06d5d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 --2- 172.21.15.97:0/3233534101 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60a80cf2a0 0x7f60a80cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 >> 172.21.15.97:0/3233534101 conn(0x7f60a80b5ce0 msgr2=0x7f60a80d3300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 shutdown_connections 2023-11-24T06:11:00.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.147+0000 7f60af4b7700 1 -- 172.21.15.97:0/3233534101 wait complete. 2023-11-24T06:11:00.151 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-11-24T06:11:00.166 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:11:00.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.290+0000 7f108c91a700 1 Processor -- start 2023-11-24T06:11:00.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.290+0000 7f108c91a700 1 -- start start 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.291+0000 7f108c91a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 0x7f10880b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.291+0000 7f108c91a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10880af4d0 0x7f10880afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.291+0000 7f108c91a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10880ae430 con 0x7f108806c000 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.291+0000 7f108c91a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10880af0b0 con 0x7f10880af4d0 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.291+0000 7f108c91a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10880b05a0 con 0x7f108815e080 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.291+0000 7f1086d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f108806c000 0x7f1088073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41348/0 (socket says 172.21.15.97:41348) 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f1086d9d700 1 -- 172.21.15.97:0/2749983110 learned_addr learned my addr 172.21.15.97:0/2749983110 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416325091 0 0) 0x7f10880ae430 con 0x7f108806c000 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10700035b0 con 0x7f108806c000 2023-11-24T06:11:00.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108659c700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 0x7f10880b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f1085d9b700 1 --2- 172.21.15.97:0/2749983110 >> v2:172.21.15.183:3300/0 conn(0x7f10880af4d0 0x7f10880afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1938552738 0 0) 0x7f10700035b0 con 0x7f108806c000 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 msgr2=0x7f10880b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 0x7f10880b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 >> v2:172.21.15.183:3300/0 conn(0x7f10880af4d0 msgr2=0x7f10880afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 --2- 172.21.15.97:0/2749983110 >> v2:172.21.15.183:3300/0 conn(0x7f10880af4d0 0x7f10880afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10880ae710 con 0x7f108806c000 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f10800019f0 con 0x7f108806c000 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108659c700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 0x7f10880b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 >> v1:172.21.15.97:6789/0 conn(0x7f108806c000 legacy=0x7f1088073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070003a30 0x7f1070003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 0x7f1070004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1070004d90 0x7f1070005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10880af0b0 con 0x7f1070003a30 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10880b05a0 con 0x7f1070004d90 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107000bac0 con 0x7f108815e080 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108659c700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070003a30 0x7f1070003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f1085d9b700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 0x7f1070004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f1086d9d700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1070004d90 0x7f1070005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108659c700 1 -- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 msgr2=0x7f1070004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108659c700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f108815e080 0x7f1070004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108659c700 1 -- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1070004d90 msgr2=0x7f1070005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108659c700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1070004d90 0x7f1070005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f108659c700 1 -- 172.21.15.97:0/2749983110 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10700035b0 con 0x7f1070003a30 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.292+0000 7f1086d9d700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1070004d90 0x7f1070005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108659c700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070003a30 0x7f1070003e50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f107c009950 tx=0x7f1070005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775185f3351e1586 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c00c070 con 0x7f1070003a30 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c013610 con 0x7f1070003a30 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108559a700 1 -- 172.21.15.97:0/2749983110 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0139a0 con 0x7f1070003a30 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108c91a700 1 -- 172.21.15.97:0/2749983110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070003a30 msgr2=0x7f1070003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108c91a700 1 --2- 172.21.15.97:0/2749983110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1070003a30 0x7f1070003e50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f107c009950 tx=0x7f1070005880 comp rx=0 tx=0).stop 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f1086d9d700 1 -- 172.21.15.97:0/2749983110 reap_dead start 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108c91a700 1 -- 172.21.15.97:0/2749983110 shutdown_connections 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108c91a700 1 -- 172.21.15.97:0/2749983110 >> 172.21.15.97:0/2749983110 conn(0x7f1088071b90 msgr2=0x7f1088071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108c91a700 1 -- 172.21.15.97:0/2749983110 shutdown_connections 2023-11-24T06:11:00.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.293+0000 7f108c91a700 1 -- 172.21.15.97:0/2749983110 wait complete. 2023-11-24T06:11:00.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.295+0000 7f108c91a700 1 Processor -- start 2023-11-24T06:11:00.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.295+0000 7f108c91a700 1 -- start start 2023-11-24T06:11:00.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.295+0000 7f108c91a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1070003a30 0x7f10881f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.295+0000 7f108c91a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10881420e0 0x7f1088159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.295+0000 7f108c91a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088157250 0x7f10881576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.295+0000 7f108c91a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108815a640 con 0x7f1088157250 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.295+0000 7f108c91a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1088157c10 con 0x7f10881420e0 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.295+0000 7f108c91a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1088159ac0 con 0x7f1070003a30 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088157250 0x7f10881576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088157250 0x7f10881576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48908/0 (socket says 172.21.15.97:48908) 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 -- 172.21.15.97:0/3282148879 learned_addr learned my addr 172.21.15.97:0/3282148879 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f108659c700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1070003a30 0x7f10881f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1085d9b700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10881420e0 0x7f1088159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 -- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1070003a30 msgr2=0x7f10881f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1070003a30 0x7f10881f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 -- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10881420e0 msgr2=0x7f1088159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10881420e0 0x7f1088159240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 -- 172.21.15.97:0/3282148879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088159da0 con 0x7f1088157250 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f108659c700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1070003a30 0x7f10881f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1086d9d700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088157250 0x7f10881576d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1080001370 tx=0x7f108000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1076ffd700 1 -- 172.21.15.97:0/3282148879 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108000ee90 con 0x7f1088157250 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1076ffd700 1 -- 172.21.15.97:0/3282148879 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1080013420 con 0x7f1088157250 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108815a080 con 0x7f1088157250 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f1076ffd700 1 -- 172.21.15.97:0/3282148879 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10800137b0 con 0x7f1088157250 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.296+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1088142560 con 0x7f1088157250 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.297+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10880753f0 con 0x7f1088157250 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.298+0000 7f1076ffd700 1 -- 172.21.15.97:0/3282148879 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f1080013910 con 0x7f1088157250 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.298+0000 7f1076ffd700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106806b110 0x7f106806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.298+0000 7f1076ffd700 1 -- 172.21.15.97:0/3282148879 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1080089c30 con 0x7f1088157250 2023-11-24T06:11:00.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.298+0000 7f108659c700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106806b110 0x7f106806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.298+0000 7f108659c700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106806b110 0x7f106806d5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f107c016040 tx=0x7f107c01e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.302+0000 7f1076ffd700 1 -- 172.21.15.97:0/3282148879 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1080054e30 con 0x7f1088157250 2023-11-24T06:11:00.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.570+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f10880ae430 con 0x7f1088157250 2023-11-24T06:11:00.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.570+0000 7f1076ffd700 1 -- 172.21.15.97:0/3282148879 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1080058450 con 0x7f1088157250 2023-11-24T06:11:00.573 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:00.573 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:11:00.573 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlijU/IxAAp5/+zEd6ER/JFqum0ptk0Q==", 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.574 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLftpLxAAH+Ew2GQv73YtKLzZJjSBew==", 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlGtxpKhAAiQm5ore3YYETyG+sIKzEcw==", 2023-11-24T06:11:00.575 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBl05iyNhAAHZYZ5pGghTwe55n84iXXQw==", 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:00.576 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDAPWBlwipJFBAAGLCD8wGcYoSy4yM4RXb0/Q==", 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDDPWBlgW7fExAAQuUSGJn+ZfHSMwSi5r1pQg==", 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-11-24T06:11:00.578 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDGPWBl3E8uExAAJKjzbRFyYie6dmnRrP7VVw==", 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJPWBl5gnRExAAbKbgF6DcdTSv8M84GXyONw==", 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:00.579 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDPPWBlmCfaFRAAsIQWHUnlgDQcRXLndlUZKg==", 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSPWBlPxfqFRAARyMzgivfTfgmFxZRjp54JQ==", 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVPWBl8cjTFBAAsNfkNY4gGVTJwh+mMtADgg==", 2023-11-24T06:11:00.581 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDYPWBljh8EFBAA4nVPoyEXigIDTUB6enXsXA==", 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:00.582 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA==", 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.583 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlOXAsBhAABOT6CW2RVj7Iv5XMPvQTgQ==", 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlJRe4AhAAGQVWmqQr/urD7AefOfmoZQ==", 2023-11-24T06:11:00.584 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlwX2YCRAAIM9eB39aohf1xUdEvHr/tA==", 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:11:00.585 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlVBcHBhAAbu6IOCKCLUA+5APAYwbRUw==", 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:00.586 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlOezbHxAAlNbpLHDzRHvk2SXFvQoTXQ==", 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:00.587 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBla7myGxAAHEPHxXHfBzCeqi2nLjhK6g==", 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-11-24T06:11:00.588 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106806b110 msgr2=0x7f106806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106806b110 0x7f106806d5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f107c016040 tx=0x7f107c01e040 comp rx=0 tx=0).stop 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088157250 msgr2=0x7f10881576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088157250 0x7f10881576d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1080001370 tx=0x7f108000e720 comp rx=0 tx=0).stop 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 shutdown_connections 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1088157250 0x7f10881576d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f106806b110 0x7f106806d5d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1070003a30 0x7f10881f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 --2- 172.21.15.97:0/3282148879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10881420e0 0x7f1088159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 >> 172.21.15.97:0/3282148879 conn(0x7f1088071b90 msgr2=0x7f108806e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 shutdown_connections 2023-11-24T06:11:00.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.572+0000 7f108c91a700 1 -- 172.21.15.97:0/3282148879 wait complete. 2023-11-24T06:11:00.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:00.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.709+0000 7f411e7b6700 1 Processor -- start 2023-11-24T06:11:00.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.709+0000 7f411e7b6700 1 -- start start 2023-11-24T06:11:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f411e7b6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4118144660 0x7f41180b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f411e7b6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411806c710 0x7f411815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f411e7b6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411815f520 con 0x7f411815ea60 2023-11-24T06:11:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f411e7b6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41180b4230 con 0x7f4118144660 2023-11-24T06:11:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f411e7b6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41180b3c40 con 0x7f411806c710 2023-11-24T06:11:00.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f41177fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f411815ea60 0x7f411806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41366/0 (socket says 172.21.15.97:41366) 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f41177fe700 1 -- 172.21.15.97:0/2071120804 learned_addr learned my addr 172.21.15.97:0/2071120804 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f4117fff700 1 --2- 172.21.15.97:0/2071120804 >> v2:172.21.15.183:3300/0 conn(0x7f4118144660 0x7f41180b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f411cd53700 1 --2- 172.21.15.97:0/2071120804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411806c710 0x7f411815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f4116ffd700 1 -- 172.21.15.97:0/2071120804 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602241984 0 0) 0x7f411815f520 con 0x7f411815ea60 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f4116ffd700 1 -- 172.21.15.97:0/2071120804 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41000035b0 con 0x7f411815ea60 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f4116ffd700 1 -- 172.21.15.97:0/2071120804 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006757702 0 0) 0x7f41000035b0 con 0x7f411815ea60 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f4117fff700 1 -- 172.21.15.97:0/2071120804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411806c710 msgr2=0x7f411815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f4117fff700 1 --2- 172.21.15.97:0/2071120804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411806c710 0x7f411815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f4117fff700 1 -- 172.21.15.97:0/2071120804 >> v1:172.21.15.97:6789/0 conn(0x7f411815ea60 legacy=0x7f411806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.710+0000 7f4117fff700 1 -- 172.21.15.97:0/2071120804 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411815f800 con 0x7f4118144660 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f4117fff700 1 --2- 172.21.15.97:0/2071120804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 0x7f41180b4650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f410c009840 tx=0x7f410c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74de0a8d67d3ba9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f4116ffd700 1 -- 172.21.15.97:0/2071120804 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4108001670 con 0x7f4118144660 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f4116ffd700 1 -- 172.21.15.97:0/2071120804 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c004030 con 0x7f4118144660 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f4116ffd700 1 -- 172.21.15.97:0/2071120804 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0043c0 con 0x7f4118144660 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f411e7b6700 1 -- 172.21.15.97:0/2071120804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 msgr2=0x7f41180b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f411e7b6700 1 --2- 172.21.15.97:0/2071120804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 0x7f41180b4650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f410c009840 tx=0x7f410c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f411e7b6700 1 -- 172.21.15.97:0/2071120804 shutdown_connections 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f411e7b6700 1 --2- 172.21.15.97:0/2071120804 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411806c710 0x7f411815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f411e7b6700 1 --2- 172.21.15.97:0/2071120804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 0x7f41180b4650 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f411e7b6700 1 -- 172.21.15.97:0/2071120804 >> 172.21.15.97:0/2071120804 conn(0x7f4118071ac0 msgr2=0x7f4118071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f411e7b6700 1 -- 172.21.15.97:0/2071120804 shutdown_connections 2023-11-24T06:11:00.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.711+0000 7f411e7b6700 1 -- 172.21.15.97:0/2071120804 wait complete. 2023-11-24T06:11:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.712+0000 7f411e7b6700 1 Processor -- start 2023-11-24T06:11:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.713+0000 7f411e7b6700 1 -- start start 2023-11-24T06:11:00.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.713+0000 7f411e7b6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f411806c710 0x7f41181506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.713+0000 7f411e7b6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 0x7f41180afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.713+0000 7f411e7b6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411815ea60 0x7f41180affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.713+0000 7f411e7b6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180aaec0 con 0x7f411806c710 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.713+0000 7f411e7b6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180aa2b0 con 0x7f4118144660 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.713+0000 7f411e7b6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41180aa590 con 0x7f411815ea60 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.713+0000 7f4117fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f411806c710 0x7f41181506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4117fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f411806c710 0x7f41181506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48918/0 (socket says 172.21.15.97:48918) 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4117fff700 1 -- 172.21.15.97:0/3935868855 learned_addr learned my addr 172.21.15.97:0/3935868855 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f41177fe700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 0x7f41180afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f411cd53700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411815ea60 0x7f41180affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4117fff700 1 -- 172.21.15.97:0/3935868855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411815ea60 msgr2=0x7f41180affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4117fff700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411815ea60 0x7f41180affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4117fff700 1 -- 172.21.15.97:0/3935868855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 msgr2=0x7f41180afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4117fff700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 0x7f41180afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4117fff700 1 -- 172.21.15.97:0/3935868855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c0094a0 con 0x7f411806c710 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4117fff700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f411806c710 0x7f41181506c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f410c0090b0 tx=0x7f410c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4114ff9700 1 -- 172.21.15.97:0/3935868855 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c003e20 con 0x7f411806c710 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4114ff9700 1 -- 172.21.15.97:0/3935868855 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c01f070 con 0x7f411806c710 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41180aa870 con 0x7f411806c710 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f4114ff9700 1 -- 172.21.15.97:0/3935868855 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c011870 con 0x7f411806c710 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.714+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41180ab350 con 0x7f411806c710 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.715+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41180ab4e0 con 0x7f411806c710 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.716+0000 7f4114ff9700 1 -- 172.21.15.97:0/3935868855 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f410c0119d0 con 0x7f411806c710 2023-11-24T06:11:00.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.716+0000 7f4114ff9700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410406b110 0x7f410406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.716+0000 7f4114ff9700 1 -- 172.21.15.97:0/3935868855 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f410c021070 con 0x7f411806c710 2023-11-24T06:11:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.716+0000 7f41177fe700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410406b110 0x7f410406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.716+0000 7f41177fe700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410406b110 0x7f410406d5d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4108001eb0 tx=0x7f4108001c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.720+0000 7f4114ff9700 1 -- 172.21.15.97:0/3935868855 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f410c056110 con 0x7f411806c710 2023-11-24T06:11:00.914 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.912+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41180b5690 con 0x7f411806c710 2023-11-24T06:11:00.915 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.913+0000 7f4114ff9700 1 -- 172.21.15.97:0/3935868855 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12869 (secure 0 0 0) 0x7f410c059730 con 0x7f411806c710 2023-11-24T06:11:00.916 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:00.916 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:00.151590+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410406b110 msgr2=0x7f410406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410406b110 0x7f410406d5d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4108001eb0 tx=0x7f4108001c40 comp rx=0 tx=0).stop 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f411806c710 msgr2=0x7f41181506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f411806c710 0x7f41181506c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f410c0090b0 tx=0x7f410c004750 comp rx=0 tx=0).stop 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 shutdown_connections 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410406b110 0x7f410406d5d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f411815ea60 0x7f41180affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f411806c710 0x7f41181506c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 --2- 172.21.15.97:0/3935868855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4118144660 0x7f41180afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 >> 172.21.15.97:0/3935868855 conn(0x7f4118071ac0 msgr2=0x7f411806ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:00.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 shutdown_connections 2023-11-24T06:11:00.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:00.915+0000 7f411e7b6700 1 -- 172.21.15.97:0/3935868855 wait complete. 2023-11-24T06:11:00.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:00.930 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:01.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.050+0000 7f6691ed2700 1 Processor -- start 2023-11-24T06:11:01.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.051+0000 7f6691ed2700 1 -- start start 2023-11-24T06:11:01.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.051+0000 7f6691ed2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f668c15bf50 0x7f668c0ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.051+0000 7f6691ed2700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0d0140 con 0x7f668c0ce830 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.051+0000 7f6691ed2700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668c0cf3c0 con 0x7f668c0c6c30 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.051+0000 7f6691ed2700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668c15c330 con 0x7f668c15bf50 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.051+0000 7f668affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f668c0ce830 0x7f668c0cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41380/0 (socket says 172.21.15.97:41380) 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.051+0000 7f668affd700 1 -- 172.21.15.97:0/3585800061 learned_addr learned my addr 172.21.15.97:0/3585800061 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845209155 0 0) 0x7f668c0d0140 con 0x7f668c0ce830 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66740035b0 con 0x7f668c0ce830 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668b7fe700 1 --2- 172.21.15.97:0/3585800061 >> v2:172.21.15.183:3300/0 conn(0x7f668c15bf50 0x7f668c0ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3066232468 0 0) 0x7f66740035b0 con 0x7f668c0ce830 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 >> v2:172.21.15.183:3300/0 conn(0x7f668c15bf50 msgr2=0x7f668c0ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 --2- 172.21.15.97:0/3585800061 >> v2:172.21.15.183:3300/0 conn(0x7f668c15bf50 0x7f668c0ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 >> v1:172.21.15.183:6789/0 conn(0x7f668c0c6c30 legacy=0x7f668c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0d03e0 con 0x7f668c0ce830 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f667c001670 con 0x7f668c0ce830 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 >> v1:172.21.15.97:6789/0 conn(0x7f668c0ce830 legacy=0x7f668c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6674003a00 0x7f6674003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6674004850 0x7f6674004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674005210 0x7f667400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c15c330 con 0x7f6674003a00 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667400c160 con 0x7f6674005210 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667400c320 con 0x7f6674004850 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668b7fe700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6674003a00 0x7f6674003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668affd700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6674004850 0x7f6674004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668bfff700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674005210 0x7f667400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668b7fe700 1 -- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6674004850 msgr2=0x7f6674004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668b7fe700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6674004850 0x7f6674004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668b7fe700 1 -- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674005210 msgr2=0x7f667400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668b7fe700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674005210 0x7f667400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668b7fe700 1 -- 172.21.15.97:0/3585800061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66740035b0 con 0x7f6674003a00 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668bfff700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674005210 0x7f667400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.052+0000 7f668bfff700 1 -- 172.21.15.97:0/3585800061 reap_dead start 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f668b7fe700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6674003a00 0x7f6674003e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6680009890 tx=0x7f6674005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e9f611108f2c56 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000c070 con 0x7f6674003a00 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680013610 con 0x7f6674003a00 2023-11-24T06:11:01.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f668a7fc700 1 -- 172.21.15.97:0/3585800061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66800139a0 con 0x7f6674003a00 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f6691ed2700 1 -- 172.21.15.97:0/3585800061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6674003a00 msgr2=0x7f6674003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f6691ed2700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6674003a00 0x7f6674003e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6680009890 tx=0x7f6674005880 comp rx=0 tx=0).stop 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f6691ed2700 1 -- 172.21.15.97:0/3585800061 shutdown_connections 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f6691ed2700 1 --2- 172.21.15.97:0/3585800061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6674003a00 0x7f6674003e20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f6691ed2700 1 -- 172.21.15.97:0/3585800061 >> 172.21.15.97:0/3585800061 conn(0x7f668c0b3780 msgr2=0x7f668c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f6691ed2700 1 -- 172.21.15.97:0/3585800061 shutdown_connections 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.053+0000 7f6691ed2700 1 -- 172.21.15.97:0/3585800061 wait complete. 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.054+0000 7f6691ed2700 1 Processor -- start 2023-11-24T06:11:01.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.055+0000 7f6691ed2700 1 -- start start 2023-11-24T06:11:01.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.055+0000 7f6691ed2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674003a00 0x7f668c0b7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.055+0000 7f6691ed2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f668c1f70a0 0x7f668c1f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.055+0000 7f6691ed2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c06b700 0x7f668c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.055+0000 7f6691ed2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1f1520 con 0x7f668c06b700 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.055+0000 7f6691ed2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1f7610 con 0x7f6674003a00 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.055+0000 7f6691ed2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c1f78f0 con 0x7f668c1f70a0 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c06b700 0x7f668c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c06b700 0x7f668c0ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48942/0 (socket says 172.21.15.97:48942) 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 -- 172.21.15.97:0/851377446 learned_addr learned my addr 172.21.15.97:0/851377446 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668b7fe700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674003a00 0x7f668c0b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668affd700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f668c1f70a0 0x7f668c1f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 -- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f668c1f70a0 msgr2=0x7f668c1f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f668c1f70a0 0x7f668c1f0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 -- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674003a00 msgr2=0x7f668c0b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674003a00 0x7f668c0b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 -- 172.21.15.97:0/851377446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c1f7bd0 con 0x7f668c06b700 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668b7fe700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674003a00 0x7f668c0b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668affd700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f668c1f70a0 0x7f668c1f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f668bfff700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c06b700 0x7f668c0ccc00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f668c0cc190 tx=0x7f668400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f6673fff700 1 -- 172.21.15.97:0/851377446 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684003c10 con 0x7f668c06b700 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f6673fff700 1 -- 172.21.15.97:0/851377446 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6684003d70 con 0x7f668c06b700 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c1f7eb0 con 0x7f668c06b700 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f6673fff700 1 -- 172.21.15.97:0/851377446 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684005310 con 0x7f668c06b700 2023-11-24T06:11:01.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.056+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c06b1f0 con 0x7f668c06b700 2023-11-24T06:11:01.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.057+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c1f3e50 con 0x7f668c06b700 2023-11-24T06:11:01.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.058+0000 7f6673fff700 1 -- 172.21.15.97:0/851377446 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6684005470 con 0x7f668c06b700 2023-11-24T06:11:01.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.058+0000 7f6673fff700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f667806b110 0x7f667806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.058+0000 7f6673fff700 1 -- 172.21.15.97:0/851377446 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6684089670 con 0x7f668c06b700 2023-11-24T06:11:01.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.058+0000 7f668b7fe700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f667806b110 0x7f667806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.058+0000 7f668b7fe700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f667806b110 0x7f667806d5d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6680016040 tx=0x7f668000c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.062+0000 7f6673fff700 1 -- 172.21.15.97:0/851377446 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6684054710 con 0x7f668c06b700 2023-11-24T06:11:01.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.257+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f668c1f3c90 con 0x7f668c06b700 2023-11-24T06:11:01.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.257+0000 7f6673fff700 1 -- 172.21.15.97:0/851377446 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6684057d30 con 0x7f668c06b700 2023-11-24T06:11:01.260 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:01.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f667806b110 msgr2=0x7f667806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f667806b110 0x7f667806d5d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6680016040 tx=0x7f668000c000 comp rx=0 tx=0).stop 2023-11-24T06:11:01.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c06b700 msgr2=0x7f668c0ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c06b700 0x7f668c0ccc00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f668c0cc190 tx=0x7f668400ee00 comp rx=0 tx=0).stop 2023-11-24T06:11:01.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 shutdown_connections 2023-11-24T06:11:01.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c06b700 0x7f668c0ccc00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6674003a00 0x7f668c0b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f667806b110 0x7f667806d5d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 --2- 172.21.15.97:0/851377446 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f668c1f70a0 0x7f668c1f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 >> 172.21.15.97:0/851377446 conn(0x7f668c0b3780 msgr2=0x7f668c15b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 shutdown_connections 2023-11-24T06:11:01.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.259+0000 7f6691ed2700 1 -- 172.21.15.97:0/851377446 wait complete. 2023-11-24T06:11:01.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:11:01.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.391+0000 7f324cc3a700 1 Processor -- start 2023-11-24T06:11:01.394 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.391+0000 7f324cc3a700 1 -- start start 2023-11-24T06:11:01.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f324cc3a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f324815e2a0 0x7f324806ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f324cc3a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f32480ce090 0x7f324806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f324cc3a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324806e7f0 con 0x7f3248073f90 2023-11-24T06:11:01.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f324cc3a700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324806dec0 con 0x7f324815e2a0 2023-11-24T06:11:01.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f324cc3a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324806e1a0 con 0x7f32480ce090 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f324659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f324815e2a0 0x7f324806ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f324659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f324815e2a0 0x7f324806ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48948/0 (socket says 172.21.15.97:48948) 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f324659c700 1 -- 172.21.15.97:0/3188648162 learned_addr learned my addr 172.21.15.97:0/3188648162 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.392+0000 7f3245d9b700 1 --2- 172.21.15.97:0/3188648162 >> v2:172.21.15.183:3300/0 conn(0x7f32480ce090 0x7f324806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324659c700 1 -- 172.21.15.97:0/3188648162 >> v2:172.21.15.183:3300/0 conn(0x7f32480ce090 msgr2=0x7f324806ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324659c700 1 --2- 172.21.15.97:0/3188648162 >> v2:172.21.15.183:3300/0 conn(0x7f32480ce090 0x7f324806ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324659c700 1 -- 172.21.15.97:0/3188648162 >> v1:172.21.15.183:6789/0 conn(0x7f3248073f90 legacy=0x7f32480cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324659c700 1 -- 172.21.15.97:0/3188648162 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324806e480 con 0x7f324815e2a0 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324659c700 1 --2- 172.21.15.97:0/3188648162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f324815e2a0 0x7f324806ba70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3248071840 tx=0x7f323c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be26510c929fcb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324559a700 1 -- 172.21.15.97:0/3188648162 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c004230 con 0x7f324815e2a0 2023-11-24T06:11:01.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324559a700 1 -- 172.21.15.97:0/3188648162 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c004390 con 0x7f324815e2a0 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324559a700 1 -- 172.21.15.97:0/3188648162 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c0036a0 con 0x7f324815e2a0 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324cc3a700 1 -- 172.21.15.97:0/3188648162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f324815e2a0 msgr2=0x7f324806ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3188648162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f324815e2a0 0x7f324806ba70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3248071840 tx=0x7f323c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324cc3a700 1 -- 172.21.15.97:0/3188648162 shutdown_connections 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3188648162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f324815e2a0 0x7f324806ba70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3188648162 >> v2:172.21.15.183:3300/0 conn(0x7f32480ce090 0x7f324806ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.393+0000 7f324cc3a700 1 -- 172.21.15.97:0/3188648162 >> 172.21.15.97:0/3188648162 conn(0x7f32480cd570 msgr2=0x7f32480cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.394+0000 7f324cc3a700 1 -- 172.21.15.97:0/3188648162 shutdown_connections 2023-11-24T06:11:01.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.394+0000 7f324cc3a700 1 -- 172.21.15.97:0/3188648162 wait complete. 2023-11-24T06:11:01.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.395+0000 7f324cc3a700 1 Processor -- start 2023-11-24T06:11:01.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.395+0000 7f324cc3a700 1 -- start start 2023-11-24T06:11:01.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324cc3a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3248073f90 0x7f32481f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324cc3a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32480ce090 0x7f32481f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324cc3a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f324815e2a0 0x7f32480cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324cc3a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324815f9a0 con 0x7f3248073f90 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324cc3a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32481f7570 con 0x7f324815e2a0 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324cc3a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32481f7840 con 0x7f32480ce090 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3248073f90 0x7f32481f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3248073f90 0x7f32481f4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48952/0 (socket says 172.21.15.97:48952) 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 -- 172.21.15.97:0/3704248997 learned_addr learned my addr 172.21.15.97:0/3704248997 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f3246d9d700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f324815e2a0 0x7f32480cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f3245d9b700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32480ce090 0x7f32481f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 -- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32480ce090 msgr2=0x7f32481f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32480ce090 0x7f32481f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 -- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f324815e2a0 msgr2=0x7f32480cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f324815e2a0 0x7f32480cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 -- 172.21.15.97:0/3704248997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c009550 con 0x7f3248073f90 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f3246d9d700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f324815e2a0 0x7f32480cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f3245d9b700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32480ce090 0x7f32481f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324659c700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3248073f90 0x7f32481f4690 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f323c009130 tx=0x7f323c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f3236ffd700 1 -- 172.21.15.97:0/3704248997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c011630 con 0x7f3248073f90 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f3236ffd700 1 -- 172.21.15.97:0/3704248997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c011790 con 0x7f3248073f90 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32481f7b20 con 0x7f3248073f90 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f3236ffd700 1 -- 172.21.15.97:0/3704248997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c011b40 con 0x7f3248073f90 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.396+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324815fbf0 con 0x7f3248073f90 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.397+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324815fd80 con 0x7f3248073f90 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.398+0000 7f3236ffd700 1 -- 172.21.15.97:0/3704248997 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f323c011ca0 con 0x7f3248073f90 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.398+0000 7f3236ffd700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f322806b110 0x7f322806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.398+0000 7f3236ffd700 1 -- 172.21.15.97:0/3704248997 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f323c08ac20 con 0x7f3248073f90 2023-11-24T06:11:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.398+0000 7f3245d9b700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f322806b110 0x7f322806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.399+0000 7f3245d9b700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f322806b110 0x7f322806d5d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f324815b690 tx=0x7f3238008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.403+0000 7f3236ffd700 1 -- 172.21.15.97:0/3704248997 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f323c055d80 con 0x7f3248073f90 2023-11-24T06:11:01.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.592+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f32480af880 con 0x7f3248073f90 2023-11-24T06:11:01.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.592+0000 7f3236ffd700 1 -- 172.21.15.97:0/3704248997 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+65 (secure 0 0 0) 0x7f323c0593a0 con 0x7f3248073f90 2023-11-24T06:11:01.595 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:11:01.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:01.596 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-11-24T06:11:01.596 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-11-24T06:11:01.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:11:01.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f322806b110 msgr2=0x7f322806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f322806b110 0x7f322806d5d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f324815b690 tx=0x7f3238008040 comp rx=0 tx=0).stop 2023-11-24T06:11:01.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3248073f90 msgr2=0x7f32481f4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3248073f90 0x7f32481f4690 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f323c009130 tx=0x7f323c004750 comp rx=0 tx=0).stop 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 shutdown_connections 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f324815e2a0 0x7f32480cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3248073f90 0x7f32481f4690 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f322806b110 0x7f322806d5d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 --2- 172.21.15.97:0/3704248997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f32480ce090 0x7f32481f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.594+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 >> 172.21.15.97:0/3704248997 conn(0x7f32480cd570 msgr2=0x7f32480b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.595+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 shutdown_connections 2023-11-24T06:11:01.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.595+0000 7f324cc3a700 1 -- 172.21.15.97:0/3704248997 wait complete. 2023-11-24T06:11:01.609 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:11:01.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:11:01.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.731+0000 7f50acd25700 1 Processor -- start 2023-11-24T06:11:01.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.731+0000 7f50acd25700 1 -- start start 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50acd25700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f50a80746a0 0x7f50a806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50acd25700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50a80b59c0 0x7f50a80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50acd25700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a80b5e50 con 0x7f50a80b01e0 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50acd25700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a81470a0 con 0x7f50a80b59c0 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50acd25700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a80b05c0 con 0x7f50a80746a0 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a6d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50a80b59c0 0x7f50a80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f50a80b01e0 0x7f50a8073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41412/0 (socket says 172.21.15.97:41412) 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a659c700 1 -- 172.21.15.97:0/4155679653 learned_addr learned my addr 172.21.15.97:0/4155679653 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:01.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a5d9b700 1 --2- 172.21.15.97:0/4155679653 >> v2:172.21.15.183:3300/0 conn(0x7f50a80746a0 0x7f50a806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a6d9d700 1 -- 172.21.15.97:0/4155679653 >> v2:172.21.15.183:3300/0 conn(0x7f50a80746a0 msgr2=0x7f50a806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a6d9d700 1 --2- 172.21.15.97:0/4155679653 >> v2:172.21.15.183:3300/0 conn(0x7f50a80746a0 0x7f50a806e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a6d9d700 1 -- 172.21.15.97:0/4155679653 >> v1:172.21.15.97:6789/0 conn(0x7f50a80b01e0 legacy=0x7f50a8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a6d9d700 1 -- 172.21.15.97:0/4155679653 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a80b4250 con 0x7f50a80b59c0 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a559a700 1 -- 172.21.15.97:0/4155679653 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353469898 0 0) 0x7f5098001110 con 0x7f50a80b01e0 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a6d9d700 1 --2- 172.21.15.97:0/4155679653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a80b3b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f50a80d1970 tx=0x7f50a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e801826fd63a345 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a559a700 1 -- 172.21.15.97:0/4155679653 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a000e070 con 0x7f50a80b59c0 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a559a700 1 -- 172.21.15.97:0/4155679653 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a000bbd0 con 0x7f50a80b59c0 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.732+0000 7f50a559a700 1 -- 172.21.15.97:0/4155679653 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0012430 con 0x7f50a80b59c0 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.733+0000 7f50acd25700 1 -- 172.21.15.97:0/4155679653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 msgr2=0x7f50a80b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.733+0000 7f50acd25700 1 --2- 172.21.15.97:0/4155679653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a80b3b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f50a80d1970 tx=0x7f50a000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:01.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.733+0000 7f50acd25700 1 -- 172.21.15.97:0/4155679653 shutdown_connections 2023-11-24T06:11:01.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.733+0000 7f50acd25700 1 --2- 172.21.15.97:0/4155679653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a80b3b50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.733+0000 7f50acd25700 1 --2- 172.21.15.97:0/4155679653 >> v2:172.21.15.183:3300/0 conn(0x7f50a80746a0 0x7f50a806e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.733+0000 7f50acd25700 1 -- 172.21.15.97:0/4155679653 >> 172.21.15.97:0/4155679653 conn(0x7f50a80d0b40 msgr2=0x7f50a80d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:01.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.733+0000 7f50acd25700 1 -- 172.21.15.97:0/4155679653 shutdown_connections 2023-11-24T06:11:01.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.733+0000 7f50acd25700 1 -- 172.21.15.97:0/4155679653 wait complete. 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.734+0000 7f50acd25700 1 Processor -- start 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.734+0000 7f50acd25700 1 -- start start 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50acd25700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a80746a0 0x7f50a815f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50acd25700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a80b01e0 0x7f50a81e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50acd25700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a8142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50acd25700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81e7e30 con 0x7f50a80b59c0 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50acd25700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81e8060 con 0x7f50a80746a0 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50acd25700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a81e8340 con 0x7f50a80b01e0 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50a6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a8142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50a6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a8142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48968/0 (socket says 172.21.15.97:48968) 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50a6d9d700 1 -- 172.21.15.97:0/2265001962 learned_addr learned my addr 172.21.15.97:0/2265001962 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.735+0000 7f50a659c700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a80746a0 0x7f50a815f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50a5d9b700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a80b01e0 0x7f50a81e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50a6d9d700 1 -- 172.21.15.97:0/2265001962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a80b01e0 msgr2=0x7f50a81e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50a6d9d700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a80b01e0 0x7f50a81e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50a6d9d700 1 -- 172.21.15.97:0/2265001962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a80746a0 msgr2=0x7f50a815f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50a6d9d700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a80746a0 0x7f50a815f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50a6d9d700 1 -- 172.21.15.97:0/2265001962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a0009a70 con 0x7f50a80b59c0 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50a6d9d700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a8142440 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f50a0014040 tx=0x7f50a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f5096ffd700 1 -- 172.21.15.97:0/2265001962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0012430 con 0x7f50a80b59c0 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f5096ffd700 1 -- 172.21.15.97:0/2265001962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a0020070 con 0x7f50a80b59c0 2023-11-24T06:11:01.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a81e8620 con 0x7f50a80b59c0 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f5096ffd700 1 -- 172.21.15.97:0/2265001962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a0004280 con 0x7f50a80b59c0 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.736+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a815c0a0 con 0x7f50a80b59c0 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.737+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a815c230 con 0x7f50a80b59c0 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.737+0000 7f5096ffd700 1 -- 172.21.15.97:0/2265001962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f50a00043e0 con 0x7f50a80b59c0 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.738+0000 7f5096ffd700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f508806b110 0x7f508806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.738+0000 7f5096ffd700 1 -- 172.21.15.97:0/2265001962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f50a0022070 con 0x7f50a80b59c0 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.738+0000 7f50a659c700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f508806b110 0x7f508806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.738+0000 7f50a659c700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f508806b110 0x7f508806d5d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5098000f60 tx=0x7f5098007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:01.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.742+0000 7f5096ffd700 1 -- 172.21.15.97:0/2265001962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50a0056940 con 0x7f50a80b59c0 2023-11-24T06:11:01.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:01.931+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f50a8067f60 con 0x7f50a80b59c0 2023-11-24T06:11:02.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.158+0000 7f5096ffd700 1 -- 172.21.15.97:0/2265001962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7f50a001ae60 con 0x7f50a80b59c0 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f508806b110 msgr2=0x7f508806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f508806b110 0x7f508806d5d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5098000f60 tx=0x7f5098007a40 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 msgr2=0x7f50a8142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a8142440 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f50a0014040 tx=0x7f50a0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 shutdown_connections 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50a80b59c0 0x7f50a8142440 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50a80746a0 0x7f50a815f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f508806b110 0x7f508806d5d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 --2- 172.21.15.97:0/2265001962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f50a80b01e0 0x7f50a81e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 >> 172.21.15.97:0/2265001962 conn(0x7f50a80d0b40 msgr2=0x7f50a80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:02.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 shutdown_connections 2023-11-24T06:11:02.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.160+0000 7f50acd25700 1 -- 172.21.15.97:0/2265001962 wait complete. 2023-11-24T06:11:02.164 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-11-24T06:11:02.178 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:11:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.303+0000 7f2ead148700 1 Processor -- start 2023-11-24T06:11:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.304+0000 7f2ead148700 1 -- start start 2023-11-24T06:11:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.304+0000 7f2ead148700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2ea815b370 0x7f2ea815b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.304+0000 7f2ead148700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 0x7f2ea80b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.304+0000 7f2ead148700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80ae5e0 con 0x7f2ea81587f0 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.304+0000 7f2ead148700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea80b4630 con 0x7f2ea815b370 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.304+0000 7f2ead148700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea815be30 con 0x7f2ea80b3c60 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.304+0000 7f2ea6d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2ea81587f0 0x7f2ea806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41426/0 (socket says 172.21.15.97:41426) 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.304+0000 7f2ea6d9d700 1 -- 172.21.15.97:0/3115629946 learned_addr learned my addr 172.21.15.97:0/3115629946 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785966243 0 0) 0x7f2ea80ae5e0 con 0x7f2ea81587f0 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e900035b0 con 0x7f2ea81587f0 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea659c700 1 --2- 172.21.15.97:0/3115629946 >> v2:172.21.15.183:3300/0 conn(0x7f2ea815b370 0x7f2ea815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea759e700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 0x7f2ea80b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2271233473 0 0) 0x7f2e900035b0 con 0x7f2ea81587f0 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 msgr2=0x7f2ea80b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 0x7f2ea80b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 >> v2:172.21.15.183:3300/0 conn(0x7f2ea815b370 msgr2=0x7f2ea815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 --2- 172.21.15.97:0/3115629946 >> v2:172.21.15.183:3300/0 conn(0x7f2ea815b370 0x7f2ea815b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea815c110 con 0x7f2ea81587f0 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea759e700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 0x7f2ea80b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f2ea00019f0 con 0x7f2ea81587f0 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 >> v1:172.21.15.97:6789/0 conn(0x7f2ea81587f0 legacy=0x7f2ea806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90003a00 0x7f2e90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 0x7f2e90004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e90004d90 0x7f2e90005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80b4630 con 0x7f2e90003a00 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea815be30 con 0x7f2e90004d90 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9000dc20 con 0x7f2ea80b3c60 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea6d9d700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90003a00 0x7f2e90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea759e700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e90004d90 0x7f2e90005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea659c700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 0x7f2e90004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea6d9d700 1 -- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 msgr2=0x7f2e90004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea6d9d700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 0x7f2e90004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea6d9d700 1 -- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e90004d90 msgr2=0x7f2e90005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea6d9d700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e90004d90 0x7f2e90005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea6d9d700 1 -- 172.21.15.97:0/3115629946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e900035b0 con 0x7f2e90003a00 2023-11-24T06:11:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.305+0000 7f2ea659c700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea80b3c60 0x7f2e90004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ea6d9d700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90003a00 0x7f2e90003e20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000a830 tx=0x7f2e90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a6d378c87067f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000a2f0 con 0x7f2e90003a00 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea0012620 con 0x7f2e90003a00 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ea5d9b700 1 -- 172.21.15.97:0/3115629946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea00129d0 con 0x7f2e90003a00 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ead148700 1 -- 172.21.15.97:0/3115629946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90003a00 msgr2=0x7f2e90003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ead148700 1 --2- 172.21.15.97:0/3115629946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90003a00 0x7f2e90003e20 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000a830 tx=0x7f2e90005880 comp rx=0 tx=0).stop 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ea759e700 1 -- 172.21.15.97:0/3115629946 reap_dead start 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ead148700 1 -- 172.21.15.97:0/3115629946 shutdown_connections 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ead148700 1 -- 172.21.15.97:0/3115629946 >> 172.21.15.97:0/3115629946 conn(0x7f2ea80d0bf0 msgr2=0x7f2ea80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ead148700 1 -- 172.21.15.97:0/3115629946 shutdown_connections 2023-11-24T06:11:02.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.306+0000 7f2ead148700 1 -- 172.21.15.97:0/3115629946 wait complete. 2023-11-24T06:11:02.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.308+0000 7f2ead148700 1 Processor -- start 2023-11-24T06:11:02.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.308+0000 7f2ead148700 1 -- start start 2023-11-24T06:11:02.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ead148700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e90003a00 0x7f2ea809dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ead148700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ea8154240 0x7f2ea80b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ead148700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea8155290 0x7f2ea8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ead148700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea8155c50 con 0x7f2ea8155290 2023-11-24T06:11:02.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ead148700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80b14a0 con 0x7f2ea8154240 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ead148700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80b1780 con 0x7f2e90003a00 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea8155290 0x7f2ea8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea8155290 0x7f2ea8155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48990/0 (socket says 172.21.15.97:48990) 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 -- 172.21.15.97:0/4077183945 learned_addr learned my addr 172.21.15.97:0/4077183945 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea659c700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ea8154240 0x7f2ea80b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea6d9d700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e90003a00 0x7f2ea809dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 -- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e90003a00 msgr2=0x7f2ea809dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e90003a00 0x7f2ea809dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 -- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ea8154240 msgr2=0x7f2ea80b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ea8154240 0x7f2ea80b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 -- 172.21.15.97:0/4077183945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80b1a60 con 0x7f2ea8155290 2023-11-24T06:11:02.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea6d9d700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e90003a00 0x7f2ea809dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ea759e700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea8155290 0x7f2ea8155710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80b4a50 tx=0x7f2e98009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2e8f7fe700 1 -- 172.21.15.97:0/4077183945 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98014610 con 0x7f2ea8155290 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2e8f7fe700 1 -- 172.21.15.97:0/4077183945 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e98014770 con 0x7f2ea8155290 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea80b1d40 con 0x7f2ea8155290 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2e8f7fe700 1 -- 172.21.15.97:0/4077183945 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98014b60 con 0x7f2ea8155290 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.309+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea80b1190 con 0x7f2ea8155290 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.310+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea809ad70 con 0x7f2ea8155290 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.311+0000 7f2e8f7fe700 1 -- 172.21.15.97:0/4077183945 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2e98014cc0 con 0x7f2ea8155290 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.311+0000 7f2e8f7fe700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e9406b110 0x7f2e9406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.311+0000 7f2e8f7fe700 1 -- 172.21.15.97:0/4077183945 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2e98089750 con 0x7f2ea8155290 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.311+0000 7f2ea6d9d700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e9406b110 0x7f2e9406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:02.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.312+0000 7f2ea6d9d700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e9406b110 0x7f2e9406d5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000afd0 tx=0x7f2ea0007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:02.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.316+0000 7f2e8f7fe700 1 -- 172.21.15.97:0/4077183945 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e98054690 con 0x7f2ea8155290 2023-11-24T06:11:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:02.507+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2ea80b15e0 con 0x7f2ea8155290 2023-11-24T06:11:03.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.164+0000 7f2e8f7fe700 1 -- 172.21.15.97:0/4077183945 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2ea80b15e0 con 0x7f2ea8155290 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.167+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e9406b110 msgr2=0x7f2e9406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.167+0000 7f2ead148700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e9406b110 0x7f2e9406d5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000afd0 tx=0x7f2ea0007000 comp rx=0 tx=0).stop 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.167+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea8155290 msgr2=0x7f2ea8155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.167+0000 7f2ead148700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea8155290 0x7f2ea8155710 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80b4a50 tx=0x7f2e98009860 comp rx=0 tx=0).stop 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.168+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 shutdown_connections 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.168+0000 7f2ead148700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea8155290 0x7f2ea8155710 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.168+0000 7f2ead148700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e9406b110 0x7f2e9406d5d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.168+0000 7f2ead148700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2e90003a00 0x7f2ea809dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.168+0000 7f2ead148700 1 --2- 172.21.15.97:0/4077183945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ea8154240 0x7f2ea80b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.168+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 >> 172.21.15.97:0/4077183945 conn(0x7f2ea80d0bf0 msgr2=0x7f2ea8156910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.168+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 shutdown_connections 2023-11-24T06:11:03.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.168+0000 7f2ead148700 1 -- 172.21.15.97:0/4077183945 wait complete. 2023-11-24T06:11:03.180 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-11-24T06:11:03.198 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:11:03.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.324+0000 7fa14a2bb700 1 Processor -- start 2023-11-24T06:11:03.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.324+0000 7fa14a2bb700 1 -- start start 2023-11-24T06:11:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa14a2bb700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1440b5da0 0x7fa1440afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa14a2bb700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1440b3c60 con 0x7fa1440746a0 2023-11-24T06:11:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa14a2bb700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1440b58c0 con 0x7fa14415e160 2023-11-24T06:11:03.329 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa14a2bb700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1440b4500 con 0x7fa1440b5da0 2023-11-24T06:11:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa1437fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa1440746a0 0x7fa14406d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41448/0 (socket says 172.21.15.97:41448) 2023-11-24T06:11:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa1437fe700 1 -- 172.21.15.97:0/737855065 learned_addr learned my addr 172.21.15.97:0/737855065 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa148858700 1 --2- 172.21.15.97:0/737855065 >> v2:172.21.15.183:3300/0 conn(0x7fa1440b5da0 0x7fa1440afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa142ffd700 1 -- 172.21.15.97:0/737855065 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992380359 0 0) 0x7fa1440b3c60 con 0x7fa1440746a0 2023-11-24T06:11:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa142ffd700 1 -- 172.21.15.97:0/737855065 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa12c0035b0 con 0x7fa1440746a0 2023-11-24T06:11:03.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa142ffd700 1 -- 172.21.15.97:0/737855065 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1497663001 0 0) 0x7fa12c0035b0 con 0x7fa1440746a0 2023-11-24T06:11:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa148858700 1 -- 172.21.15.97:0/737855065 >> v1:172.21.15.183:6789/0 conn(0x7fa14415e160 legacy=0x7fa144073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa148858700 1 -- 172.21.15.97:0/737855065 >> v1:172.21.15.97:6789/0 conn(0x7fa1440746a0 legacy=0x7fa14406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.325+0000 7fa148858700 1 -- 172.21.15.97:0/737855065 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14406bdc0 con 0x7fa1440b5da0 2023-11-24T06:11:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa148858700 1 --2- 172.21.15.97:0/737855065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 0x7fa1440afc90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009d90 tx=0x7fa13c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a42613a2f2c729 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa142ffd700 1 -- 172.21.15.97:0/737855065 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00e070 con 0x7fa1440b5da0 2023-11-24T06:11:03.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa142ffd700 1 -- 172.21.15.97:0/737855065 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c003a20 con 0x7fa1440b5da0 2023-11-24T06:11:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa142ffd700 1 -- 172.21.15.97:0/737855065 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138001670 con 0x7fa1440b5da0 2023-11-24T06:11:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa14a2bb700 1 -- 172.21.15.97:0/737855065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 msgr2=0x7fa1440afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa14a2bb700 1 --2- 172.21.15.97:0/737855065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 0x7fa1440afc90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009d90 tx=0x7fa13c00b330 comp rx=0 tx=0).stop 2023-11-24T06:11:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa14a2bb700 1 -- 172.21.15.97:0/737855065 shutdown_connections 2023-11-24T06:11:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa14a2bb700 1 --2- 172.21.15.97:0/737855065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 0x7fa1440afc90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa14a2bb700 1 -- 172.21.15.97:0/737855065 >> 172.21.15.97:0/737855065 conn(0x7fa1440d0bf0 msgr2=0x7fa1440d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa14a2bb700 1 -- 172.21.15.97:0/737855065 shutdown_connections 2023-11-24T06:11:03.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.326+0000 7fa14a2bb700 1 -- 172.21.15.97:0/737855065 wait complete. 2023-11-24T06:11:03.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.327+0000 7fa14a2bb700 1 Processor -- start 2023-11-24T06:11:03.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.328+0000 7fa14a2bb700 1 -- start start 2023-11-24T06:11:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.328+0000 7fa14a2bb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1440746a0 0x7fa14406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.328+0000 7fa14a2bb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 0x7fa14406de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.328+0000 7fa14a2bb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa14415e160 0x7fa144160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.328+0000 7fa14a2bb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa144153c20 con 0x7fa14415e160 2023-11-24T06:11:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.328+0000 7fa14a2bb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441528b0 con 0x7fa1440b5da0 2023-11-24T06:11:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.328+0000 7fa14a2bb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1441532c0 con 0x7fa1440746a0 2023-11-24T06:11:03.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.328+0000 7fa148858700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa14415e160 0x7fa144160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa148858700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa14415e160 0x7fa144160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49004/0 (socket says 172.21.15.97:49004) 2023-11-24T06:11:03.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa148858700 1 -- 172.21.15.97:0/1312867437 learned_addr learned my addr 172.21.15.97:0/1312867437 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:03.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa1437fe700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 0x7fa14406de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa143fff700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1440746a0 0x7fa14406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa148858700 1 -- 172.21.15.97:0/1312867437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1440746a0 msgr2=0x7fa14406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa148858700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1440746a0 0x7fa14406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa148858700 1 -- 172.21.15.97:0/1312867437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 msgr2=0x7fa14406de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa148858700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 0x7fa14406de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa148858700 1 -- 172.21.15.97:0/1312867437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c009a70 con 0x7fa14415e160 2023-11-24T06:11:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa148858700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa14415e160 0x7fa144160820 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa144072ed0 tx=0x7fa13c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa140ff9700 1 -- 172.21.15.97:0/1312867437 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c004540 con 0x7fa14415e160 2023-11-24T06:11:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa140ff9700 1 -- 172.21.15.97:0/1312867437 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c0046a0 con 0x7fa14415e160 2023-11-24T06:11:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa144153560 con 0x7fa14415e160 2023-11-24T06:11:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa140ff9700 1 -- 172.21.15.97:0/1312867437 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c01d470 con 0x7fa14415e160 2023-11-24T06:11:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.329+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa144161430 con 0x7fa14415e160 2023-11-24T06:11:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.330+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa144068fb0 con 0x7fa14415e160 2023-11-24T06:11:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.331+0000 7fa140ff9700 1 -- 172.21.15.97:0/1312867437 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa13c01d5d0 con 0x7fa14415e160 2023-11-24T06:11:03.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.331+0000 7fa140ff9700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa13006b110 0x7fa13006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.331+0000 7fa140ff9700 1 -- 172.21.15.97:0/1312867437 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa13c08b270 con 0x7fa14415e160 2023-11-24T06:11:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.331+0000 7fa143fff700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa13006b110 0x7fa13006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.332+0000 7fa143fff700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa13006b110 0x7fa13006d5d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa134007fd0 tx=0x7fa134009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:03.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.336+0000 7fa140ff9700 1 -- 172.21.15.97:0/1312867437 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa13c055f00 con 0x7fa14415e160 2023-11-24T06:11:03.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:03.570+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa14406c140 con 0x7fa14415e160 2023-11-24T06:11:04.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.166+0000 7fa140ff9700 1 -- 172.21.15.97:0/1312867437 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa13c059520 con 0x7fa14415e160 2023-11-24T06:11:04.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa13006b110 msgr2=0x7fa13006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa13006b110 0x7fa13006d5d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa134007fd0 tx=0x7fa134009040 comp rx=0 tx=0).stop 2023-11-24T06:11:04.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa14415e160 msgr2=0x7fa144160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa14415e160 0x7fa144160820 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa144072ed0 tx=0x7fa13c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 shutdown_connections 2023-11-24T06:11:04.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa14415e160 0x7fa144160820 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa13006b110 0x7fa13006d5d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1440746a0 0x7fa14406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 --2- 172.21.15.97:0/1312867437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1440b5da0 0x7fa14406de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.170+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 >> 172.21.15.97:0/1312867437 conn(0x7fa1440d0bf0 msgr2=0x7fa1440d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.171+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 shutdown_connections 2023-11-24T06:11:04.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.171+0000 7fa14a2bb700 1 -- 172.21.15.97:0/1312867437 wait complete. 2023-11-24T06:11:04.184 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 8 and data pool 9 2023-11-24T06:11:04.207 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:04.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.328+0000 7f2c961e1700 1 Processor -- start 2023-11-24T06:11:04.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.328+0000 7f2c961e1700 1 -- start start 2023-11-24T06:11:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.328+0000 7f2c961e1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c9015e080 0x7f2c900b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c961e1700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2c9006c710 0x7f2c900afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c961e1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c900ae430 con 0x7f2c900af4d0 2023-11-24T06:11:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c961e1700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c900af0b0 con 0x7f2c9006c710 2023-11-24T06:11:04.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c961e1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900b05a0 con 0x7f2c9015e080 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8ffff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2c9006c710 0x7f2c900afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8f7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c9015e080 0x7f2c900b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8effd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f2c900af4d0 0x7f2c9006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:50794/0 (socket says 172.21.15.97:50794) 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8ffff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2c9006c710 0x7f2c900afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:43184/0 (socket says 172.21.15.97:43184) 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8effd700 1 -- 172.21.15.97:0/3445149221 learned_addr learned my addr 172.21.15.97:0/3445149221 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8f7fe700 1 -- 172.21.15.97:0/3445149221 >> v2:172.21.15.183:3300/0 conn(0x7f2c9006c710 msgr2=0x7f2c900afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8f7fe700 1 --2- 172.21.15.97:0/3445149221 >> v2:172.21.15.183:3300/0 conn(0x7f2c9006c710 0x7f2c900afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8f7fe700 1 -- 172.21.15.97:0/3445149221 >> v1:172.21.15.183:6789/0 conn(0x7f2c900af4d0 legacy=0x7f2c9006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8e7fc700 1 -- 172.21.15.97:0/3445149221 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618908496 0 0) 0x7f2c900ae430 con 0x7f2c900af4d0 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8f7fe700 1 -- 172.21.15.97:0/3445149221 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c900ae710 con 0x7f2c9015e080 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.329+0000 7f2c8ffff700 1 --2- 172.21.15.97:0/3445149221 >> v2:172.21.15.183:3300/0 conn(0x7f2c9006c710 0x7f2c900afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:04.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c8f7fe700 1 --2- 172.21.15.97:0/3445149221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c9015e080 0x7f2c900b5050 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009840 tx=0x7f2c80009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebcdb3a1ec6596d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c8e7fc700 1 -- 172.21.15.97:0/3445149221 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80011830 con 0x7f2c9015e080 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c8e7fc700 1 -- 172.21.15.97:0/3445149221 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80011990 con 0x7f2c9015e080 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c8e7fc700 1 -- 172.21.15.97:0/3445149221 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80011d20 con 0x7f2c9015e080 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c961e1700 1 -- 172.21.15.97:0/3445149221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c9015e080 msgr2=0x7f2c900b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c961e1700 1 --2- 172.21.15.97:0/3445149221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c9015e080 0x7f2c900b5050 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009840 tx=0x7f2c80009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c961e1700 1 -- 172.21.15.97:0/3445149221 shutdown_connections 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c961e1700 1 --2- 172.21.15.97:0/3445149221 >> v2:172.21.15.183:3300/0 conn(0x7f2c9006c710 0x7f2c900afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c961e1700 1 --2- 172.21.15.97:0/3445149221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c9015e080 0x7f2c900b5050 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c961e1700 1 -- 172.21.15.97:0/3445149221 >> 172.21.15.97:0/3445149221 conn(0x7f2c90071b90 msgr2=0x7f2c90071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c961e1700 1 -- 172.21.15.97:0/3445149221 shutdown_connections 2023-11-24T06:11:04.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.330+0000 7f2c961e1700 1 -- 172.21.15.97:0/3445149221 wait complete. 2023-11-24T06:11:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.332+0000 7f2c961e1700 1 Processor -- start 2023-11-24T06:11:04.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.332+0000 7f2c961e1700 1 -- start start 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.332+0000 7f2c961e1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9006c710 0x7f2c90153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.332+0000 7f2c961e1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c900af4d0 0x7f2c901574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c961e1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c9015e080 0x7f2c9006bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c961e1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c901537a0 con 0x7f2c9006c710 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c961e1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90158a40 con 0x7f2c9015e080 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c961e1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90157ec0 con 0x7f2c900af4d0 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9006c710 0x7f2c90153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9006c710 0x7f2c90153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49026/0 (socket says 172.21.15.97:49026) 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 -- 172.21.15.97:0/2311570444 learned_addr learned my addr 172.21.15.97:0/2311570444 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8ffff700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c9015e080 0x7f2c9006bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.339 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 -- 172.21.15.97:0/2311570444 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c900af4d0 msgr2=0x7f2c901574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c900af4d0 0x7f2c901574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 -- 172.21.15.97:0/2311570444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c9015e080 msgr2=0x7f2c9006bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c9015e080 0x7f2c9006bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 -- 172.21.15.97:0/2311570444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c88009a70 con 0x7f2c9006c710 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8ffff700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c9015e080 0x7f2c9006bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c8f7fe700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9006c710 0x7f2c90153260 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009fd0 tx=0x7f2c80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c77fff700 1 -- 172.21.15.97:0/2311570444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800036a0 con 0x7f2c9006c710 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c77fff700 1 -- 172.21.15.97:0/2311570444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8001e070 con 0x7f2c9006c710 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c800094a0 con 0x7f2c9006c710 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c77fff700 1 -- 172.21.15.97:0/2311570444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80003a40 con 0x7f2c9006c710 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.333+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c90158480 con 0x7f2c9006c710 2023-11-24T06:11:04.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.334+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c900753f0 con 0x7f2c9006c710 2023-11-24T06:11:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.335+0000 7f2c77fff700 1 -- 172.21.15.97:0/2311570444 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2c80003ba0 con 0x7f2c9006c710 2023-11-24T06:11:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.335+0000 7f2c77fff700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2c7c06b110 0x7f2c7c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.335+0000 7f2c77fff700 1 -- 172.21.15.97:0/2311570444 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c8008a920 con 0x7f2c9006c710 2023-11-24T06:11:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.335+0000 7f2c8effd700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2c7c06b110 0x7f2c7c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.336+0000 7f2c8effd700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2c7c06b110 0x7f2c7c06d5d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2c840011e0 tx=0x7f2c840079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.340+0000 7f2c77fff700 1 -- 172.21.15.97:0/2311570444 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c80055700 con 0x7f2c9006c710 2023-11-24T06:11:04.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.533+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c900b00d0 con 0x7f2c9006c710 2023-11-24T06:11:04.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.534+0000 7f2c77fff700 1 -- 172.21.15.97:0/2311570444 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16590 (secure 0 0 0) 0x7f2c900b00d0 con 0x7f2c9006c710 2023-11-24T06:11:04.537 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:04.537 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":59,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:03.571960+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:01.950100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:02.526161+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2c7c06b110 msgr2=0x7f2c7c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2c7c06b110 0x7f2c7c06d5d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2c840011e0 tx=0x7f2c840079b0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9006c710 msgr2=0x7f2c90153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9006c710 0x7f2c90153260 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009fd0 tx=0x7f2c80004750 comp rx=0 tx=0).stop 2023-11-24T06:11:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 shutdown_connections 2023-11-24T06:11:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c9015e080 0x7f2c9006bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9006c710 0x7f2c90153260 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2c7c06b110 0x7f2c7c06d5d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 --2- 172.21.15.97:0/2311570444 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2c900af4d0 0x7f2c901574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 >> 172.21.15.97:0/2311570444 conn(0x7f2c90071b90 msgr2=0x7f2c900b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 shutdown_connections 2023-11-24T06:11:04.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.536+0000 7f2c961e1700 1 -- 172.21.15.97:0/2311570444 wait complete. 2023-11-24T06:11:04.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:04.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.668+0000 7fcf70645700 1 Processor -- start 2023-11-24T06:11:04.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.669+0000 7fcf70645700 1 -- start start 2023-11-24T06:11:04.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.669+0000 7fcf70645700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf680c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.669+0000 7fcf70645700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcf68162a30 0x7fcf680c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.669+0000 7fcf70645700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf680cfff0 con 0x7fcf68158670 2023-11-24T06:11:04.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.669+0000 7fcf70645700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf680629b0 con 0x7fcf68162a30 2023-11-24T06:11:04.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.669+0000 7fcf70645700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680cfd10 con 0x7fcf68158a50 2023-11-24T06:11:04.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6ebe2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcf68162a30 0x7fcf680c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6e3e1700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fcf68158670 0x7fcf680622a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:50808/0 (socket says 172.21.15.97:50808) 2023-11-24T06:11:04.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6e3e1700 1 -- 172.21.15.97:0/1436108991 learned_addr learned my addr 172.21.15.97:0/1436108991 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:04.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6dbe0700 1 --2- 172.21.15.97:0/1436108991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf680c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6d3df700 1 -- 172.21.15.97:0/1436108991 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753930732 0 0) 0x7fcf680cfff0 con 0x7fcf68158670 2023-11-24T06:11:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6d3df700 1 -- 172.21.15.97:0/1436108991 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf540035b0 con 0x7fcf68158670 2023-11-24T06:11:04.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6dbe0700 1 -- 172.21.15.97:0/1436108991 >> v2:172.21.15.183:3300/0 conn(0x7fcf68162a30 msgr2=0x7fcf680c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6dbe0700 1 --2- 172.21.15.97:0/1436108991 >> v2:172.21.15.183:3300/0 conn(0x7fcf68162a30 0x7fcf680c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6dbe0700 1 -- 172.21.15.97:0/1436108991 >> v1:172.21.15.183:6789/0 conn(0x7fcf68158670 legacy=0x7fcf680622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6dbe0700 1 -- 172.21.15.97:0/1436108991 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf68161f50 con 0x7fcf68158a50 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.670+0000 7fcf6dbe0700 1 --2- 172.21.15.97:0/1436108991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf680c98b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009840 tx=0x7fcf64009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd21809d70a7d4b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf6d3df700 1 -- 172.21.15.97:0/1436108991 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64011830 con 0x7fcf68158a50 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf6d3df700 1 -- 172.21.15.97:0/1436108991 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64011990 con 0x7fcf68158a50 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf6d3df700 1 -- 172.21.15.97:0/1436108991 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64011d00 con 0x7fcf68158a50 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf70645700 1 -- 172.21.15.97:0/1436108991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 msgr2=0x7fcf680c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf70645700 1 --2- 172.21.15.97:0/1436108991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf680c98b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009840 tx=0x7fcf64009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf70645700 1 -- 172.21.15.97:0/1436108991 shutdown_connections 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf70645700 1 --2- 172.21.15.97:0/1436108991 >> v2:172.21.15.183:3300/0 conn(0x7fcf68162a30 0x7fcf680c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf70645700 1 --2- 172.21.15.97:0/1436108991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf680c98b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf70645700 1 -- 172.21.15.97:0/1436108991 >> 172.21.15.97:0/1436108991 conn(0x7fcf680b57b0 msgr2=0x7fcf680b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf70645700 1 -- 172.21.15.97:0/1436108991 shutdown_connections 2023-11-24T06:11:04.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.671+0000 7fcf70645700 1 -- 172.21.15.97:0/1436108991 wait complete. 2023-11-24T06:11:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.672+0000 7fcf70645700 1 Processor -- start 2023-11-24T06:11:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf70645700 1 -- start start 2023-11-24T06:11:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf70645700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf68158670 0x7fcf681f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf70645700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf6815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf70645700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf68162a30 0x7fcf6815f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf70645700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6814dfc0 con 0x7fcf68158670 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf70645700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68156fd0 con 0x7fcf68162a30 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf70645700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6815da90 con 0x7fcf68158a50 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf6e3e1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf68158670 0x7fcf681f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf6e3e1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf68158670 0x7fcf681f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49040/0 (socket says 172.21.15.97:49040) 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.673+0000 7fcf6e3e1700 1 -- 172.21.15.97:0/1111664762 learned_addr learned my addr 172.21.15.97:0/1111664762 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf6dbe0700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf6815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf6ebe2700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf68162a30 0x7fcf6815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf6e3e1700 1 -- 172.21.15.97:0/1111664762 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 msgr2=0x7fcf6815cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf6e3e1700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf6815cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf6e3e1700 1 -- 172.21.15.97:0/1111664762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf68162a30 msgr2=0x7fcf6815f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf6e3e1700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf68162a30 0x7fcf6815f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf6e3e1700 1 -- 172.21.15.97:0/1111664762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf640094a0 con 0x7fcf68158670 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf6e3e1700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf68158670 0x7fcf681f3370 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c4d0 tx=0x7fcf5800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf5effd700 1 -- 172.21.15.97:0/1111664762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58005d70 con 0x7fcf68158670 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf5effd700 1 -- 172.21.15.97:0/1111664762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf580070d0 con 0x7fcf68158670 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6815dd70 con 0x7fcf68158670 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf5effd700 1 -- 172.21.15.97:0/1111664762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58007440 con 0x7fcf68158670 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.674+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6815c330 con 0x7fcf68158670 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.675+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6815e290 con 0x7fcf68158670 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.675+0000 7fcf5effd700 1 -- 172.21.15.97:0/1111664762 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fcf580075a0 con 0x7fcf68158670 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.676+0000 7fcf5effd700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcf4c06b110 0x7fcf4c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:04.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.676+0000 7fcf5effd700 1 -- 172.21.15.97:0/1111664762 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf58088da0 con 0x7fcf68158670 2023-11-24T06:11:04.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.676+0000 7fcf6dbe0700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcf4c06b110 0x7fcf4c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:04.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.676+0000 7fcf6dbe0700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcf4c06b110 0x7fcf4c06d5d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcf680b1740 tx=0x7fcf64008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:04.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.680+0000 7fcf5effd700 1 -- 172.21.15.97:0/1111664762 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf6815e290 con 0x7fcf68158670 2023-11-24T06:11:04.873 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.870+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf6815e290 con 0x7fcf68158670 2023-11-24T06:11:04.874 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.871+0000 7fcf5effd700 1 -- 172.21.15.97:0/1111664762 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16590 (secure 0 0 0) 0x7fcf6815e290 con 0x7fcf68158670 2023-11-24T06:11:04.874 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:04.875 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":59,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:03.571960+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:01.950100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:02.526161+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:04.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcf4c06b110 msgr2=0x7fcf4c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcf4c06b110 0x7fcf4c06d5d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcf680b1740 tx=0x7fcf64008030 comp rx=0 tx=0).stop 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf68158670 msgr2=0x7fcf681f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf68158670 0x7fcf681f3370 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c4d0 tx=0x7fcf5800c890 comp rx=0 tx=0).stop 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 shutdown_connections 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf68162a30 0x7fcf6815f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf68158670 0x7fcf681f3370 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcf4c06b110 0x7fcf4c06d5d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 --2- 172.21.15.97:0/1111664762 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcf68158a50 0x7fcf6815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.873+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 >> 172.21.15.97:0/1111664762 conn(0x7fcf680b57b0 msgr2=0x7fcf6806e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.874+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 shutdown_connections 2023-11-24T06:11:04.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:04.874+0000 7fcf70645700 1 -- 172.21.15.97:0/1111664762 wait complete. 2023-11-24T06:11:04.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:11:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.009+0000 7fa66aab9700 1 Processor -- start 2023-11-24T06:11:05.012 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.009+0000 7fa66aab9700 1 -- start start 2023-11-24T06:11:05.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa66aab9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6640746a0 0x7fa6640b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa66aab9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6640b3c60 con 0x7fa6640afc90 2023-11-24T06:11:05.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa66aab9700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6640b58c0 con 0x7fa66415e160 2023-11-24T06:11:05.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa66aab9700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6640b4500 con 0x7fa6640746a0 2023-11-24T06:11:05.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa663fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6640746a0 0x7fa6640b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa663fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6640746a0 0x7fa6640b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49050/0 (socket says 172.21.15.97:49050) 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa663fff700 1 -- 172.21.15.97:0/427201393 learned_addr learned my addr 172.21.15.97:0/427201393 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa669056700 1 --1- 172.21.15.97:0/427201393 >> v1:172.21.15.97:6789/0 conn(0x7fa6640afc90 0x7fa66406d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41504/0 (socket says 172.21.15.97:41504) 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa663fff700 1 -- 172.21.15.97:0/427201393 >> v1:172.21.15.183:6789/0 conn(0x7fa66415e160 legacy=0x7fa664073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa663fff700 1 -- 172.21.15.97:0/427201393 >> v1:172.21.15.97:6789/0 conn(0x7fa6640afc90 legacy=0x7fa66406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa6637fe700 1 -- 172.21.15.97:0/427201393 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939547675 0 0) 0x7fa6640b3c60 con 0x7fa6640afc90 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.010+0000 7fa663fff700 1 -- 172.21.15.97:0/427201393 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66406bdc0 con 0x7fa6640746a0 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa663fff700 1 --2- 172.21.15.97:0/427201393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640746a0 0x7fa6640b5da0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa6500098f0 tx=0x7fa650009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa94332e450b6a1c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa6637fe700 1 -- 172.21.15.97:0/427201393 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65000b8a0 con 0x7fa6640746a0 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa6637fe700 1 -- 172.21.15.97:0/427201393 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65000ba00 con 0x7fa6640746a0 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa6637fe700 1 -- 172.21.15.97:0/427201393 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65000bd90 con 0x7fa6640746a0 2023-11-24T06:11:05.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa66aab9700 1 -- 172.21.15.97:0/427201393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640746a0 msgr2=0x7fa6640b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa66aab9700 1 --2- 172.21.15.97:0/427201393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640746a0 0x7fa6640b5da0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa6500098f0 tx=0x7fa650009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:05.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa66aab9700 1 -- 172.21.15.97:0/427201393 shutdown_connections 2023-11-24T06:11:05.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa66aab9700 1 --2- 172.21.15.97:0/427201393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640746a0 0x7fa6640b5da0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa66aab9700 1 -- 172.21.15.97:0/427201393 >> 172.21.15.97:0/427201393 conn(0x7fa6640d0bf0 msgr2=0x7fa6640d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:05.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa66aab9700 1 -- 172.21.15.97:0/427201393 shutdown_connections 2023-11-24T06:11:05.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.011+0000 7fa66aab9700 1 -- 172.21.15.97:0/427201393 wait complete. 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.012+0000 7fa66aab9700 1 Processor -- start 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa66aab9700 1 -- start start 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa66aab9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6640746a0 0x7fa664157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa66aab9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640afc90 0x7fa66406e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa66aab9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa66415e160 0x7fa66406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa66aab9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6641f7e90 con 0x7fa6640afc90 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa66aab9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6641567b0 con 0x7fa6640746a0 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa66aab9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa664156a90 con 0x7fa66415e160 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa663fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640afc90 0x7fa66406e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa663fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640afc90 0x7fa66406e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49056/0 (socket says 172.21.15.97:49056) 2023-11-24T06:11:05.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.013+0000 7fa663fff700 1 -- 172.21.15.97:0/954184310 learned_addr learned my addr 172.21.15.97:0/954184310 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa663fff700 1 -- 172.21.15.97:0/954184310 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa66415e160 msgr2=0x7fa66406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa663fff700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa66415e160 0x7fa66406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa668855700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6640746a0 0x7fa664157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa663fff700 1 -- 172.21.15.97:0/954184310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6640746a0 msgr2=0x7fa664157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa663fff700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6640746a0 0x7fa664157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa663fff700 1 -- 172.21.15.97:0/954184310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa650009550 con 0x7fa6640afc90 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa663fff700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640afc90 0x7fa66406e980 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa664072ed0 tx=0x7fa650004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa6617fa700 1 -- 172.21.15.97:0/954184310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650011a70 con 0x7fa6640afc90 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa6617fa700 1 -- 172.21.15.97:0/954184310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65001f070 con 0x7fa6640afc90 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa6617fa700 1 -- 172.21.15.97:0/954184310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650011dd0 con 0x7fa6640afc90 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa664156d70 con 0x7fa6640afc90 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.014+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6641448f0 con 0x7fa6640afc90 2023-11-24T06:11:05.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.015+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa664068fb0 con 0x7fa6640afc90 2023-11-24T06:11:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.015+0000 7fa6617fa700 1 -- 172.21.15.97:0/954184310 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa65001a430 con 0x7fa6640afc90 2023-11-24T06:11:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.016+0000 7fa6617fa700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa64806b110 0x7fa64806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.016+0000 7fa6617fa700 1 -- 172.21.15.97:0/954184310 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa650021070 con 0x7fa6640afc90 2023-11-24T06:11:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.016+0000 7fa668855700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa64806b110 0x7fa64806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:05.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.016+0000 7fa668855700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa64806b110 0x7fa64806d5d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa658000f60 tx=0x7fa658009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:05.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.020+0000 7fa6617fa700 1 -- 172.21.15.97:0/954184310 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6500561f0 con 0x7fa6640afc90 2023-11-24T06:11:05.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:05.259+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa664158970 con 0x7fa6640afc90 2023-11-24T06:11:06.150 INFO:tasks.ceph.mds.a.smithi097.stderr:2023-11-24T06:11:06.147+0000 7f8e4059e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:11:06.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.199+0000 7fa6617fa700 1 -- 172.21.15.97:0/954184310 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa650059810 con 0x7fa6640afc90 2023-11-24T06:11:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa64806b110 msgr2=0x7fa64806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa64806b110 0x7fa64806d5d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa658000f60 tx=0x7fa658009040 comp rx=0 tx=0).stop 2023-11-24T06:11:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640afc90 msgr2=0x7fa66406e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640afc90 0x7fa66406e980 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa664072ed0 tx=0x7fa650004750 comp rx=0 tx=0).stop 2023-11-24T06:11:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 shutdown_connections 2023-11-24T06:11:06.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa64806b110 0x7fa64806d5d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa66415e160 0x7fa66406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6640746a0 0x7fa664157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 --2- 172.21.15.97:0/954184310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6640afc90 0x7fa66406e980 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 >> 172.21.15.97:0/954184310 conn(0x7fa6640d0bf0 msgr2=0x7fa664072540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 shutdown_connections 2023-11-24T06:11:06.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.202+0000 7fa66aab9700 1 -- 172.21.15.97:0/954184310 wait complete. 2023-11-24T06:11:06.220 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:11:06.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:11:06.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.346+0000 7f928658c700 1 Processor -- start 2023-11-24T06:11:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.346+0000 7f928658c700 1 -- start start 2023-11-24T06:11:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.347+0000 7f928658c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f92800cfdf0 0x7f9280073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.347+0000 7f928658c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92800744d0 0x7f92800b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.347+0000 7f928658c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92800b5ee0 con 0x7f928015be70 2023-11-24T06:11:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.347+0000 7f928658c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928006bfc0 con 0x7f92800744d0 2023-11-24T06:11:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.347+0000 7f928658c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9280074940 con 0x7f92800cfdf0 2023-11-24T06:11:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.347+0000 7f9284b29700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92800744d0 0x7f92800b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.347+0000 7f927ffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f928015be70 0x7f92800add40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41514/0 (socket says 172.21.15.97:41514) 2023-11-24T06:11:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.347+0000 7f927ffff700 1 -- 172.21.15.97:0/1685098067 learned_addr learned my addr 172.21.15.97:0/1685098067 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f9284b29700 1 -- 172.21.15.97:0/1685098067 >> v2:172.21.15.183:3300/0 conn(0x7f92800cfdf0 msgr2=0x7f9280073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f9284b29700 1 --2- 172.21.15.97:0/1685098067 >> v2:172.21.15.183:3300/0 conn(0x7f92800cfdf0 0x7f9280073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f9284b29700 1 -- 172.21.15.97:0/1685098067 >> v1:172.21.15.97:6789/0 conn(0x7f928015be70 legacy=0x7f92800add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f9284b29700 1 -- 172.21.15.97:0/1685098067 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928006ba00 con 0x7f92800744d0 2023-11-24T06:11:06.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f9284b29700 1 --2- 172.21.15.97:0/1685098067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800744d0 0x7f92800b3c60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9280072bc0 tx=0x7f927800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c630584507282eba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f927effd700 1 -- 172.21.15.97:0/1685098067 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927800e070 con 0x7f92800744d0 2023-11-24T06:11:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f927effd700 1 -- 172.21.15.97:0/1685098067 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927800bb90 con 0x7f92800744d0 2023-11-24T06:11:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f927effd700 1 -- 172.21.15.97:0/1685098067 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9278012430 con 0x7f92800744d0 2023-11-24T06:11:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f928658c700 1 -- 172.21.15.97:0/1685098067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800744d0 msgr2=0x7f92800b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f928658c700 1 --2- 172.21.15.97:0/1685098067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800744d0 0x7f92800b3c60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9280072bc0 tx=0x7f927800b330 comp rx=0 tx=0).stop 2023-11-24T06:11:06.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f928658c700 1 -- 172.21.15.97:0/1685098067 shutdown_connections 2023-11-24T06:11:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f928658c700 1 --2- 172.21.15.97:0/1685098067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800744d0 0x7f92800b3c60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f928658c700 1 --2- 172.21.15.97:0/1685098067 >> v2:172.21.15.183:3300/0 conn(0x7f92800cfdf0 0x7f9280073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.348+0000 7f928658c700 1 -- 172.21.15.97:0/1685098067 >> 172.21.15.97:0/1685098067 conn(0x7f92800d0bf0 msgr2=0x7f92800d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.349+0000 7f928658c700 1 -- 172.21.15.97:0/1685098067 shutdown_connections 2023-11-24T06:11:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.349+0000 7f928658c700 1 -- 172.21.15.97:0/1685098067 wait complete. 2023-11-24T06:11:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.350+0000 7f928658c700 1 Processor -- start 2023-11-24T06:11:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.350+0000 7f928658c700 1 -- start start 2023-11-24T06:11:06.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f928658c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92800744d0 0x7f928015e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f928658c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800cfdf0 0x7f928015c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f928658c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928015be70 0x7f9280151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f928658c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928015ce90 con 0x7f92800cfdf0 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f928658c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928015d170 con 0x7f928015be70 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f928658c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928015d450 con 0x7f92800744d0 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800cfdf0 0x7f928015c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800cfdf0 0x7f928015c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49070/0 (socket says 172.21.15.97:49070) 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 -- 172.21.15.97:0/97312781 learned_addr learned my addr 172.21.15.97:0/97312781 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927ffff700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92800744d0 0x7f928015e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f9284b29700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928015be70 0x7f9280151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 -- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92800744d0 msgr2=0x7f928015e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92800744d0 0x7f928015e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 -- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928015be70 msgr2=0x7f9280151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928015be70 0x7f9280151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 -- 172.21.15.97:0/97312781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278009a70 con 0x7f92800cfdf0 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927ffff700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92800744d0 0x7f928015e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f9284b29700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928015be70 0x7f9280151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927f7fe700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800cfdf0 0x7f928015c3d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f926800b530 tx=0x7f926800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927cff9700 1 -- 172.21.15.97:0/97312781 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268011830 con 0x7f92800cfdf0 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927cff9700 1 -- 172.21.15.97:0/97312781 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92680119b0 con 0x7f92800cfdf0 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928015d730 con 0x7f92800cfdf0 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f927cff9700 1 -- 172.21.15.97:0/97312781 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268011d60 con 0x7f92800cfdf0 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.351+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9280152280 con 0x7f92800cfdf0 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.352+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92800aea50 con 0x7f92800cfdf0 2023-11-24T06:11:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.353+0000 7f927cff9700 1 -- 172.21.15.97:0/97312781 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f926801a400 con 0x7f92800cfdf0 2023-11-24T06:11:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.353+0000 7f927cff9700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f926006b110 0x7f926006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.353+0000 7f927cff9700 1 -- 172.21.15.97:0/97312781 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9268089a40 con 0x7f92800cfdf0 2023-11-24T06:11:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.353+0000 7f927ffff700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f926006b110 0x7f926006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.353+0000 7f927ffff700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f926006b110 0x7f926006d5d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9270005910 tx=0x7f9270009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.357+0000 7f927cff9700 1 -- 172.21.15.97:0/97312781 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f926808be60 con 0x7f92800cfdf0 2023-11-24T06:11:06.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:06.594+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9280068fb0 con 0x7f92800cfdf0 2023-11-24T06:11:07.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.207+0000 7f927cff9700 1 -- 172.21.15.97:0/97312781 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9268015020 con 0x7f92800cfdf0 2023-11-24T06:11:07.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f926006b110 msgr2=0x7f926006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f926006b110 0x7f926006d5d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9270005910 tx=0x7f9270009040 comp rx=0 tx=0).stop 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800cfdf0 msgr2=0x7f928015c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800cfdf0 0x7f928015c3d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f926800b530 tx=0x7f926800b8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 shutdown_connections 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928015be70 0x7f9280151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f926006b110 0x7f926006d5d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f92800744d0 0x7f928015e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 --2- 172.21.15.97:0/97312781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92800cfdf0 0x7f928015c3d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 >> 172.21.15.97:0/97312781 conn(0x7f92800d0bf0 msgr2=0x7f92800add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 shutdown_connections 2023-11-24T06:11:07.215 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.211+0000 7f928658c700 1 -- 172.21.15.97:0/97312781 wait complete. 2023-11-24T06:11:07.227 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:11:07.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:07.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.349+0000 7fdf02f6a700 1 Processor -- start 2023-11-24T06:11:07.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.349+0000 7fdf02f6a700 1 -- start start 2023-11-24T06:11:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.350+0000 7fdf02f6a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 0x7fdefc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.350+0000 7fdf02f6a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0ae430 con 0x7fdefc0af4d0 2023-11-24T06:11:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.350+0000 7fdf02f6a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc0ae710 con 0x7fdefc06c710 2023-11-24T06:11:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.350+0000 7fdf02f6a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0af0b0 con 0x7fdefc15e080 2023-11-24T06:11:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.350+0000 7fdefbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdefc0af4d0 0x7fdefc06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41528/0 (socket says 172.21.15.97:41528) 2023-11-24T06:11:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.350+0000 7fdefbfff700 1 -- 172.21.15.97:0/3923113597 learned_addr learned my addr 172.21.15.97:0/3923113597 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.350+0000 7fdf00d06700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 0x7fdefc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.356 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.350+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844692572 0 0) 0x7fdefc0ae430 con 0x7fdefc0af4d0 2023-11-24T06:11:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeec0035b0 con 0x7fdefc0af4d0 2023-11-24T06:11:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3806086477 0 0) 0x7fdeec0035b0 con 0x7fdefc0af4d0 2023-11-24T06:11:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 msgr2=0x7fdefc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 0x7fdefc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 >> v1:172.21.15.183:6789/0 conn(0x7fdefc06c710 legacy=0x7fdefc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc0ae9f0 con 0x7fdefc0af4d0 2023-11-24T06:11:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fdee8001670 con 0x7fdefc0af4d0 2023-11-24T06:11:07.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf00d06700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 0x7fdefc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 >> v1:172.21.15.97:6789/0 conn(0x7fdefc0af4d0 legacy=0x7fdefc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeec003a30 0x7fdeec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 0x7fdeec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeec004d90 0x7fdeec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0af0b0 con 0x7fdeec004d90 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec00ba10 con 0x7fdeec003a30 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec00bb80 con 0x7fdefc15e080 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf01507700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeec004d90 0x7fdeec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdefbfff700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 0x7fdeec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.358 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf00d06700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeec003a30 0x7fdeec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf01507700 1 -- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 msgr2=0x7fdeec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:07.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf01507700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdefc15e080 0x7fdeec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.359 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf01507700 1 -- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeec003a30 msgr2=0x7fdeec003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:07.360 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf01507700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeec003a30 0x7fdeec003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf01507700 1 -- 172.21.15.97:0/3923113597 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00094a0 con 0x7fdeec004d90 2023-11-24T06:11:07.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.351+0000 7fdf01507700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeec004d90 0x7fdeec005210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c290 tx=0x7fdeec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabb8508af4b8075 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:07.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4001e70 con 0x7fdeec004d90 2023-11-24T06:11:07.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef400ce50 con 0x7fdeec004d90 2023-11-24T06:11:07.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdefb7fe700 1 -- 172.21.15.97:0/3923113597 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4012660 con 0x7fdeec004d90 2023-11-24T06:11:07.361 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3923113597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeec004d90 msgr2=0x7fdeec005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdf02f6a700 1 --2- 172.21.15.97:0/3923113597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeec004d90 0x7fdeec005210 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c290 tx=0x7fdeec005880 comp rx=0 tx=0).stop 2023-11-24T06:11:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdf01507700 1 -- 172.21.15.97:0/3923113597 reap_dead start 2023-11-24T06:11:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3923113597 shutdown_connections 2023-11-24T06:11:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3923113597 >> 172.21.15.97:0/3923113597 conn(0x7fdefc071b90 msgr2=0x7fdefc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3923113597 shutdown_connections 2023-11-24T06:11:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.352+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3923113597 wait complete. 2023-11-24T06:11:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.353+0000 7fdf02f6a700 1 Processor -- start 2023-11-24T06:11:07.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.353+0000 7fdf02f6a700 1 -- start start 2023-11-24T06:11:07.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf02f6a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeec004d90 0x7fdefc14eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf02f6a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdefc159b40 0x7fdefc0c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf02f6a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0c7910 0x7fdefc0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf02f6a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0c7da0 con 0x7fdefc0c7910 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf02f6a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc14df20 con 0x7fdefc159b40 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf02f6a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc0c6be0 con 0x7fdeec004d90 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf01507700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0c7910 0x7fdefc0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf01507700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0c7910 0x7fdefc0c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49092/0 (socket says 172.21.15.97:49092) 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.354+0000 7fdf01507700 1 -- 172.21.15.97:0/3106290969 learned_addr learned my addr 172.21.15.97:0/3106290969 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf00d06700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeec004d90 0x7fdefc14eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdefbfff700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdefc159b40 0x7fdefc0c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf01507700 1 -- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeec004d90 msgr2=0x7fdefc14eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf01507700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeec004d90 0x7fdefc14eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf01507700 1 -- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdefc159b40 msgr2=0x7fdefc0c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf01507700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdefc159b40 0x7fdefc0c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf01507700 1 -- 172.21.15.97:0/3106290969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc14e1c0 con 0x7fdefc0c7910 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf00d06700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeec004d90 0x7fdefc14eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdefbfff700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdefc159b40 0x7fdefc0c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf01507700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0c7910 0x7fdefc0c5e70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0047d0 tx=0x7fdef40015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdef97fa700 1 -- 172.21.15.97:0/3106290969 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40050c0 con 0x7fdefc0c7910 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdef97fa700 1 -- 172.21.15.97:0/3106290969 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4005220 con 0x7fdefc0c7910 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc14e4a0 con 0x7fdefc0c7910 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdef97fa700 1 -- 172.21.15.97:0/3106290969 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40055d0 con 0x7fdefc0c7910 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.355+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc159fc0 con 0x7fdefc0c7910 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.356+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc0753f0 con 0x7fdefc0c7910 2023-11-24T06:11:07.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.356+0000 7fdef97fa700 1 -- 172.21.15.97:0/3106290969 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fdef4023070 con 0x7fdefc0c7910 2023-11-24T06:11:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.357+0000 7fdef97fa700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdee006b110 0x7fdee006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.357+0000 7fdef97fa700 1 -- 172.21.15.97:0/3106290969 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdef408b280 con 0x7fdefc0c7910 2023-11-24T06:11:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.357+0000 7fdf00d06700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdee006b110 0x7fdee006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.357+0000 7fdf00d06700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdee006b110 0x7fdee006d5d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009ce0 tx=0x7fdef0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:07.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.361+0000 7fdef97fa700 1 -- 172.21.15.97:0/3106290969 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdef4056250 con 0x7fdefc0c7910 2023-11-24T06:11:07.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.595+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdefc14e9c0 con 0x7fdefc0c7910 2023-11-24T06:11:07.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.595+0000 7fdef97fa700 1 -- 172.21.15.97:0/3106290969 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fdefc14e9c0 con 0x7fdefc0c7910 2023-11-24T06:11:07.598 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:07.598 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3508641674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3508641674},{"type":"v1","addr":"172.21.15.183:6833","nonce":3508641674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5557,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/116508824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":116508824},{"type":"v1","addr":"172.21.15.183:6835","nonce":116508824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2785455707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2785455707},{"type":"v1","addr":"172.21.15.97:6837","nonce":2785455707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:03.572274+0000","modified":"2023-11-24T06:11:06.595718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3662495999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3662495999},{"type":"v1","addr":"172.21.15.97:6836","nonce":3662495999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:11:07.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdee006b110 msgr2=0x7fdee006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdee006b110 0x7fdee006d5d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009ce0 tx=0x7fdef0008040 comp rx=0 tx=0).stop 2023-11-24T06:11:07.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0c7910 msgr2=0x7fdefc0c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0c7910 0x7fdefc0c5e70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0047d0 tx=0x7fdef40015b0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 shutdown_connections 2023-11-24T06:11:07.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdefc0c7910 0x7fdefc0c5e70 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdee006b110 0x7fdee006d5d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeec004d90 0x7fdefc14eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 --2- 172.21.15.97:0/3106290969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdefc159b40 0x7fdefc0c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 >> 172.21.15.97:0/3106290969 conn(0x7fdefc071b90 msgr2=0x7fdefc0b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 shutdown_connections 2023-11-24T06:11:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.597+0000 7fdf02f6a700 1 -- 172.21.15.97:0/3106290969 wait complete. 2023-11-24T06:11:07.601 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 36 2023-11-24T06:11:07.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:07.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.732+0000 7ffbc2349700 1 Processor -- start 2023-11-24T06:11:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.733+0000 7ffbc2349700 1 -- start start 2023-11-24T06:11:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.733+0000 7ffbc2349700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ffbbc15bd60 0x7ffbbc0d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.733+0000 7ffbc2349700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc0cc780 con 0x7ffbbc0cf2b0 2023-11-24T06:11:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.733+0000 7ffbc2349700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc0cc9b0 con 0x7ffbbc15c140 2023-11-24T06:11:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.733+0000 7ffbc2349700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc0ccc90 con 0x7ffbbc15bd60 2023-11-24T06:11:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.733+0000 7ffbc08e6700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffbbc0cf2b0 0x7ffbbc0c57c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41552/0 (socket says 172.21.15.97:41552) 2023-11-24T06:11:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.733+0000 7ffbc08e6700 1 -- 172.21.15.97:0/1187854613 learned_addr learned my addr 172.21.15.97:0/1187854613 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132596002 0 0) 0x7ffbbc0cc780 con 0x7ffbbc0cf2b0 2023-11-24T06:11:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbbfff700 1 --2- 172.21.15.97:0/1187854613 >> v2:172.21.15.183:3300/0 conn(0x7ffbbc15bd60 0x7ffbbc0d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba40035b0 con 0x7ffbbc0cf2b0 2023-11-24T06:11:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946110316 0 0) 0x7ffba40035b0 con 0x7ffbbc0cf2b0 2023-11-24T06:11:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 >> v2:172.21.15.183:3300/0 conn(0x7ffbbc15bd60 msgr2=0x7ffbbc0d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 --2- 172.21.15.97:0/1187854613 >> v2:172.21.15.183:3300/0 conn(0x7ffbbc15bd60 0x7ffbbc0d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 >> v1:172.21.15.183:6789/0 conn(0x7ffbbc15c140 legacy=0x7ffbbc0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc0ccf70 con 0x7ffbbc0cf2b0 2023-11-24T06:11:07.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7ffbb40019f0 con 0x7ffbbc0cf2b0 2023-11-24T06:11:07.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 >> v1:172.21.15.97:6789/0 conn(0x7ffbbc0cf2b0 legacy=0x7ffbbc0c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba4003a00 0x7ffba4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffba4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffba4005210 0x7ffba400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0ccc90 con 0x7ffba4004850 2023-11-24T06:11:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba400c0b0 con 0x7ffba4003a00 2023-11-24T06:11:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba400c270 con 0x7ffba4005210 2023-11-24T06:11:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbb7fe700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffba4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbbfff700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba4003a00 0x7ffba4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbc08e6700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffba4005210 0x7ffba400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbb7fe700 1 -- 172.21.15.97:0/1187854613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffba4005210 msgr2=0x7ffba400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:07.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbb7fe700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffba4005210 0x7ffba400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbb7fe700 1 -- 172.21.15.97:0/1187854613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba4003a00 msgr2=0x7ffba4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:07.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbb7fe700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba4003a00 0x7ffba4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbbb7fe700 1 -- 172.21.15.97:0/1187854613 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba40035b0 con 0x7ffba4004850 2023-11-24T06:11:07.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.734+0000 7ffbc08e6700 1 -- 172.21.15.97:0/1187854613 reap_dead start 2023-11-24T06:11:07.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbbb7fe700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffba4004cd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00017c0 tx=0x7ffba4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b139cb3dd050ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:07.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00052d0 con 0x7ffba4004850 2023-11-24T06:11:07.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb0005430 con 0x7ffba4004850 2023-11-24T06:11:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbbaffd700 1 -- 172.21.15.97:0/1187854613 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00038c0 con 0x7ffba4004850 2023-11-24T06:11:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbc2349700 1 -- 172.21.15.97:0/1187854613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 msgr2=0x7ffba4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbc2349700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffba4004cd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00017c0 tx=0x7ffba4005880 comp rx=0 tx=0).stop 2023-11-24T06:11:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbc2349700 1 -- 172.21.15.97:0/1187854613 shutdown_connections 2023-11-24T06:11:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbc2349700 1 --2- 172.21.15.97:0/1187854613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffba4004cd0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbc2349700 1 -- 172.21.15.97:0/1187854613 >> 172.21.15.97:0/1187854613 conn(0x7ffbbc073130 msgr2=0x7ffbbc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:07.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbc2349700 1 -- 172.21.15.97:0/1187854613 shutdown_connections 2023-11-24T06:11:07.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.735+0000 7ffbc2349700 1 -- 172.21.15.97:0/1187854613 wait complete. 2023-11-24T06:11:07.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.737+0000 7ffbc2349700 1 Processor -- start 2023-11-24T06:11:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.737+0000 7ffbc2349700 1 -- start start 2023-11-24T06:11:07.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbc2349700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffbbc0c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbc2349700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbbc0c7510 0x7ffbbc156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbc2349700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffbbc156ac0 0x7ffbbc153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbc2349700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0c7080 con 0x7ffba4004850 2023-11-24T06:11:07.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbc2349700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc1f1c70 con 0x7ffbbc0c7510 2023-11-24T06:11:07.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbc2349700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0c6470 con 0x7ffbbc156ac0 2023-11-24T06:11:07.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffbbc0c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffbbc0c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49108/0 (socket says 172.21.15.97:49108) 2023-11-24T06:11:07.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 -- 172.21.15.97:0/92534152 learned_addr learned my addr 172.21.15.97:0/92534152 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbb7fe700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbbc0c7510 0x7ffbbc156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbc08e6700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffbbc156ac0 0x7ffbbc153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 -- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffbbc156ac0 msgr2=0x7ffbbc153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffbbc156ac0 0x7ffbbc153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 -- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbbc0c7510 msgr2=0x7ffbbc156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbbc0c7510 0x7ffbbc156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 -- 172.21.15.97:0/92534152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc0c6750 con 0x7ffba4004850 2023-11-24T06:11:07.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbc08e6700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffbbc156ac0 0x7ffbbc153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:07.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.738+0000 7ffbbbfff700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffbbc0c5c20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ffbac011a00 tx=0x7ffbac011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:07.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.739+0000 7ffbb8ff9700 1 -- 172.21.15.97:0/92534152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac009630 con 0x7ffba4004850 2023-11-24T06:11:07.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.739+0000 7ffbb8ff9700 1 -- 172.21.15.97:0/92534152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac004240 con 0x7ffba4004850 2023-11-24T06:11:07.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.739+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc0c6a30 con 0x7ffba4004850 2023-11-24T06:11:07.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.739+0000 7ffbb8ff9700 1 -- 172.21.15.97:0/92534152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00f730 con 0x7ffba4004850 2023-11-24T06:11:07.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.739+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc0c6160 con 0x7ffba4004850 2023-11-24T06:11:07.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.739+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc0c8140 con 0x7ffba4004850 2023-11-24T06:11:07.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.739+0000 7ffbb8ff9700 1 -- 172.21.15.97:0/92534152 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ffbac00f890 con 0x7ffba4004850 2023-11-24T06:11:07.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.739+0000 7ffbb8ff9700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffba806b110 0x7ffba806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:07.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.740+0000 7ffbb8ff9700 1 -- 172.21.15.97:0/92534152 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbac089dd0 con 0x7ffba4004850 2023-11-24T06:11:07.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.740+0000 7ffbbb7fe700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffba806b110 0x7ffba806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:07.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.740+0000 7ffbbb7fe700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffba806b110 0x7ffba806d5d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005040 tx=0x7ffbb0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:07.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.744+0000 7ffbb8ff9700 1 -- 172.21.15.97:0/92534152 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffbac054bb0 con 0x7ffba4004850 2023-11-24T06:11:07.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.934+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbbc067f60 con 0x7ffba4004850 2023-11-24T06:11:07.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.935+0000 7ffbb8ff9700 1 -- 172.21.15.97:0/92534152 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16577 (secure 0 0 0) 0x7ffbac023090 con 0x7ffba4004850 2023-11-24T06:11:07.939 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:07.940 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:05.166928+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:01.950100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:02.526161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.937+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffba806b110 msgr2=0x7ffba806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.937+0000 7ffbc2349700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffba806b110 0x7ffba806d5d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0005040 tx=0x7ffbb0009030 comp rx=0 tx=0).stop 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 msgr2=0x7ffbbc0c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffbbc0c5c20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ffbac011a00 tx=0x7ffbac011d10 comp rx=0 tx=0).stop 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 shutdown_connections 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffba806b110 0x7ffba806d5d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffbbc156ac0 0x7ffbbc153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba4004850 0x7ffbbc0c5c20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 --2- 172.21.15.97:0/92534152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbbc0c7510 0x7ffbbc156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 >> 172.21.15.97:0/92534152 conn(0x7ffbbc073130 msgr2=0x7ffbbc0ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 shutdown_connections 2023-11-24T06:11:07.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:07.938+0000 7ffbc2349700 1 -- 172.21.15.97:0/92534152 wait complete. 2023-11-24T06:11:07.954 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:11:07.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:11:08.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.070+0000 7f81d38a0700 1 Processor -- start 2023-11-24T06:11:08.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.070+0000 7f81d38a0700 1 -- start start 2023-11-24T06:11:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d38a0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc15e110 0x7f81cc0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d38a0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f81cc0b1570 0x7f81cc0d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d38a0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc0ce4a0 con 0x7f81cc0d2040 2023-11-24T06:11:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d38a0700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc0d24b0 con 0x7f81cc0b1570 2023-11-24T06:11:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d38a0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0cefe0 con 0x7f81cc15e110 2023-11-24T06:11:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d0e3b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f81cc0b1570 0x7f81cc0d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d0e3b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f81cc0b1570 0x7f81cc0d1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49120/0 (socket says 172.21.15.97:49120) 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d0e3b700 1 -- 172.21.15.97:0/3042122077 learned_addr learned my addr 172.21.15.97:0/3042122077 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d163c700 1 --2- 172.21.15.97:0/3042122077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc15e110 0x7f81cc0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d0e3b700 1 -- 172.21.15.97:0/3042122077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc15e110 msgr2=0x7f81cc0b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d0e3b700 1 --2- 172.21.15.97:0/3042122077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc15e110 0x7f81cc0b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d0e3b700 1 -- 172.21.15.97:0/3042122077 >> v1:172.21.15.183:6789/0 conn(0x7f81cc0d2040 legacy=0x7f81cc161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d0e3b700 1 -- 172.21.15.97:0/3042122077 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0cf470 con 0x7f81cc0b1570 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81d0e3b700 1 --2- 172.21.15.97:0/3042122077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc0b1570 0x7f81cc0d1b00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0098f0 tx=0x7f81bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d259a310ac6fae5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81c3fff700 1 -- 172.21.15.97:0/3042122077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00b8a0 con 0x7f81cc0b1570 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81c3fff700 1 -- 172.21.15.97:0/3042122077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81bc00ba00 con 0x7f81cc0b1570 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.071+0000 7f81c3fff700 1 -- 172.21.15.97:0/3042122077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00bd90 con 0x7f81cc0b1570 2023-11-24T06:11:08.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.072+0000 7f81d38a0700 1 -- 172.21.15.97:0/3042122077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc0b1570 msgr2=0x7f81cc0d1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.072+0000 7f81d38a0700 1 --2- 172.21.15.97:0/3042122077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc0b1570 0x7f81cc0d1b00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0098f0 tx=0x7f81bc009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.072+0000 7f81d38a0700 1 -- 172.21.15.97:0/3042122077 shutdown_connections 2023-11-24T06:11:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.072+0000 7f81d38a0700 1 --2- 172.21.15.97:0/3042122077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc15e110 0x7f81cc0b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.072+0000 7f81d38a0700 1 --2- 172.21.15.97:0/3042122077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc0b1570 0x7f81cc0d1b00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.072+0000 7f81d38a0700 1 -- 172.21.15.97:0/3042122077 >> 172.21.15.97:0/3042122077 conn(0x7f81cc0730a0 msgr2=0x7f81cc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.072+0000 7f81d38a0700 1 -- 172.21.15.97:0/3042122077 shutdown_connections 2023-11-24T06:11:08.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.072+0000 7f81d38a0700 1 -- 172.21.15.97:0/3042122077 wait complete. 2023-11-24T06:11:08.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.073+0000 7f81d38a0700 1 Processor -- start 2023-11-24T06:11:08.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.073+0000 7f81d38a0700 1 -- start start 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d38a0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc0b1570 0x7f81cc0c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d38a0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81cc0d2040 0x7f81cc14f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d38a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc15e110 0x7f81cc0aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d38a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc14f5d0 con 0x7f81cc15e110 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d38a0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0c8d90 con 0x7f81cc0d2040 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d38a0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0c9070 con 0x7f81cc0b1570 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d1e3d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc15e110 0x7f81cc0aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d1e3d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc15e110 0x7f81cc0aff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49126/0 (socket says 172.21.15.97:49126) 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d1e3d700 1 -- 172.21.15.97:0/1649967220 learned_addr learned my addr 172.21.15.97:0/1649967220 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d0e3b700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81cc0d2040 0x7f81cc14f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d163c700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc0b1570 0x7f81cc0c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d1e3d700 1 -- 172.21.15.97:0/1649967220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc0b1570 msgr2=0x7f81cc0c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d1e3d700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc0b1570 0x7f81cc0c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d1e3d700 1 -- 172.21.15.97:0/1649967220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81cc0d2040 msgr2=0x7f81cc14f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d1e3d700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81cc0d2040 0x7f81cc14f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.074+0000 7f81d1e3d700 1 -- 172.21.15.97:0/1649967220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc009550 con 0x7f81cc15e110 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.075+0000 7f81d1e3d700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc15e110 0x7f81cc0aff10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f81c400cc50 tx=0x7f81c400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.075+0000 7f81c1ffb700 1 -- 172.21.15.97:0/1649967220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4003c00 con 0x7f81cc15e110 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.075+0000 7f81c1ffb700 1 -- 172.21.15.97:0/1649967220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c4003d60 con 0x7f81cc15e110 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.075+0000 7f81c1ffb700 1 -- 172.21.15.97:0/1649967220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c400aab0 con 0x7f81cc15e110 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.075+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc0c9350 con 0x7f81cc15e110 2023-11-24T06:11:08.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.075+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc0c9910 con 0x7f81cc15e110 2023-11-24T06:11:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.076+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc0cec50 con 0x7f81cc15e110 2023-11-24T06:11:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.076+0000 7f81c1ffb700 1 -- 172.21.15.97:0/1649967220 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f81c400ac10 con 0x7f81cc15e110 2023-11-24T06:11:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.077+0000 7f81c1ffb700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81b006b110 0x7f81b006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.077+0000 7f81c1ffb700 1 -- 172.21.15.97:0/1649967220 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81c40899e0 con 0x7f81cc15e110 2023-11-24T06:11:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.077+0000 7f81d163c700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81b006b110 0x7f81b006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.077+0000 7f81d163c700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81b006b110 0x7f81b006d5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f81c8006b20 tx=0x7f81c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.081+0000 7f81c1ffb700 1 -- 172.21.15.97:0/1649967220 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81cc0cec50 con 0x7f81cc15e110 2023-11-24T06:11:08.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.270+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81cc0b3050 con 0x7f81b006b110 2023-11-24T06:11:08.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.278+0000 7f81c1ffb700 1 -- 172.21.15.97:0/1649967220 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289068 (secure 0 0 0) 0x7f81cc0b3050 con 0x7f81b006b110 2023-11-24T06:11:08.285 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81b006b110 msgr2=0x7f81b006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81b006b110 0x7f81b006d5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f81c8006b20 tx=0x7f81c8008040 comp rx=0 tx=0).stop 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc15e110 msgr2=0x7f81cc0aff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc15e110 0x7f81cc0aff10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f81c400cc50 tx=0x7f81c400cf60 comp rx=0 tx=0).stop 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 shutdown_connections 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81cc15e110 0x7f81cc0aff10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81b006b110 0x7f81b006d5d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f81cc0b1570 0x7f81cc0c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 --2- 172.21.15.97:0/1649967220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81cc0d2040 0x7f81cc14f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.281+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 >> 172.21.15.97:0/1649967220 conn(0x7f81cc0730a0 msgr2=0x7f81cc15bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:08.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.282+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 shutdown_connections 2023-11-24T06:11:08.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.282+0000 7f81d38a0700 1 -- 172.21.15.97:0/1649967220 wait complete. 2023-11-24T06:11:08.289 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:11:08.297 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2023-11-24T06:11:06.350430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222828,"kb_used_data":6640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751892,"statfs":{"total":773094113280,"available":772865937408,"internally_reserved":0,"allocated":6799360,"data_stored":3514062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.189131"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.198867+0000","last_change":"2023-11-24T06:11:04.195308+0000","last_active":"2023-11-24T06:11:04.198867+0000","last_peered":"2023-11-24T06:11:04.198867+0000","last_clean":"2023-11-24T06:11:04.198867+0000","last_became_active":"2023-11-24T06:11:04.195019+0000","last_became_peered":"2023-11-24T06:11:04.195019+0000","last_unstale":"2023-11-24T06:11:04.198867+0000","last_undegraded":"2023-11-24T06:11:04.198867+0000","last_fullsized":"2023-11-24T06:11:04.198867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:58.686397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.203499+0000","last_change":"2023-11-24T06:11:04.194515+0000","last_active":"2023-11-24T06:11:04.203499+0000","last_peered":"2023-11-24T06:11:04.203499+0000","last_clean":"2023-11-24T06:11:04.203499+0000","last_became_active":"2023-11-24T06:11:04.194185+0000","last_became_peered":"2023-11-24T06:11:04.194185+0000","last_unstale":"2023-11-24T06:11:04.203499+0000","last_undegraded":"2023-11-24T06:11:04.203499+0000","last_fullsized":"2023-11-24T06:11:04.203499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:06:53.543382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195390+0000","last_change":"2023-11-24T06:11:04.189706+0000","last_active":"2023-11-24T06:11:04.195390+0000","last_peered":"2023-11-24T06:11:04.195390+0000","last_clean":"2023-11-24T06:11:04.195390+0000","last_became_active":"2023-11-24T06:11:04.189380+0000","last_became_peered":"2023-11-24T06:11:04.189380+0000","last_unstale":"2023-11-24T06:11:04.195390+0000","last_undegraded":"2023-11-24T06:11:04.195390+0000","last_fullsized":"2023-11-24T06:11:04.195390+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:14.947513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183574+0000","last_change":"2023-11-24T06:11:04.180414+0000","last_active":"2023-11-24T06:11:04.183574+0000","last_peered":"2023-11-24T06:11:04.183574+0000","last_clean":"2023-11-24T06:11:04.183574+0000","last_became_active":"2023-11-24T06:11:04.180124+0000","last_became_peered":"2023-11-24T06:11:04.180124+0000","last_unstale":"2023-11-24T06:11:04.183574+0000","last_undegraded":"2023-11-24T06:11:04.183574+0000","last_fullsized":"2023-11-24T06:11:04.183574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:44.649472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185983+0000","last_change":"2023-11-24T06:11:04.183956+0000","last_active":"2023-11-24T06:11:04.185983+0000","last_peered":"2023-11-24T06:11:04.185983+0000","last_clean":"2023-11-24T06:11:04.185983+0000","last_became_active":"2023-11-24T06:11:04.183607+0000","last_became_peered":"2023-11-24T06:11:04.183607+0000","last_unstale":"2023-11-24T06:11:04.185983+0000","last_undegraded":"2023-11-24T06:11:04.185983+0000","last_fullsized":"2023-11-24T06:11:04.185983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:22:17.116277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185858+0000","last_change":"2023-11-24T06:11:04.180341+0000","last_active":"2023-11-24T06:11:04.185858+0000","last_peered":"2023-11-24T06:11:04.185858+0000","last_clean":"2023-11-24T06:11:04.185858+0000","last_became_active":"2023-11-24T06:11:04.179989+0000","last_became_peered":"2023-11-24T06:11:04.179989+0000","last_unstale":"2023-11-24T06:11:04.185858+0000","last_undegraded":"2023-11-24T06:11:04.185858+0000","last_fullsized":"2023-11-24T06:11:04.185858+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:54.702175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183751+0000","last_change":"2023-11-24T06:11:04.177751+0000","last_active":"2023-11-24T06:11:04.183751+0000","last_peered":"2023-11-24T06:11:04.183751+0000","last_clean":"2023-11-24T06:11:04.183751+0000","last_became_active":"2023-11-24T06:11:04.177470+0000","last_became_peered":"2023-11-24T06:11:04.177470+0000","last_unstale":"2023-11-24T06:11:04.183751+0000","last_undegraded":"2023-11-24T06:11:04.183751+0000","last_fullsized":"2023-11-24T06:11:04.183751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:26:52.488945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205848+0000","last_change":"2023-11-24T06:11:04.200447+0000","last_active":"2023-11-24T06:11:04.205848+0000","last_peered":"2023-11-24T06:11:04.205848+0000","last_clean":"2023-11-24T06:11:04.205848+0000","last_became_active":"2023-11-24T06:11:04.197503+0000","last_became_peered":"2023-11-24T06:11:04.197503+0000","last_unstale":"2023-11-24T06:11:04.205848+0000","last_undegraded":"2023-11-24T06:11:04.205848+0000","last_fullsized":"2023-11-24T06:11:04.205848+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:40.810920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.188703+0000","last_change":"2023-11-24T06:11:04.186747+0000","last_active":"2023-11-24T06:11:04.188703+0000","last_peered":"2023-11-24T06:11:04.188703+0000","last_clean":"2023-11-24T06:11:04.188703+0000","last_became_active":"2023-11-24T06:11:04.186476+0000","last_became_peered":"2023-11-24T06:11:04.186476+0000","last_unstale":"2023-11-24T06:11:04.188703+0000","last_undegraded":"2023-11-24T06:11:04.188703+0000","last_fullsized":"2023-11-24T06:11:04.188703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:14.947513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197128+0000","last_change":"2023-11-24T06:11:04.189046+0000","last_active":"2023-11-24T06:11:04.197128+0000","last_peered":"2023-11-24T06:11:04.197128+0000","last_clean":"2023-11-24T06:11:04.197128+0000","last_became_active":"2023-11-24T06:11:04.188805+0000","last_became_peered":"2023-11-24T06:11:04.188805+0000","last_unstale":"2023-11-24T06:11:04.197128+0000","last_undegraded":"2023-11-24T06:11:04.197128+0000","last_fullsized":"2023-11-24T06:11:04.197128+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:14.016212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197954+0000","last_change":"2023-11-24T06:11:04.189281+0000","last_active":"2023-11-24T06:11:04.197954+0000","last_peered":"2023-11-24T06:11:04.197954+0000","last_clean":"2023-11-24T06:11:04.197954+0000","last_became_active":"2023-11-24T06:11:04.188736+0000","last_became_peered":"2023-11-24T06:11:04.188736+0000","last_unstale":"2023-11-24T06:11:04.197954+0000","last_undegraded":"2023-11-24T06:11:04.197954+0000","last_fullsized":"2023-11-24T06:11:04.197954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:42.079045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195412+0000","last_change":"2023-11-24T06:11:04.189330+0000","last_active":"2023-11-24T06:11:04.195412+0000","last_peered":"2023-11-24T06:11:04.195412+0000","last_clean":"2023-11-24T06:11:04.195412+0000","last_became_active":"2023-11-24T06:11:04.189059+0000","last_became_peered":"2023-11-24T06:11:04.189059+0000","last_unstale":"2023-11-24T06:11:04.195412+0000","last_undegraded":"2023-11-24T06:11:04.195412+0000","last_fullsized":"2023-11-24T06:11:04.195412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:03.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.200854+0000","last_change":"2023-11-24T06:11:04.198923+0000","last_active":"2023-11-24T06:11:04.200854+0000","last_peered":"2023-11-24T06:11:04.200854+0000","last_clean":"2023-11-24T06:11:04.200854+0000","last_became_active":"2023-11-24T06:11:04.198533+0000","last_became_peered":"2023-11-24T06:11:04.198533+0000","last_unstale":"2023-11-24T06:11:04.200854+0000","last_undegraded":"2023-11-24T06:11:04.200854+0000","last_fullsized":"2023-11-24T06:11:04.200854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:32.690697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185252+0000","last_change":"2023-11-24T06:11:04.181224+0000","last_active":"2023-11-24T06:11:04.185252+0000","last_peered":"2023-11-24T06:11:04.185252+0000","last_clean":"2023-11-24T06:11:04.185252+0000","last_became_active":"2023-11-24T06:11:04.180958+0000","last_became_peered":"2023-11-24T06:11:04.180958+0000","last_unstale":"2023-11-24T06:11:04.185252+0000","last_undegraded":"2023-11-24T06:11:04.185252+0000","last_fullsized":"2023-11-24T06:11:04.185252+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:30.803405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185067+0000","last_change":"2023-11-24T06:11:04.180479+0000","last_active":"2023-11-24T06:11:04.185067+0000","last_peered":"2023-11-24T06:11:04.185067+0000","last_clean":"2023-11-24T06:11:04.185067+0000","last_became_active":"2023-11-24T06:11:04.180177+0000","last_became_peered":"2023-11-24T06:11:04.180177+0000","last_unstale":"2023-11-24T06:11:04.185067+0000","last_undegraded":"2023-11-24T06:11:04.185067+0000","last_fullsized":"2023-11-24T06:11:04.185067+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:21.633855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.200922+0000","last_change":"2023-11-24T06:11:04.193957+0000","last_active":"2023-11-24T06:11:04.200922+0000","last_peered":"2023-11-24T06:11:04.200922+0000","last_clean":"2023-11-24T06:11:04.200922+0000","last_became_active":"2023-11-24T06:11:04.192891+0000","last_became_peered":"2023-11-24T06:11:04.192891+0000","last_unstale":"2023-11-24T06:11:04.200922+0000","last_undegraded":"2023-11-24T06:11:04.200922+0000","last_fullsized":"2023-11-24T06:11:04.200922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:41:06.897965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205660+0000","last_change":"2023-11-24T06:11:04.197521+0000","last_active":"2023-11-24T06:11:04.205660+0000","last_peered":"2023-11-24T06:11:04.205660+0000","last_clean":"2023-11-24T06:11:04.205660+0000","last_became_active":"2023-11-24T06:11:04.194573+0000","last_became_peered":"2023-11-24T06:11:04.194573+0000","last_unstale":"2023-11-24T06:11:04.205660+0000","last_undegraded":"2023-11-24T06:11:04.205660+0000","last_fullsized":"2023-11-24T06:11:04.205660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:51.190432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199615+0000","last_change":"2023-11-24T06:11:04.197700+0000","last_active":"2023-11-24T06:11:04.199615+0000","last_peered":"2023-11-24T06:11:04.199615+0000","last_clean":"2023-11-24T06:11:04.199615+0000","last_became_active":"2023-11-24T06:11:04.197453+0000","last_became_peered":"2023-11-24T06:11:04.197453+0000","last_unstale":"2023-11-24T06:11:04.199615+0000","last_undegraded":"2023-11-24T06:11:04.199615+0000","last_fullsized":"2023-11-24T06:11:04.199615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:34.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.189970+0000","last_change":"2023-11-24T06:11:04.187922+0000","last_active":"2023-11-24T06:11:04.189970+0000","last_peered":"2023-11-24T06:11:04.189970+0000","last_clean":"2023-11-24T06:11:04.189970+0000","last_became_active":"2023-11-24T06:11:04.187686+0000","last_became_peered":"2023-11-24T06:11:04.187686+0000","last_unstale":"2023-11-24T06:11:04.189970+0000","last_undegraded":"2023-11-24T06:11:04.189970+0000","last_fullsized":"2023-11-24T06:11:04.189970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:11.765091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206081+0000","last_change":"2023-11-24T06:11:04.200673+0000","last_active":"2023-11-24T06:11:04.206081+0000","last_peered":"2023-11-24T06:11:04.206081+0000","last_clean":"2023-11-24T06:11:04.206081+0000","last_became_active":"2023-11-24T06:11:04.198364+0000","last_became_peered":"2023-11-24T06:11:04.198364+0000","last_unstale":"2023-11-24T06:11:04.206081+0000","last_undegraded":"2023-11-24T06:11:04.206081+0000","last_fullsized":"2023-11-24T06:11:04.206081+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:50:22.336183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195183+0000","last_change":"2023-11-24T06:11:04.190805+0000","last_active":"2023-11-24T06:11:04.195183+0000","last_peered":"2023-11-24T06:11:04.195183+0000","last_clean":"2023-11-24T06:11:04.195183+0000","last_became_active":"2023-11-24T06:11:04.190505+0000","last_became_peered":"2023-11-24T06:11:04.190505+0000","last_unstale":"2023-11-24T06:11:04.195183+0000","last_undegraded":"2023-11-24T06:11:04.195183+0000","last_fullsized":"2023-11-24T06:11:04.195183+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:26:52.488945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206183+0000","last_change":"2023-11-24T06:11:04.204576+0000","last_active":"2023-11-24T06:11:04.206183+0000","last_peered":"2023-11-24T06:11:04.206183+0000","last_clean":"2023-11-24T06:11:04.206183+0000","last_became_active":"2023-11-24T06:11:04.204173+0000","last_became_peered":"2023-11-24T06:11:04.204173+0000","last_unstale":"2023-11-24T06:11:04.206183+0000","last_undegraded":"2023-11-24T06:11:04.206183+0000","last_fullsized":"2023-11-24T06:11:04.206183+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:46:57.861583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201165+0000","last_change":"2023-11-24T06:11:04.199261+0000","last_active":"2023-11-24T06:11:04.201165+0000","last_peered":"2023-11-24T06:11:04.201165+0000","last_clean":"2023-11-24T06:11:04.201165+0000","last_became_active":"2023-11-24T06:11:04.199038+0000","last_became_peered":"2023-11-24T06:11:04.199038+0000","last_unstale":"2023-11-24T06:11:04.201165+0000","last_undegraded":"2023-11-24T06:11:04.201165+0000","last_fullsized":"2023-11-24T06:11:04.201165+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:33.509180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206164+0000","last_change":"2023-11-24T06:11:04.201669+0000","last_active":"2023-11-24T06:11:04.206164+0000","last_peered":"2023-11-24T06:11:04.206164+0000","last_clean":"2023-11-24T06:11:04.206164+0000","last_became_active":"2023-11-24T06:11:04.201399+0000","last_became_peered":"2023-11-24T06:11:04.201399+0000","last_unstale":"2023-11-24T06:11:04.206164+0000","last_undegraded":"2023-11-24T06:11:04.206164+0000","last_fullsized":"2023-11-24T06:11:04.206164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:15.448301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.182529+0000","last_change":"2023-11-24T06:11:04.181049+0000","last_active":"2023-11-24T06:11:04.182529+0000","last_peered":"2023-11-24T06:11:04.182529+0000","last_clean":"2023-11-24T06:11:04.182529+0000","last_became_active":"2023-11-24T06:11:04.180758+0000","last_became_peered":"2023-11-24T06:11:04.180758+0000","last_unstale":"2023-11-24T06:11:04.182529+0000","last_undegraded":"2023-11-24T06:11:04.182529+0000","last_fullsized":"2023-11-24T06:11:04.182529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:15.081903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185742+0000","last_change":"2023-11-24T06:11:04.183326+0000","last_active":"2023-11-24T06:11:04.185742+0000","last_peered":"2023-11-24T06:11:04.185742+0000","last_clean":"2023-11-24T06:11:04.185742+0000","last_became_active":"2023-11-24T06:11:04.183043+0000","last_became_peered":"2023-11-24T06:11:04.183043+0000","last_unstale":"2023-11-24T06:11:04.185742+0000","last_undegraded":"2023-11-24T06:11:04.185742+0000","last_fullsized":"2023-11-24T06:11:04.185742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:14.947513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201114+0000","last_change":"2023-11-24T06:11:04.193767+0000","last_active":"2023-11-24T06:11:04.201114+0000","last_peered":"2023-11-24T06:11:04.201114+0000","last_clean":"2023-11-24T06:11:04.201114+0000","last_became_active":"2023-11-24T06:11:04.193335+0000","last_became_peered":"2023-11-24T06:11:04.193335+0000","last_unstale":"2023-11-24T06:11:04.201114+0000","last_undegraded":"2023-11-24T06:11:04.201114+0000","last_fullsized":"2023-11-24T06:11:04.201114+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:06.583278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.187659+0000","last_change":"2023-11-24T06:11:04.184419+0000","last_active":"2023-11-24T06:11:04.187659+0000","last_peered":"2023-11-24T06:11:04.187659+0000","last_clean":"2023-11-24T06:11:04.187659+0000","last_became_active":"2023-11-24T06:11:04.183921+0000","last_became_peered":"2023-11-24T06:11:04.183921+0000","last_unstale":"2023-11-24T06:11:04.187659+0000","last_undegraded":"2023-11-24T06:11:04.187659+0000","last_fullsized":"2023-11-24T06:11:04.187659+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:41.209210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.203604+0000","last_change":"2023-11-24T06:11:04.193544+0000","last_active":"2023-11-24T06:11:04.203604+0000","last_peered":"2023-11-24T06:11:04.203604+0000","last_clean":"2023-11-24T06:11:04.203604+0000","last_became_active":"2023-11-24T06:11:04.193294+0000","last_became_peered":"2023-11-24T06:11:04.193294+0000","last_unstale":"2023-11-24T06:11:04.203604+0000","last_undegraded":"2023-11-24T06:11:04.203604+0000","last_fullsized":"2023-11-24T06:11:04.203604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:04.507852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195140+0000","last_change":"2023-11-24T06:11:04.188600+0000","last_active":"2023-11-24T06:11:04.195140+0000","last_peered":"2023-11-24T06:11:04.195140+0000","last_clean":"2023-11-24T06:11:04.195140+0000","last_became_active":"2023-11-24T06:11:04.186678+0000","last_became_peered":"2023-11-24T06:11:04.186678+0000","last_unstale":"2023-11-24T06:11:04.195140+0000","last_undegraded":"2023-11-24T06:11:04.195140+0000","last_fullsized":"2023-11-24T06:11:04.195140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:30.803405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.182469+0000","last_change":"2023-11-24T06:11:04.181028+0000","last_active":"2023-11-24T06:11:04.182469+0000","last_peered":"2023-11-24T06:11:04.182469+0000","last_clean":"2023-11-24T06:11:04.182469+0000","last_became_active":"2023-11-24T06:11:04.180720+0000","last_became_peered":"2023-11-24T06:11:04.180720+0000","last_unstale":"2023-11-24T06:11:04.182469+0000","last_undegraded":"2023-11-24T06:11:04.182469+0000","last_fullsized":"2023-11-24T06:11:04.182469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.466418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.208310+0000","last_change":"2023-11-24T06:11:04.201575+0000","last_active":"2023-11-24T06:11:04.208310+0000","last_peered":"2023-11-24T06:11:04.208310+0000","last_clean":"2023-11-24T06:11:04.208310+0000","last_became_active":"2023-11-24T06:11:04.201079+0000","last_became_peered":"2023-11-24T06:11:04.201079+0000","last_unstale":"2023-11-24T06:11:04.208310+0000","last_undegraded":"2023-11-24T06:11:04.208310+0000","last_fullsized":"2023-11-24T06:11:04.208310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:57:49.776816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197178+0000","last_change":"2023-11-24T06:11:04.191794+0000","last_active":"2023-11-24T06:11:04.197178+0000","last_peered":"2023-11-24T06:11:04.197178+0000","last_clean":"2023-11-24T06:11:04.197178+0000","last_became_active":"2023-11-24T06:11:04.187184+0000","last_became_peered":"2023-11-24T06:11:04.187184+0000","last_unstale":"2023-11-24T06:11:04.197178+0000","last_undegraded":"2023-11-24T06:11:04.197178+0000","last_fullsized":"2023-11-24T06:11:04.197178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:45.874095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206821+0000","last_change":"2023-11-24T06:11:03.184684+0000","last_active":"2023-11-24T06:11:04.206821+0000","last_peered":"2023-11-24T06:11:04.206821+0000","last_clean":"2023-11-24T06:11:04.206821+0000","last_became_active":"2023-11-24T06:11:03.184356+0000","last_became_peered":"2023-11-24T06:11:03.184356+0000","last_unstale":"2023-11-24T06:11:04.206821+0000","last_undegraded":"2023-11-24T06:11:04.206821+0000","last_fullsized":"2023-11-24T06:11:04.206821+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:14.780098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.198180+0000","last_change":"2023-11-24T06:11:04.189851+0000","last_active":"2023-11-24T06:11:04.198180+0000","last_peered":"2023-11-24T06:11:04.198180+0000","last_clean":"2023-11-24T06:11:04.198180+0000","last_became_active":"2023-11-24T06:11:04.189500+0000","last_became_peered":"2023-11-24T06:11:04.189500+0000","last_unstale":"2023-11-24T06:11:04.198180+0000","last_undegraded":"2023-11-24T06:11:04.198180+0000","last_fullsized":"2023-11-24T06:11:04.198180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:23.340839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206285+0000","last_change":"2023-11-24T06:11:03.186650+0000","last_active":"2023-11-24T06:11:04.206285+0000","last_peered":"2023-11-24T06:11:04.206285+0000","last_clean":"2023-11-24T06:11:04.206285+0000","last_became_active":"2023-11-24T06:11:03.186357+0000","last_became_peered":"2023-11-24T06:11:03.186357+0000","last_unstale":"2023-11-24T06:11:04.206285+0000","last_undegraded":"2023-11-24T06:11:04.206285+0000","last_fullsized":"2023-11-24T06:11:04.206285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:48.119927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197178+0000","last_change":"2023-11-24T06:11:04.192216+0000","last_active":"2023-11-24T06:11:04.197178+0000","last_peered":"2023-11-24T06:11:04.197178+0000","last_clean":"2023-11-24T06:11:04.197178+0000","last_became_active":"2023-11-24T06:11:04.187320+0000","last_became_peered":"2023-11-24T06:11:04.187320+0000","last_unstale":"2023-11-24T06:11:04.197178+0000","last_undegraded":"2023-11-24T06:11:04.197178+0000","last_fullsized":"2023-11-24T06:11:04.197178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:23.129399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201205+0000","last_change":"2023-11-24T06:11:03.182775+0000","last_active":"2023-11-24T06:11:04.201205+0000","last_peered":"2023-11-24T06:11:04.201205+0000","last_clean":"2023-11-24T06:11:04.201205+0000","last_became_active":"2023-11-24T06:11:03.182503+0000","last_became_peered":"2023-11-24T06:11:03.182503+0000","last_unstale":"2023-11-24T06:11:04.201205+0000","last_undegraded":"2023-11-24T06:11:04.201205+0000","last_fullsized":"2023-11-24T06:11:04.201205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:06.913576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201132+0000","last_change":"2023-11-24T06:11:04.193874+0000","last_active":"2023-11-24T06:11:04.201132+0000","last_peered":"2023-11-24T06:11:04.201132+0000","last_clean":"2023-11-24T06:11:04.201132+0000","last_became_active":"2023-11-24T06:11:04.193582+0000","last_became_peered":"2023-11-24T06:11:04.193582+0000","last_unstale":"2023-11-24T06:11:04.201132+0000","last_undegraded":"2023-11-24T06:11:04.201132+0000","last_fullsized":"2023-11-24T06:11:04.201132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:46.252673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.196039+0000","last_change":"2023-11-24T06:11:03.185909+0000","last_active":"2023-11-24T06:11:04.196039+0000","last_peered":"2023-11-24T06:11:04.196039+0000","last_clean":"2023-11-24T06:11:04.196039+0000","last_became_active":"2023-11-24T06:11:03.185627+0000","last_became_peered":"2023-11-24T06:11:03.185627+0000","last_unstale":"2023-11-24T06:11:04.196039+0000","last_undegraded":"2023-11-24T06:11:04.196039+0000","last_fullsized":"2023-11-24T06:11:04.196039+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:22.423162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206155+0000","last_change":"2023-11-24T06:11:04.204365+0000","last_active":"2023-11-24T06:11:04.206155+0000","last_peered":"2023-11-24T06:11:04.206155+0000","last_clean":"2023-11-24T06:11:04.206155+0000","last_became_active":"2023-11-24T06:11:04.203907+0000","last_became_peered":"2023-11-24T06:11:04.203907+0000","last_unstale":"2023-11-24T06:11:04.206155+0000","last_undegraded":"2023-11-24T06:11:04.206155+0000","last_fullsized":"2023-11-24T06:11:04.206155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:33:56.830325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183757+0000","last_change":"2023-11-24T06:11:03.180339+0000","last_active":"2023-11-24T06:11:04.183757+0000","last_peered":"2023-11-24T06:11:04.183757+0000","last_clean":"2023-11-24T06:11:04.183757+0000","last_became_active":"2023-11-24T06:11:03.180010+0000","last_became_peered":"2023-11-24T06:11:03.180010+0000","last_unstale":"2023-11-24T06:11:04.183757+0000","last_undegraded":"2023-11-24T06:11:04.183757+0000","last_fullsized":"2023-11-24T06:11:04.183757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:43.645339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205879+0000","last_change":"2023-11-24T06:11:04.200610+0000","last_active":"2023-11-24T06:11:04.205879+0000","last_peered":"2023-11-24T06:11:04.205879+0000","last_clean":"2023-11-24T06:11:04.205879+0000","last_became_active":"2023-11-24T06:11:04.198352+0000","last_became_peered":"2023-11-24T06:11:04.198352+0000","last_unstale":"2023-11-24T06:11:04.205879+0000","last_undegraded":"2023-11-24T06:11:04.205879+0000","last_fullsized":"2023-11-24T06:11:04.205879+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:57:49.776816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197514+0000","last_change":"2023-11-24T06:11:03.178694+0000","last_active":"2023-11-24T06:11:04.197514+0000","last_peered":"2023-11-24T06:11:04.197514+0000","last_clean":"2023-11-24T06:11:04.197514+0000","last_became_active":"2023-11-24T06:11:03.178418+0000","last_became_peered":"2023-11-24T06:11:03.178418+0000","last_unstale":"2023-11-24T06:11:04.197514+0000","last_undegraded":"2023-11-24T06:11:04.197514+0000","last_fullsized":"2023-11-24T06:11:04.197514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:33.280235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195741+0000","last_change":"2023-11-24T06:11:04.192942+0000","last_active":"2023-11-24T06:11:04.195741+0000","last_peered":"2023-11-24T06:11:04.195741+0000","last_clean":"2023-11-24T06:11:04.195741+0000","last_became_active":"2023-11-24T06:11:04.192648+0000","last_became_peered":"2023-11-24T06:11:04.192648+0000","last_unstale":"2023-11-24T06:11:04.195741+0000","last_undegraded":"2023-11-24T06:11:04.195741+0000","last_fullsized":"2023-11-24T06:11:04.195741+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:39.158782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176901+0000","last_change":"2023-11-24T06:11:03.184298+0000","last_active":"2023-11-24T06:11:04.176901+0000","last_peered":"2023-11-24T06:11:04.176901+0000","last_clean":"2023-11-24T06:11:04.176901+0000","last_became_active":"2023-11-24T06:11:03.183471+0000","last_became_peered":"2023-11-24T06:11:03.183471+0000","last_unstale":"2023-11-24T06:11:04.176901+0000","last_undegraded":"2023-11-24T06:11:04.176901+0000","last_fullsized":"2023-11-24T06:11:04.176901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:41:05.893832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205600+0000","last_change":"2023-11-24T06:11:04.196544+0000","last_active":"2023-11-24T06:11:04.205600+0000","last_peered":"2023-11-24T06:11:04.205600+0000","last_clean":"2023-11-24T06:11:04.205600+0000","last_became_active":"2023-11-24T06:11:04.194406+0000","last_became_peered":"2023-11-24T06:11:04.194406+0000","last_unstale":"2023-11-24T06:11:04.205600+0000","last_undegraded":"2023-11-24T06:11:04.205600+0000","last_fullsized":"2023-11-24T06:11:04.205600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:10.411069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.173470+0000","last_change":"2023-11-24T06:11:03.189506+0000","last_active":"2023-11-24T06:11:04.173470+0000","last_peered":"2023-11-24T06:11:04.173470+0000","last_clean":"2023-11-24T06:11:04.173470+0000","last_became_active":"2023-11-24T06:11:03.189223+0000","last_became_peered":"2023-11-24T06:11:03.189223+0000","last_unstale":"2023-11-24T06:11:04.173470+0000","last_undegraded":"2023-11-24T06:11:04.173470+0000","last_fullsized":"2023-11-24T06:11:04.173470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:59.937947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185298+0000","last_change":"2023-11-24T06:11:04.182876+0000","last_active":"2023-11-24T06:11:04.185298+0000","last_peered":"2023-11-24T06:11:04.185298+0000","last_clean":"2023-11-24T06:11:04.185298+0000","last_became_active":"2023-11-24T06:11:04.181971+0000","last_became_peered":"2023-11-24T06:11:04.181971+0000","last_unstale":"2023-11-24T06:11:04.185298+0000","last_undegraded":"2023-11-24T06:11:04.185298+0000","last_fullsized":"2023-11-24T06:11:04.185298+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:09.830311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.181381+0000","last_change":"2023-11-24T06:11:03.183311+0000","last_active":"2023-11-24T06:11:04.181381+0000","last_peered":"2023-11-24T06:11:04.181381+0000","last_clean":"2023-11-24T06:11:04.181381+0000","last_became_active":"2023-11-24T06:11:03.182809+0000","last_became_peered":"2023-11-24T06:11:03.182809+0000","last_unstale":"2023-11-24T06:11:04.181381+0000","last_undegraded":"2023-11-24T06:11:04.181381+0000","last_fullsized":"2023-11-24T06:11:04.181381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:59.937947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199596+0000","last_change":"2023-11-24T06:11:04.197502+0000","last_active":"2023-11-24T06:11:04.199596+0000","last_peered":"2023-11-24T06:11:04.199596+0000","last_clean":"2023-11-24T06:11:04.199596+0000","last_became_active":"2023-11-24T06:11:04.197194+0000","last_became_peered":"2023-11-24T06:11:04.197194+0000","last_unstale":"2023-11-24T06:11:04.199596+0000","last_undegraded":"2023-11-24T06:11:04.199596+0000","last_fullsized":"2023-11-24T06:11:04.199596+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:41:06.897965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199837+0000","last_change":"2023-11-24T06:11:03.186637+0000","last_active":"2023-11-24T06:11:04.199837+0000","last_peered":"2023-11-24T06:11:04.199837+0000","last_clean":"2023-11-24T06:11:04.199837+0000","last_became_active":"2023-11-24T06:11:03.186332+0000","last_became_peered":"2023-11-24T06:11:03.186332+0000","last_unstale":"2023-11-24T06:11:04.199837+0000","last_undegraded":"2023-11-24T06:11:04.199837+0000","last_fullsized":"2023-11-24T06:11:04.199837+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:55.994309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185689+0000","last_change":"2023-11-24T06:11:04.184915+0000","last_active":"2023-11-24T06:11:04.185689+0000","last_peered":"2023-11-24T06:11:04.185689+0000","last_clean":"2023-11-24T06:11:04.185689+0000","last_became_active":"2023-11-24T06:11:04.184674+0000","last_became_peered":"2023-11-24T06:11:04.184674+0000","last_unstale":"2023-11-24T06:11:04.185689+0000","last_undegraded":"2023-11-24T06:11:04.185689+0000","last_fullsized":"2023-11-24T06:11:04.185689+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:01:41.115525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199459+0000","last_change":"2023-11-24T06:11:03.178337+0000","last_active":"2023-11-24T06:11:04.199459+0000","last_peered":"2023-11-24T06:11:04.199459+0000","last_clean":"2023-11-24T06:11:04.199459+0000","last_became_active":"2023-11-24T06:11:03.177980+0000","last_became_peered":"2023-11-24T06:11:03.177980+0000","last_unstale":"2023-11-24T06:11:04.199459+0000","last_undegraded":"2023-11-24T06:11:04.199459+0000","last_fullsized":"2023-11-24T06:11:04.199459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:22.423162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195409+0000","last_change":"2023-11-24T06:11:04.190817+0000","last_active":"2023-11-24T06:11:04.195409+0000","last_peered":"2023-11-24T06:11:04.195409+0000","last_clean":"2023-11-24T06:11:04.195409+0000","last_became_active":"2023-11-24T06:11:04.190482+0000","last_became_peered":"2023-11-24T06:11:04.190482+0000","last_unstale":"2023-11-24T06:11:04.195409+0000","last_undegraded":"2023-11-24T06:11:04.195409+0000","last_fullsized":"2023-11-24T06:11:04.195409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:03.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.208532+0000","last_change":"2023-11-24T06:11:03.188014+0000","last_active":"2023-11-24T06:11:04.208532+0000","last_peered":"2023-11-24T06:11:04.208532+0000","last_clean":"2023-11-24T06:11:04.208532+0000","last_became_active":"2023-11-24T06:11:03.187732+0000","last_became_peered":"2023-11-24T06:11:03.187732+0000","last_unstale":"2023-11-24T06:11:04.208532+0000","last_undegraded":"2023-11-24T06:11:04.208532+0000","last_fullsized":"2023-11-24T06:11:04.208532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:02.323123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183357+0000","last_change":"2023-11-24T06:11:04.180194+0000","last_active":"2023-11-24T06:11:04.183357+0000","last_peered":"2023-11-24T06:11:04.183357+0000","last_clean":"2023-11-24T06:11:04.183357+0000","last_became_active":"2023-11-24T06:11:04.179893+0000","last_became_peered":"2023-11-24T06:11:04.179893+0000","last_unstale":"2023-11-24T06:11:04.183357+0000","last_undegraded":"2023-11-24T06:11:04.183357+0000","last_fullsized":"2023-11-24T06:11:04.183357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:06.446082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206246+0000","last_change":"2023-11-24T06:11:03.186245+0000","last_active":"2023-11-24T06:11:04.206246+0000","last_peered":"2023-11-24T06:11:04.206246+0000","last_clean":"2023-11-24T06:11:04.206246+0000","last_became_active":"2023-11-24T06:11:03.185943+0000","last_became_peered":"2023-11-24T06:11:03.185943+0000","last_unstale":"2023-11-24T06:11:04.206246+0000","last_undegraded":"2023-11-24T06:11:04.206246+0000","last_fullsized":"2023-11-24T06:11:04.206246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:58.064530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195164+0000","last_change":"2023-11-24T06:11:04.189331+0000","last_active":"2023-11-24T06:11:04.195164+0000","last_peered":"2023-11-24T06:11:04.195164+0000","last_clean":"2023-11-24T06:11:04.195164+0000","last_became_active":"2023-11-24T06:11:04.189023+0000","last_became_peered":"2023-11-24T06:11:04.189023+0000","last_unstale":"2023-11-24T06:11:04.195164+0000","last_undegraded":"2023-11-24T06:11:04.195164+0000","last_fullsized":"2023-11-24T06:11:04.195164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:59.683946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.179467+0000","last_change":"2023-11-24T06:11:03.177059+0000","last_active":"2023-11-24T06:11:04.179467+0000","last_peered":"2023-11-24T06:11:04.179467+0000","last_clean":"2023-11-24T06:11:04.179467+0000","last_became_active":"2023-11-24T06:11:03.176749+0000","last_became_peered":"2023-11-24T06:11:03.176749+0000","last_unstale":"2023-11-24T06:11:04.179467+0000","last_undegraded":"2023-11-24T06:11:04.179467+0000","last_fullsized":"2023-11-24T06:11:04.179467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:52:14.402838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.191832+0000","last_change":"2023-11-24T06:11:04.189779+0000","last_active":"2023-11-24T06:11:04.191832+0000","last_peered":"2023-11-24T06:11:04.191832+0000","last_clean":"2023-11-24T06:11:04.191832+0000","last_became_active":"2023-11-24T06:11:04.189582+0000","last_became_peered":"2023-11-24T06:11:04.189582+0000","last_unstale":"2023-11-24T06:11:04.191832+0000","last_undegraded":"2023-11-24T06:11:04.191832+0000","last_fullsized":"2023-11-24T06:11:04.191832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:59.683946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.177011+0000","last_change":"2023-11-24T06:11:03.184703+0000","last_active":"2023-11-24T06:11:04.177011+0000","last_peered":"2023-11-24T06:11:04.177011+0000","last_clean":"2023-11-24T06:11:04.177011+0000","last_became_active":"2023-11-24T06:11:03.183731+0000","last_became_peered":"2023-11-24T06:11:03.183731+0000","last_unstale":"2023-11-24T06:11:04.177011+0000","last_undegraded":"2023-11-24T06:11:04.177011+0000","last_fullsized":"2023-11-24T06:11:04.177011+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:33.516342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206085+0000","last_change":"2023-11-24T06:11:04.199456+0000","last_active":"2023-11-24T06:11:04.206085+0000","last_peered":"2023-11-24T06:11:04.206085+0000","last_clean":"2023-11-24T06:11:04.206085+0000","last_became_active":"2023-11-24T06:11:04.199084+0000","last_became_peered":"2023-11-24T06:11:04.199084+0000","last_unstale":"2023-11-24T06:11:04.206085+0000","last_undegraded":"2023-11-24T06:11:04.206085+0000","last_fullsized":"2023-11-24T06:11:04.206085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:44.649472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.203882+0000","last_change":"2023-11-24T06:11:03.182763+0000","last_active":"2023-11-24T06:11:04.203882+0000","last_peered":"2023-11-24T06:11:04.203882+0000","last_clean":"2023-11-24T06:11:04.203882+0000","last_became_active":"2023-11-24T06:11:03.182496+0000","last_became_peered":"2023-11-24T06:11:03.182496+0000","last_unstale":"2023-11-24T06:11:04.203882+0000","last_undegraded":"2023-11-24T06:11:04.203882+0000","last_fullsized":"2023-11-24T06:11:04.203882+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:08.826178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.198249+0000","last_change":"2023-11-24T06:11:04.192044+0000","last_active":"2023-11-24T06:11:04.198249+0000","last_peered":"2023-11-24T06:11:04.198249+0000","last_clean":"2023-11-24T06:11:04.198249+0000","last_became_active":"2023-11-24T06:11:04.191633+0000","last_became_peered":"2023-11-24T06:11:04.191633+0000","last_unstale":"2023-11-24T06:11:04.198249+0000","last_undegraded":"2023-11-24T06:11:04.198249+0000","last_fullsized":"2023-11-24T06:11:04.198249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:06.583278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206172+0000","last_change":"2023-11-24T06:11:03.187454+0000","last_active":"2023-11-24T06:11:04.206172+0000","last_peered":"2023-11-24T06:11:04.206172+0000","last_clean":"2023-11-24T06:11:04.206172+0000","last_became_active":"2023-11-24T06:11:03.187186+0000","last_became_peered":"2023-11-24T06:11:03.187186+0000","last_unstale":"2023-11-24T06:11:04.206172+0000","last_undegraded":"2023-11-24T06:11:04.206172+0000","last_fullsized":"2023-11-24T06:11:04.206172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:50:59.733135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183914+0000","last_change":"2023-11-24T06:11:04.182314+0000","last_active":"2023-11-24T06:11:04.183914+0000","last_peered":"2023-11-24T06:11:04.183914+0000","last_clean":"2023-11-24T06:11:04.183914+0000","last_became_active":"2023-11-24T06:11:04.181877+0000","last_became_peered":"2023-11-24T06:11:04.181877+0000","last_unstale":"2023-11-24T06:11:04.183914+0000","last_undegraded":"2023-11-24T06:11:04.183914+0000","last_fullsized":"2023-11-24T06:11:04.183914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:59.683946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199552+0000","last_change":"2023-11-24T06:11:03.187037+0000","last_active":"2023-11-24T06:11:04.199552+0000","last_peered":"2023-11-24T06:11:04.199552+0000","last_clean":"2023-11-24T06:11:04.199552+0000","last_became_active":"2023-11-24T06:11:03.186748+0000","last_became_peered":"2023-11-24T06:11:03.186748+0000","last_unstale":"2023-11-24T06:11:04.199552+0000","last_undegraded":"2023-11-24T06:11:04.199552+0000","last_fullsized":"2023-11-24T06:11:04.199552+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:33.516342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.194622+0000","last_change":"2023-11-24T06:11:04.184957+0000","last_active":"2023-11-24T06:11:04.194622+0000","last_peered":"2023-11-24T06:11:04.194622+0000","last_clean":"2023-11-24T06:11:04.194622+0000","last_became_active":"2023-11-24T06:11:04.184657+0000","last_became_peered":"2023-11-24T06:11:04.184657+0000","last_unstale":"2023-11-24T06:11:04.194622+0000","last_undegraded":"2023-11-24T06:11:04.194622+0000","last_fullsized":"2023-11-24T06:11:04.194622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:15.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195900+0000","last_change":"2023-11-24T06:11:03.185867+0000","last_active":"2023-11-24T06:11:04.195900+0000","last_peered":"2023-11-24T06:11:04.195900+0000","last_clean":"2023-11-24T06:11:04.195900+0000","last_became_active":"2023-11-24T06:11:03.185518+0000","last_became_peered":"2023-11-24T06:11:03.185518+0000","last_unstale":"2023-11-24T06:11:04.195900+0000","last_undegraded":"2023-11-24T06:11:04.195900+0000","last_fullsized":"2023-11-24T06:11:04.195900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:53.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206099+0000","last_change":"2023-11-24T06:11:04.199502+0000","last_active":"2023-11-24T06:11:04.206099+0000","last_peered":"2023-11-24T06:11:04.206099+0000","last_clean":"2023-11-24T06:11:04.206099+0000","last_became_active":"2023-11-24T06:11:04.198901+0000","last_became_peered":"2023-11-24T06:11:04.198901+0000","last_unstale":"2023-11-24T06:11:04.206099+0000","last_undegraded":"2023-11-24T06:11:04.206099+0000","last_fullsized":"2023-11-24T06:11:04.206099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:54.702175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.170479+0000","last_change":"2023-11-24T06:11:03.188060+0000","last_active":"2023-11-24T06:11:04.170479+0000","last_peered":"2023-11-24T06:11:04.170479+0000","last_clean":"2023-11-24T06:11:04.170479+0000","last_became_active":"2023-11-24T06:11:03.187761+0000","last_became_peered":"2023-11-24T06:11:03.187761+0000","last_unstale":"2023-11-24T06:11:04.170479+0000","last_undegraded":"2023-11-24T06:11:04.170479+0000","last_fullsized":"2023-11-24T06:11:04.170479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:32.505047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183378+0000","last_change":"2023-11-24T06:11:04.180281+0000","last_active":"2023-11-24T06:11:04.183378+0000","last_peered":"2023-11-24T06:11:04.183378+0000","last_clean":"2023-11-24T06:11:04.183378+0000","last_became_active":"2023-11-24T06:11:04.179961+0000","last_became_peered":"2023-11-24T06:11:04.179961+0000","last_unstale":"2023-11-24T06:11:04.183378+0000","last_undegraded":"2023-11-24T06:11:04.183378+0000","last_fullsized":"2023-11-24T06:11:04.183378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:00.942080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206879+0000","last_change":"2023-11-24T06:11:03.186597+0000","last_active":"2023-11-24T06:11:04.206879+0000","last_peered":"2023-11-24T06:11:04.206879+0000","last_clean":"2023-11-24T06:11:04.206879+0000","last_became_active":"2023-11-24T06:11:03.186316+0000","last_became_peered":"2023-11-24T06:11:03.186316+0000","last_unstale":"2023-11-24T06:11:04.206879+0000","last_undegraded":"2023-11-24T06:11:04.206879+0000","last_fullsized":"2023-11-24T06:11:04.206879+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:13:54.640391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197747+0000","last_change":"2023-11-24T06:11:04.193045+0000","last_active":"2023-11-24T06:11:04.197747+0000","last_peered":"2023-11-24T06:11:04.197747+0000","last_clean":"2023-11-24T06:11:04.197747+0000","last_became_active":"2023-11-24T06:11:04.192792+0000","last_became_peered":"2023-11-24T06:11:04.192792+0000","last_unstale":"2023-11-24T06:11:04.197747+0000","last_undegraded":"2023-11-24T06:11:04.197747+0000","last_fullsized":"2023-11-24T06:11:04.197747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:06:33.622081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176376+0000","last_change":"2023-11-24T06:11:03.180198+0000","last_active":"2023-11-24T06:11:04.176376+0000","last_peered":"2023-11-24T06:11:04.176376+0000","last_clean":"2023-11-24T06:11:04.176376+0000","last_became_active":"2023-11-24T06:11:03.179329+0000","last_became_peered":"2023-11-24T06:11:03.179329+0000","last_unstale":"2023-11-24T06:11:04.176376+0000","last_undegraded":"2023-11-24T06:11:04.176376+0000","last_fullsized":"2023-11-24T06:11:04.176376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:42.310119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197187+0000","last_change":"2023-11-24T06:11:04.191954+0000","last_active":"2023-11-24T06:11:04.197187+0000","last_peered":"2023-11-24T06:11:04.197187+0000","last_clean":"2023-11-24T06:11:04.197187+0000","last_became_active":"2023-11-24T06:11:04.191670+0000","last_became_peered":"2023-11-24T06:11:04.191670+0000","last_unstale":"2023-11-24T06:11:04.197187+0000","last_undegraded":"2023-11-24T06:11:04.197187+0000","last_fullsized":"2023-11-24T06:11:04.197187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:58.121927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176921+0000","last_change":"2023-11-24T06:11:03.191521+0000","last_active":"2023-11-24T06:11:04.176921+0000","last_peered":"2023-11-24T06:11:04.176921+0000","last_clean":"2023-11-24T06:11:04.176921+0000","last_became_active":"2023-11-24T06:11:03.191264+0000","last_became_peered":"2023-11-24T06:11:03.191264+0000","last_unstale":"2023-11-24T06:11:04.176921+0000","last_undegraded":"2023-11-24T06:11:04.176921+0000","last_fullsized":"2023-11-24T06:11:04.176921+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:05.579145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183458+0000","last_change":"2023-11-24T06:11:04.181054+0000","last_active":"2023-11-24T06:11:04.183458+0000","last_peered":"2023-11-24T06:11:04.183458+0000","last_clean":"2023-11-24T06:11:04.183458+0000","last_became_active":"2023-11-24T06:11:04.180796+0000","last_became_peered":"2023-11-24T06:11:04.180796+0000","last_unstale":"2023-11-24T06:11:04.183458+0000","last_undegraded":"2023-11-24T06:11:04.183458+0000","last_fullsized":"2023-11-24T06:11:04.183458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:09.830311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.203886+0000","last_change":"2023-11-24T06:11:03.190074+0000","last_active":"2023-11-24T06:11:04.203886+0000","last_peered":"2023-11-24T06:11:04.203886+0000","last_clean":"2023-11-24T06:11:04.203886+0000","last_became_active":"2023-11-24T06:11:03.189744+0000","last_became_peered":"2023-11-24T06:11:03.189744+0000","last_unstale":"2023-11-24T06:11:04.203886+0000","last_undegraded":"2023-11-24T06:11:04.203886+0000","last_fullsized":"2023-11-24T06:11:04.203886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:52:14.402838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.188726+0000","last_change":"2023-11-24T06:11:04.187442+0000","last_active":"2023-11-24T06:11:04.188726+0000","last_peered":"2023-11-24T06:11:04.188726+0000","last_clean":"2023-11-24T06:11:04.188726+0000","last_became_active":"2023-11-24T06:11:04.187162+0000","last_became_peered":"2023-11-24T06:11:04.187162+0000","last_unstale":"2023-11-24T06:11:04.188726+0000","last_undegraded":"2023-11-24T06:11:04.188726+0000","last_fullsized":"2023-11-24T06:11:04.188726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:30.803405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.170466+0000","last_change":"2023-11-24T06:11:03.181114+0000","last_active":"2023-11-24T06:11:04.170466+0000","last_peered":"2023-11-24T06:11:04.170466+0000","last_clean":"2023-11-24T06:11:04.170466+0000","last_became_active":"2023-11-24T06:11:03.180828+0000","last_became_peered":"2023-11-24T06:11:03.180828+0000","last_unstale":"2023-11-24T06:11:04.170466+0000","last_undegraded":"2023-11-24T06:11:04.170466+0000","last_fullsized":"2023-11-24T06:11:04.170466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:22.423162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.192661+0000","last_change":"2023-11-24T06:11:04.185379+0000","last_active":"2023-11-24T06:11:04.192661+0000","last_peered":"2023-11-24T06:11:04.192661+0000","last_clean":"2023-11-24T06:11:04.192661+0000","last_became_active":"2023-11-24T06:11:04.185031+0000","last_became_peered":"2023-11-24T06:11:04.185031+0000","last_unstale":"2023-11-24T06:11:04.192661+0000","last_undegraded":"2023-11-24T06:11:04.192661+0000","last_fullsized":"2023-11-24T06:11:04.192661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:52.004170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.182798+0000","last_change":"2023-11-24T06:11:03.177439+0000","last_active":"2023-11-24T06:11:04.182798+0000","last_peered":"2023-11-24T06:11:04.182798+0000","last_clean":"2023-11-24T06:11:04.182798+0000","last_became_active":"2023-11-24T06:11:03.177076+0000","last_became_peered":"2023-11-24T06:11:03.177076+0000","last_unstale":"2023-11-24T06:11:04.182798+0000","last_undegraded":"2023-11-24T06:11:04.182798+0000","last_fullsized":"2023-11-24T06:11:04.182798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:37:16.886042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201093+0000","last_change":"2023-11-24T06:11:04.193602+0000","last_active":"2023-11-24T06:11:04.201093+0000","last_peered":"2023-11-24T06:11:04.201093+0000","last_clean":"2023-11-24T06:11:04.201093+0000","last_became_active":"2023-11-24T06:11:04.193316+0000","last_became_peered":"2023-11-24T06:11:04.193316+0000","last_unstale":"2023-11-24T06:11:04.201093+0000","last_undegraded":"2023-11-24T06:11:04.201093+0000","last_fullsized":"2023-11-24T06:11:04.201093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:13.777194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.177260+0000","last_change":"2023-11-24T06:11:03.183951+0000","last_active":"2023-11-24T06:11:04.177260+0000","last_peered":"2023-11-24T06:11:04.177260+0000","last_clean":"2023-11-24T06:11:04.177260+0000","last_became_active":"2023-11-24T06:11:03.183689+0000","last_became_peered":"2023-11-24T06:11:03.183689+0000","last_unstale":"2023-11-24T06:11:04.177260+0000","last_undegraded":"2023-11-24T06:11:04.177260+0000","last_fullsized":"2023-11-24T06:11:04.177260+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:32.505047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206100+0000","last_change":"2023-11-24T06:11:04.203686+0000","last_active":"2023-11-24T06:11:04.206100+0000","last_peered":"2023-11-24T06:11:04.206100+0000","last_clean":"2023-11-24T06:11:04.206100+0000","last_became_active":"2023-11-24T06:11:04.203407+0000","last_became_peered":"2023-11-24T06:11:04.203407+0000","last_unstale":"2023-11-24T06:11:04.206100+0000","last_undegraded":"2023-11-24T06:11:04.206100+0000","last_fullsized":"2023-11-24T06:11:04.206100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:57:37.335707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176465+0000","last_change":"2023-11-24T06:11:03.185143+0000","last_active":"2023-11-24T06:11:04.176465+0000","last_peered":"2023-11-24T06:11:04.176465+0000","last_clean":"2023-11-24T06:11:04.176465+0000","last_became_active":"2023-11-24T06:11:03.184866+0000","last_became_peered":"2023-11-24T06:11:03.184866+0000","last_unstale":"2023-11-24T06:11:04.176465+0000","last_undegraded":"2023-11-24T06:11:04.176465+0000","last_fullsized":"2023-11-24T06:11:04.176465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:25.278280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199557+0000","last_change":"2023-11-24T06:11:04.194739+0000","last_active":"2023-11-24T06:11:04.199557+0000","last_peered":"2023-11-24T06:11:04.199557+0000","last_clean":"2023-11-24T06:11:04.199557+0000","last_became_active":"2023-11-24T06:11:04.194501+0000","last_became_peered":"2023-11-24T06:11:04.194501+0000","last_unstale":"2023-11-24T06:11:04.199557+0000","last_undegraded":"2023-11-24T06:11:04.199557+0000","last_fullsized":"2023-11-24T06:11:04.199557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:06.446082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.173074+0000","last_change":"2023-11-24T06:11:03.188018+0000","last_active":"2023-11-24T06:11:04.173074+0000","last_peered":"2023-11-24T06:11:04.173074+0000","last_clean":"2023-11-24T06:11:04.173074+0000","last_became_active":"2023-11-24T06:11:03.187777+0000","last_became_peered":"2023-11-24T06:11:03.187777+0000","last_unstale":"2023-11-24T06:11:04.173074+0000","last_undegraded":"2023-11-24T06:11:04.173074+0000","last_fullsized":"2023-11-24T06:11:04.173074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:05.441949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.196751+0000","last_change":"2023-11-24T06:11:04.185798+0000","last_active":"2023-11-24T06:11:04.196751+0000","last_peered":"2023-11-24T06:11:04.196751+0000","last_clean":"2023-11-24T06:11:04.196751+0000","last_became_active":"2023-11-24T06:11:04.185535+0000","last_became_peered":"2023-11-24T06:11:04.185535+0000","last_unstale":"2023-11-24T06:11:04.196751+0000","last_undegraded":"2023-11-24T06:11:04.196751+0000","last_fullsized":"2023-11-24T06:11:04.196751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:37.567668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176942+0000","last_change":"2023-11-24T06:11:03.183029+0000","last_active":"2023-11-24T06:11:04.176942+0000","last_peered":"2023-11-24T06:11:04.176942+0000","last_clean":"2023-11-24T06:11:04.176942+0000","last_became_active":"2023-11-24T06:11:03.182745+0000","last_became_peered":"2023-11-24T06:11:03.182745+0000","last_unstale":"2023-11-24T06:11:04.176942+0000","last_undegraded":"2023-11-24T06:11:04.176942+0000","last_fullsized":"2023-11-24T06:11:04.176942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:38.154648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.194175+0000","last_change":"2023-11-24T06:11:04.188063+0000","last_active":"2023-11-24T06:11:04.194175+0000","last_peered":"2023-11-24T06:11:04.194175+0000","last_clean":"2023-11-24T06:11:04.194175+0000","last_became_active":"2023-11-24T06:11:04.187865+0000","last_became_peered":"2023-11-24T06:11:04.187865+0000","last_unstale":"2023-11-24T06:11:04.194175+0000","last_undegraded":"2023-11-24T06:11:04.194175+0000","last_fullsized":"2023-11-24T06:11:04.194175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:23.546146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.202571+0000","last_change":"2023-11-24T06:11:03.188747+0000","last_active":"2023-11-24T06:11:04.202571+0000","last_peered":"2023-11-24T06:11:04.202571+0000","last_clean":"2023-11-24T06:11:04.202571+0000","last_became_active":"2023-11-24T06:11:03.188526+0000","last_became_peered":"2023-11-24T06:11:03.188526+0000","last_unstale":"2023-11-24T06:11:04.202571+0000","last_undegraded":"2023-11-24T06:11:04.202571+0000","last_fullsized":"2023-11-24T06:11:04.202571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:04:29.610240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205914+0000","last_change":"2023-11-24T06:11:04.202133+0000","last_active":"2023-11-24T06:11:04.205914+0000","last_peered":"2023-11-24T06:11:04.205914+0000","last_clean":"2023-11-24T06:11:04.205914+0000","last_became_active":"2023-11-24T06:11:04.200380+0000","last_became_peered":"2023-11-24T06:11:04.200380+0000","last_unstale":"2023-11-24T06:11:04.205914+0000","last_undegraded":"2023-11-24T06:11:04.205914+0000","last_fullsized":"2023-11-24T06:11:04.205914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:51:04.213597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183749+0000","last_change":"2023-11-24T06:11:03.182919+0000","last_active":"2023-11-24T06:11:04.183749+0000","last_peered":"2023-11-24T06:11:04.183749+0000","last_clean":"2023-11-24T06:11:04.183749+0000","last_became_active":"2023-11-24T06:11:03.182470+0000","last_became_peered":"2023-11-24T06:11:03.182470+0000","last_unstale":"2023-11-24T06:11:04.183749+0000","last_undegraded":"2023-11-24T06:11:04.183749+0000","last_fullsized":"2023-11-24T06:11:04.183749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:08.706979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":198,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:11:03.170894+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:11:03.170894+0000","last_peered":"2023-11-24T06:11:03.170894+0000","last_clean":"2023-11-24T06:11:03.170894+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:11:03.170894+0000","last_undegraded":"2023-11-24T06:11:03.170894+0000","last_fullsized":"2023-11-24T06:11:03.170894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28304,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343536,"statfs":{"total":96636764160,"available":96607780864,"internally_reserved":0,"allocated":1310720,"data_stored":887248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.87,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68000000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.377}]},{"osd":5,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.373,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.41799999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.76800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34200000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.83099999999999996,"15min":0.88900000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.621,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":1.371,"15min":1.371},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66100000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.83099999999999996,"15min":0.875},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70299999999999996,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":5.2830000000000004,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70199999999999996,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71599999999999997,"5min":1.407,"15min":1.407},"last":0.628}]},{"osd":4,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78400000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.33600000000000002,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45100000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.25900000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.48599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47399999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.375,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.52600000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.434},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.497,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71299999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.753}]},{"osd":6,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.46800000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.78100000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69599999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81100000000000005,"5min":0.88,"15min":0.88},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.75700000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.79200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1290240,"data_stored":880313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.72599999999999998,"15min":0.79200000000000004},"min":{"1min":0.224,"5min":0.216,"15min":0.216},"max":{"1min":0.46100000000000002,"5min":12.378,"15min":12.378},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58799999999999997,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.501,"15min":0.52000000000000002},"min":{"1min":0.26200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.54300000000000004,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59899999999999998,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.41099999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.0920000000000001,"15min":1.169},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97499999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83099999999999996,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93300000000000005,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.621,"5min":1.091,"15min":1.169},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0289999999999999,"5min":15.356,"15min":15.356},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.72899999999999998,"15min":0.75},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83999999999999997,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.752,"15min":0.77400000000000002},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":4.726,"15min":4.726},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.755,"15min":0.77600000000000002},"min":{"1min":0.45100000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":1.129,"15min":1.2090000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":15.388,"15min":15.388},"last":0.69499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.214},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33500000000000002,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.49299999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.251}]},{"osd":1,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.25900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.38900000000000001,"15min":0.39200000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56299999999999994,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.29199999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72899999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.42299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0580000000000001,"5min":1.173,"15min":1.173},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99099999999999999,"5min":1.339,"15min":1.339},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":0.91100000000000003,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92500000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.018,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.502}]},{"osd":7,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.079,"5min":1.405,"15min":1.405},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95399999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.52100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":296776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.33600000000000002,"15min":0.34100000000000003},"min":{"1min":0.23799999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.42299999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.39400000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.28399999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.432,"5min":1.032,"15min":1.032},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.47899999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.52100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40100000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90100000000000002,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.878,"5min":1.728,"15min":1.728},"last":0.59699999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88500000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.161,"5min":1.222,"15min":1.222},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.499,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63300000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.39200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.79500000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.436},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.497},{"interface":"front","average":{"1min":0.626,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89800000000000002,"5min":3.335,"15min":3.335},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.628}]},{"osd":1,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81299999999999994,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77600000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72299999999999998,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76200000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71899999999999997,"15min":0.73999999999999999},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.80700000000000005,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79300000000000004,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.503}]},{"osd":4,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76200000000000001,"5min":1.536,"15min":1.536},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.59999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82899999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.39400000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77300000000000002,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.42899999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80900000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.35999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:11:08.304 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:11:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.454+0000 7f821cd1d700 1 Processor -- start 2023-11-24T06:11:08.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.454+0000 7f821cd1d700 1 -- start start 2023-11-24T06:11:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f821cd1d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f821806c710 0x7f82180b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f821cd1d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82180af4d0 0x7f82180afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f821cd1d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82180ae430 con 0x7f821815e080 2023-11-24T06:11:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f821cd1d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82180af0b0 con 0x7f821806c710 2023-11-24T06:11:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f821cd1d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82180b05a0 con 0x7f82180af4d0 2023-11-24T06:11:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f8215d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f821806c710 0x7f82180b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f821659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f821815e080 0x7f821806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41582/0 (socket says 172.21.15.97:41582) 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f821659c700 1 -- 172.21.15.97:0/3816931116 learned_addr learned my addr 172.21.15.97:0/3816931116 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f8216d9d700 1 --2- 172.21.15.97:0/3816931116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82180af4d0 0x7f82180afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f8215d9b700 1 -- 172.21.15.97:0/3816931116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82180af4d0 msgr2=0x7f82180afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f8215d9b700 1 --2- 172.21.15.97:0/3816931116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82180af4d0 0x7f82180afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f8215d9b700 1 -- 172.21.15.97:0/3816931116 >> v1:172.21.15.97:6789/0 conn(0x7f821815e080 legacy=0x7f821806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f8215d9b700 1 -- 172.21.15.97:0/3816931116 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82180ae710 con 0x7f821806c710 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f821559a700 1 -- 172.21.15.97:0/3816931116 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758878773 0 0) 0x7f82180ae430 con 0x7f821815e080 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f8216d9d700 1 --2- 172.21.15.97:0/3816931116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82180af4d0 0x7f82180afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.455+0000 7f8215d9b700 1 --2- 172.21.15.97:0/3816931116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821806c710 0x7f82180b5050 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f820c009840 tx=0x7f820c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd9f66be01ade77 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821559a700 1 -- 172.21.15.97:0/3816931116 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c011830 con 0x7f821806c710 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821559a700 1 -- 172.21.15.97:0/3816931116 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c011990 con 0x7f821806c710 2023-11-24T06:11:08.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821559a700 1 -- 172.21.15.97:0/3816931116 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c011d20 con 0x7f821806c710 2023-11-24T06:11:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821cd1d700 1 -- 172.21.15.97:0/3816931116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821806c710 msgr2=0x7f82180b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821cd1d700 1 --2- 172.21.15.97:0/3816931116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821806c710 0x7f82180b5050 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f820c009840 tx=0x7f820c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821cd1d700 1 -- 172.21.15.97:0/3816931116 shutdown_connections 2023-11-24T06:11:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821cd1d700 1 --2- 172.21.15.97:0/3816931116 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82180af4d0 0x7f82180afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821cd1d700 1 --2- 172.21.15.97:0/3816931116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821806c710 0x7f82180b5050 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821cd1d700 1 -- 172.21.15.97:0/3816931116 >> 172.21.15.97:0/3816931116 conn(0x7f8218071b90 msgr2=0x7f8218071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821cd1d700 1 -- 172.21.15.97:0/3816931116 shutdown_connections 2023-11-24T06:11:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.456+0000 7f821cd1d700 1 -- 172.21.15.97:0/3816931116 wait complete. 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f821cd1d700 1 Processor -- start 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f821cd1d700 1 -- start start 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f821cd1d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f821806c710 0x7f8218158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f821cd1d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82180af4d0 0x7f82180d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f821cd1d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821815e080 0x7f821814a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f821cd1d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82180d0620 con 0x7f821815e080 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f821cd1d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82181586b0 con 0x7f82180af4d0 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f821cd1d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8218158990 con 0x7f821806c710 2023-11-24T06:11:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f8216d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821815e080 0x7f821814a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f8216d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821815e080 0x7f821814a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49140/0 (socket says 172.21.15.97:49140) 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.458+0000 7f8216d9d700 1 -- 172.21.15.97:0/2936357298 learned_addr learned my addr 172.21.15.97:0/2936357298 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f821659c700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f821806c710 0x7f8218158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8215d9b700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82180af4d0 0x7f82180d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8216d9d700 1 -- 172.21.15.97:0/2936357298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f821806c710 msgr2=0x7f8218158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8216d9d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f821806c710 0x7f8218158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8216d9d700 1 -- 172.21.15.97:0/2936357298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82180af4d0 msgr2=0x7f82180d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8216d9d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82180af4d0 0x7f82180d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8216d9d700 1 -- 172.21.15.97:0/2936357298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c0094a0 con 0x7f821815e080 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8216d9d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821815e080 0x7f821814a3d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f821000cef0 tx=0x7f821000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8206ffd700 1 -- 172.21.15.97:0/2936357298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821000ae50 con 0x7f821815e080 2023-11-24T06:11:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8206ffd700 1 -- 172.21.15.97:0/2936357298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8210015440 con 0x7f821815e080 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8218158c70 con 0x7f821815e080 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f8206ffd700 1 -- 172.21.15.97:0/2936357298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82100157f0 con 0x7f821815e080 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.459+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82180cd900 con 0x7f821815e080 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.460+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82180753f0 con 0x7f821815e080 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.460+0000 7f8206ffd700 1 -- 172.21.15.97:0/2936357298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f821000d430 con 0x7f821815e080 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.461+0000 7f8206ffd700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81f806b110 0x7f81f806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.461+0000 7f8206ffd700 1 -- 172.21.15.97:0/2936357298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f821008a070 con 0x7f821815e080 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.461+0000 7f821659c700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81f806b110 0x7f81f806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.461+0000 7f821659c700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81f806b110 0x7f81f806d5d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8208000f60 tx=0x7f8208007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.465+0000 7f8206ffd700 1 -- 172.21.15.97:0/2936357298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8210054dc0 con 0x7f821815e080 2023-11-24T06:11:08.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.651+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82180aecd0 con 0x7f81f806b110 2023-11-24T06:11:08.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.658+0000 7f8206ffd700 1 -- 172.21.15.97:0/2936357298 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289068 (secure 0 0 0) 0x7f8208012380 con 0x7f81f806b110 2023-11-24T06:11:08.669 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81f806b110 msgr2=0x7f81f806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81f806b110 0x7f81f806d5d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8208000f60 tx=0x7f8208007c90 comp rx=0 tx=0).stop 2023-11-24T06:11:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821815e080 msgr2=0x7f821814a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821815e080 0x7f821814a3d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f821000cef0 tx=0x7f821000a340 comp rx=0 tx=0).stop 2023-11-24T06:11:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 shutdown_connections 2023-11-24T06:11:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f821815e080 0x7f821814a3d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f81f806b110 0x7f81f806d5d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f821806c710 0x7f8218158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 --2- 172.21.15.97:0/2936357298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82180af4d0 0x7f82180d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 >> 172.21.15.97:0/2936357298 conn(0x7f8218071b90 msgr2=0x7f82180b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:08.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 shutdown_connections 2023-11-24T06:11:08.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.662+0000 7f821cd1d700 1 -- 172.21.15.97:0/2936357298 wait complete. 2023-11-24T06:11:08.675 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:11:08.678 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2023-11-24T06:11:08.350790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222828,"kb_used_data":6640,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751892,"statfs":{"total":773094113280,"available":772865937408,"internally_reserved":0,"allocated":6799360,"data_stored":3514062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001104"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.198867+0000","last_change":"2023-11-24T06:11:04.195308+0000","last_active":"2023-11-24T06:11:04.198867+0000","last_peered":"2023-11-24T06:11:04.198867+0000","last_clean":"2023-11-24T06:11:04.198867+0000","last_became_active":"2023-11-24T06:11:04.195019+0000","last_became_peered":"2023-11-24T06:11:04.195019+0000","last_unstale":"2023-11-24T06:11:04.198867+0000","last_undegraded":"2023-11-24T06:11:04.198867+0000","last_fullsized":"2023-11-24T06:11:04.198867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:58.686397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.203499+0000","last_change":"2023-11-24T06:11:04.194515+0000","last_active":"2023-11-24T06:11:04.203499+0000","last_peered":"2023-11-24T06:11:04.203499+0000","last_clean":"2023-11-24T06:11:04.203499+0000","last_became_active":"2023-11-24T06:11:04.194185+0000","last_became_peered":"2023-11-24T06:11:04.194185+0000","last_unstale":"2023-11-24T06:11:04.203499+0000","last_undegraded":"2023-11-24T06:11:04.203499+0000","last_fullsized":"2023-11-24T06:11:04.203499+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:06:53.543382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195390+0000","last_change":"2023-11-24T06:11:04.189706+0000","last_active":"2023-11-24T06:11:04.195390+0000","last_peered":"2023-11-24T06:11:04.195390+0000","last_clean":"2023-11-24T06:11:04.195390+0000","last_became_active":"2023-11-24T06:11:04.189380+0000","last_became_peered":"2023-11-24T06:11:04.189380+0000","last_unstale":"2023-11-24T06:11:04.195390+0000","last_undegraded":"2023-11-24T06:11:04.195390+0000","last_fullsized":"2023-11-24T06:11:04.195390+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:14.947513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183574+0000","last_change":"2023-11-24T06:11:04.180414+0000","last_active":"2023-11-24T06:11:04.183574+0000","last_peered":"2023-11-24T06:11:04.183574+0000","last_clean":"2023-11-24T06:11:04.183574+0000","last_became_active":"2023-11-24T06:11:04.180124+0000","last_became_peered":"2023-11-24T06:11:04.180124+0000","last_unstale":"2023-11-24T06:11:04.183574+0000","last_undegraded":"2023-11-24T06:11:04.183574+0000","last_fullsized":"2023-11-24T06:11:04.183574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:44.649472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185983+0000","last_change":"2023-11-24T06:11:04.183956+0000","last_active":"2023-11-24T06:11:04.185983+0000","last_peered":"2023-11-24T06:11:04.185983+0000","last_clean":"2023-11-24T06:11:04.185983+0000","last_became_active":"2023-11-24T06:11:04.183607+0000","last_became_peered":"2023-11-24T06:11:04.183607+0000","last_unstale":"2023-11-24T06:11:04.185983+0000","last_undegraded":"2023-11-24T06:11:04.185983+0000","last_fullsized":"2023-11-24T06:11:04.185983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:22:17.116277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185858+0000","last_change":"2023-11-24T06:11:04.180341+0000","last_active":"2023-11-24T06:11:04.185858+0000","last_peered":"2023-11-24T06:11:04.185858+0000","last_clean":"2023-11-24T06:11:04.185858+0000","last_became_active":"2023-11-24T06:11:04.179989+0000","last_became_peered":"2023-11-24T06:11:04.179989+0000","last_unstale":"2023-11-24T06:11:04.185858+0000","last_undegraded":"2023-11-24T06:11:04.185858+0000","last_fullsized":"2023-11-24T06:11:04.185858+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:54.702175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183751+0000","last_change":"2023-11-24T06:11:04.177751+0000","last_active":"2023-11-24T06:11:04.183751+0000","last_peered":"2023-11-24T06:11:04.183751+0000","last_clean":"2023-11-24T06:11:04.183751+0000","last_became_active":"2023-11-24T06:11:04.177470+0000","last_became_peered":"2023-11-24T06:11:04.177470+0000","last_unstale":"2023-11-24T06:11:04.183751+0000","last_undegraded":"2023-11-24T06:11:04.183751+0000","last_fullsized":"2023-11-24T06:11:04.183751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:26:52.488945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205848+0000","last_change":"2023-11-24T06:11:04.200447+0000","last_active":"2023-11-24T06:11:04.205848+0000","last_peered":"2023-11-24T06:11:04.205848+0000","last_clean":"2023-11-24T06:11:04.205848+0000","last_became_active":"2023-11-24T06:11:04.197503+0000","last_became_peered":"2023-11-24T06:11:04.197503+0000","last_unstale":"2023-11-24T06:11:04.205848+0000","last_undegraded":"2023-11-24T06:11:04.205848+0000","last_fullsized":"2023-11-24T06:11:04.205848+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:40.810920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.188703+0000","last_change":"2023-11-24T06:11:04.186747+0000","last_active":"2023-11-24T06:11:04.188703+0000","last_peered":"2023-11-24T06:11:04.188703+0000","last_clean":"2023-11-24T06:11:04.188703+0000","last_became_active":"2023-11-24T06:11:04.186476+0000","last_became_peered":"2023-11-24T06:11:04.186476+0000","last_unstale":"2023-11-24T06:11:04.188703+0000","last_undegraded":"2023-11-24T06:11:04.188703+0000","last_fullsized":"2023-11-24T06:11:04.188703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:14.947513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197128+0000","last_change":"2023-11-24T06:11:04.189046+0000","last_active":"2023-11-24T06:11:04.197128+0000","last_peered":"2023-11-24T06:11:04.197128+0000","last_clean":"2023-11-24T06:11:04.197128+0000","last_became_active":"2023-11-24T06:11:04.188805+0000","last_became_peered":"2023-11-24T06:11:04.188805+0000","last_unstale":"2023-11-24T06:11:04.197128+0000","last_undegraded":"2023-11-24T06:11:04.197128+0000","last_fullsized":"2023-11-24T06:11:04.197128+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:14.016212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197954+0000","last_change":"2023-11-24T06:11:04.189281+0000","last_active":"2023-11-24T06:11:04.197954+0000","last_peered":"2023-11-24T06:11:04.197954+0000","last_clean":"2023-11-24T06:11:04.197954+0000","last_became_active":"2023-11-24T06:11:04.188736+0000","last_became_peered":"2023-11-24T06:11:04.188736+0000","last_unstale":"2023-11-24T06:11:04.197954+0000","last_undegraded":"2023-11-24T06:11:04.197954+0000","last_fullsized":"2023-11-24T06:11:04.197954+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:44:42.079045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195412+0000","last_change":"2023-11-24T06:11:04.189330+0000","last_active":"2023-11-24T06:11:04.195412+0000","last_peered":"2023-11-24T06:11:04.195412+0000","last_clean":"2023-11-24T06:11:04.195412+0000","last_became_active":"2023-11-24T06:11:04.189059+0000","last_became_peered":"2023-11-24T06:11:04.189059+0000","last_unstale":"2023-11-24T06:11:04.195412+0000","last_undegraded":"2023-11-24T06:11:04.195412+0000","last_fullsized":"2023-11-24T06:11:04.195412+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:03.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.200854+0000","last_change":"2023-11-24T06:11:04.198923+0000","last_active":"2023-11-24T06:11:04.200854+0000","last_peered":"2023-11-24T06:11:04.200854+0000","last_clean":"2023-11-24T06:11:04.200854+0000","last_became_active":"2023-11-24T06:11:04.198533+0000","last_became_peered":"2023-11-24T06:11:04.198533+0000","last_unstale":"2023-11-24T06:11:04.200854+0000","last_undegraded":"2023-11-24T06:11:04.200854+0000","last_fullsized":"2023-11-24T06:11:04.200854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:32.690697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185252+0000","last_change":"2023-11-24T06:11:04.181224+0000","last_active":"2023-11-24T06:11:04.185252+0000","last_peered":"2023-11-24T06:11:04.185252+0000","last_clean":"2023-11-24T06:11:04.185252+0000","last_became_active":"2023-11-24T06:11:04.180958+0000","last_became_peered":"2023-11-24T06:11:04.180958+0000","last_unstale":"2023-11-24T06:11:04.185252+0000","last_undegraded":"2023-11-24T06:11:04.185252+0000","last_fullsized":"2023-11-24T06:11:04.185252+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:30.803405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185067+0000","last_change":"2023-11-24T06:11:04.180479+0000","last_active":"2023-11-24T06:11:04.185067+0000","last_peered":"2023-11-24T06:11:04.185067+0000","last_clean":"2023-11-24T06:11:04.185067+0000","last_became_active":"2023-11-24T06:11:04.180177+0000","last_became_peered":"2023-11-24T06:11:04.180177+0000","last_unstale":"2023-11-24T06:11:04.185067+0000","last_undegraded":"2023-11-24T06:11:04.185067+0000","last_fullsized":"2023-11-24T06:11:04.185067+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:21.633855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.200922+0000","last_change":"2023-11-24T06:11:04.193957+0000","last_active":"2023-11-24T06:11:04.200922+0000","last_peered":"2023-11-24T06:11:04.200922+0000","last_clean":"2023-11-24T06:11:04.200922+0000","last_became_active":"2023-11-24T06:11:04.192891+0000","last_became_peered":"2023-11-24T06:11:04.192891+0000","last_unstale":"2023-11-24T06:11:04.200922+0000","last_undegraded":"2023-11-24T06:11:04.200922+0000","last_fullsized":"2023-11-24T06:11:04.200922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:41:06.897965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205660+0000","last_change":"2023-11-24T06:11:04.197521+0000","last_active":"2023-11-24T06:11:04.205660+0000","last_peered":"2023-11-24T06:11:04.205660+0000","last_clean":"2023-11-24T06:11:04.205660+0000","last_became_active":"2023-11-24T06:11:04.194573+0000","last_became_peered":"2023-11-24T06:11:04.194573+0000","last_unstale":"2023-11-24T06:11:04.205660+0000","last_undegraded":"2023-11-24T06:11:04.205660+0000","last_fullsized":"2023-11-24T06:11:04.205660+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:51.190432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199615+0000","last_change":"2023-11-24T06:11:04.197700+0000","last_active":"2023-11-24T06:11:04.199615+0000","last_peered":"2023-11-24T06:11:04.199615+0000","last_clean":"2023-11-24T06:11:04.199615+0000","last_became_active":"2023-11-24T06:11:04.197453+0000","last_became_peered":"2023-11-24T06:11:04.197453+0000","last_unstale":"2023-11-24T06:11:04.199615+0000","last_undegraded":"2023-11-24T06:11:04.199615+0000","last_fullsized":"2023-11-24T06:11:04.199615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:34.520475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.189970+0000","last_change":"2023-11-24T06:11:04.187922+0000","last_active":"2023-11-24T06:11:04.189970+0000","last_peered":"2023-11-24T06:11:04.189970+0000","last_clean":"2023-11-24T06:11:04.189970+0000","last_became_active":"2023-11-24T06:11:04.187686+0000","last_became_peered":"2023-11-24T06:11:04.187686+0000","last_unstale":"2023-11-24T06:11:04.189970+0000","last_undegraded":"2023-11-24T06:11:04.189970+0000","last_fullsized":"2023-11-24T06:11:04.189970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:11.765091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206081+0000","last_change":"2023-11-24T06:11:04.200673+0000","last_active":"2023-11-24T06:11:04.206081+0000","last_peered":"2023-11-24T06:11:04.206081+0000","last_clean":"2023-11-24T06:11:04.206081+0000","last_became_active":"2023-11-24T06:11:04.198364+0000","last_became_peered":"2023-11-24T06:11:04.198364+0000","last_unstale":"2023-11-24T06:11:04.206081+0000","last_undegraded":"2023-11-24T06:11:04.206081+0000","last_fullsized":"2023-11-24T06:11:04.206081+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:50:22.336183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195183+0000","last_change":"2023-11-24T06:11:04.190805+0000","last_active":"2023-11-24T06:11:04.195183+0000","last_peered":"2023-11-24T06:11:04.195183+0000","last_clean":"2023-11-24T06:11:04.195183+0000","last_became_active":"2023-11-24T06:11:04.190505+0000","last_became_peered":"2023-11-24T06:11:04.190505+0000","last_unstale":"2023-11-24T06:11:04.195183+0000","last_undegraded":"2023-11-24T06:11:04.195183+0000","last_fullsized":"2023-11-24T06:11:04.195183+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:26:52.488945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206183+0000","last_change":"2023-11-24T06:11:04.204576+0000","last_active":"2023-11-24T06:11:04.206183+0000","last_peered":"2023-11-24T06:11:04.206183+0000","last_clean":"2023-11-24T06:11:04.206183+0000","last_became_active":"2023-11-24T06:11:04.204173+0000","last_became_peered":"2023-11-24T06:11:04.204173+0000","last_unstale":"2023-11-24T06:11:04.206183+0000","last_undegraded":"2023-11-24T06:11:04.206183+0000","last_fullsized":"2023-11-24T06:11:04.206183+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:46:57.861583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201165+0000","last_change":"2023-11-24T06:11:04.199261+0000","last_active":"2023-11-24T06:11:04.201165+0000","last_peered":"2023-11-24T06:11:04.201165+0000","last_clean":"2023-11-24T06:11:04.201165+0000","last_became_active":"2023-11-24T06:11:04.199038+0000","last_became_peered":"2023-11-24T06:11:04.199038+0000","last_unstale":"2023-11-24T06:11:04.201165+0000","last_undegraded":"2023-11-24T06:11:04.201165+0000","last_fullsized":"2023-11-24T06:11:04.201165+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:33.509180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206164+0000","last_change":"2023-11-24T06:11:04.201669+0000","last_active":"2023-11-24T06:11:04.206164+0000","last_peered":"2023-11-24T06:11:04.206164+0000","last_clean":"2023-11-24T06:11:04.206164+0000","last_became_active":"2023-11-24T06:11:04.201399+0000","last_became_peered":"2023-11-24T06:11:04.201399+0000","last_unstale":"2023-11-24T06:11:04.206164+0000","last_undegraded":"2023-11-24T06:11:04.206164+0000","last_fullsized":"2023-11-24T06:11:04.206164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:15.448301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.182529+0000","last_change":"2023-11-24T06:11:04.181049+0000","last_active":"2023-11-24T06:11:04.182529+0000","last_peered":"2023-11-24T06:11:04.182529+0000","last_clean":"2023-11-24T06:11:04.182529+0000","last_became_active":"2023-11-24T06:11:04.180758+0000","last_became_peered":"2023-11-24T06:11:04.180758+0000","last_unstale":"2023-11-24T06:11:04.182529+0000","last_undegraded":"2023-11-24T06:11:04.182529+0000","last_fullsized":"2023-11-24T06:11:04.182529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:15.081903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185742+0000","last_change":"2023-11-24T06:11:04.183326+0000","last_active":"2023-11-24T06:11:04.185742+0000","last_peered":"2023-11-24T06:11:04.185742+0000","last_clean":"2023-11-24T06:11:04.185742+0000","last_became_active":"2023-11-24T06:11:04.183043+0000","last_became_peered":"2023-11-24T06:11:04.183043+0000","last_unstale":"2023-11-24T06:11:04.185742+0000","last_undegraded":"2023-11-24T06:11:04.185742+0000","last_fullsized":"2023-11-24T06:11:04.185742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:14.947513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201114+0000","last_change":"2023-11-24T06:11:04.193767+0000","last_active":"2023-11-24T06:11:04.201114+0000","last_peered":"2023-11-24T06:11:04.201114+0000","last_clean":"2023-11-24T06:11:04.201114+0000","last_became_active":"2023-11-24T06:11:04.193335+0000","last_became_peered":"2023-11-24T06:11:04.193335+0000","last_unstale":"2023-11-24T06:11:04.201114+0000","last_undegraded":"2023-11-24T06:11:04.201114+0000","last_fullsized":"2023-11-24T06:11:04.201114+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:06.583278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.187659+0000","last_change":"2023-11-24T06:11:04.184419+0000","last_active":"2023-11-24T06:11:04.187659+0000","last_peered":"2023-11-24T06:11:04.187659+0000","last_clean":"2023-11-24T06:11:04.187659+0000","last_became_active":"2023-11-24T06:11:04.183921+0000","last_became_peered":"2023-11-24T06:11:04.183921+0000","last_unstale":"2023-11-24T06:11:04.187659+0000","last_undegraded":"2023-11-24T06:11:04.187659+0000","last_fullsized":"2023-11-24T06:11:04.187659+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:41.209210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.203604+0000","last_change":"2023-11-24T06:11:04.193544+0000","last_active":"2023-11-24T06:11:04.203604+0000","last_peered":"2023-11-24T06:11:04.203604+0000","last_clean":"2023-11-24T06:11:04.203604+0000","last_became_active":"2023-11-24T06:11:04.193294+0000","last_became_peered":"2023-11-24T06:11:04.193294+0000","last_unstale":"2023-11-24T06:11:04.203604+0000","last_undegraded":"2023-11-24T06:11:04.203604+0000","last_fullsized":"2023-11-24T06:11:04.203604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:04.507852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195140+0000","last_change":"2023-11-24T06:11:04.188600+0000","last_active":"2023-11-24T06:11:04.195140+0000","last_peered":"2023-11-24T06:11:04.195140+0000","last_clean":"2023-11-24T06:11:04.195140+0000","last_became_active":"2023-11-24T06:11:04.186678+0000","last_became_peered":"2023-11-24T06:11:04.186678+0000","last_unstale":"2023-11-24T06:11:04.195140+0000","last_undegraded":"2023-11-24T06:11:04.195140+0000","last_fullsized":"2023-11-24T06:11:04.195140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:30.803405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.182469+0000","last_change":"2023-11-24T06:11:04.181028+0000","last_active":"2023-11-24T06:11:04.182469+0000","last_peered":"2023-11-24T06:11:04.182469+0000","last_clean":"2023-11-24T06:11:04.182469+0000","last_became_active":"2023-11-24T06:11:04.180720+0000","last_became_peered":"2023-11-24T06:11:04.180720+0000","last_unstale":"2023-11-24T06:11:04.182469+0000","last_undegraded":"2023-11-24T06:11:04.182469+0000","last_fullsized":"2023-11-24T06:11:04.182469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.466418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.208310+0000","last_change":"2023-11-24T06:11:04.201575+0000","last_active":"2023-11-24T06:11:04.208310+0000","last_peered":"2023-11-24T06:11:04.208310+0000","last_clean":"2023-11-24T06:11:04.208310+0000","last_became_active":"2023-11-24T06:11:04.201079+0000","last_became_peered":"2023-11-24T06:11:04.201079+0000","last_unstale":"2023-11-24T06:11:04.208310+0000","last_undegraded":"2023-11-24T06:11:04.208310+0000","last_fullsized":"2023-11-24T06:11:04.208310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:57:49.776816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197178+0000","last_change":"2023-11-24T06:11:04.191794+0000","last_active":"2023-11-24T06:11:04.197178+0000","last_peered":"2023-11-24T06:11:04.197178+0000","last_clean":"2023-11-24T06:11:04.197178+0000","last_became_active":"2023-11-24T06:11:04.187184+0000","last_became_peered":"2023-11-24T06:11:04.187184+0000","last_unstale":"2023-11-24T06:11:04.197178+0000","last_undegraded":"2023-11-24T06:11:04.197178+0000","last_fullsized":"2023-11-24T06:11:04.197178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:45.874095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206821+0000","last_change":"2023-11-24T06:11:03.184684+0000","last_active":"2023-11-24T06:11:04.206821+0000","last_peered":"2023-11-24T06:11:04.206821+0000","last_clean":"2023-11-24T06:11:04.206821+0000","last_became_active":"2023-11-24T06:11:03.184356+0000","last_became_peered":"2023-11-24T06:11:03.184356+0000","last_unstale":"2023-11-24T06:11:04.206821+0000","last_undegraded":"2023-11-24T06:11:04.206821+0000","last_fullsized":"2023-11-24T06:11:04.206821+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:14.780098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.198180+0000","last_change":"2023-11-24T06:11:04.189851+0000","last_active":"2023-11-24T06:11:04.198180+0000","last_peered":"2023-11-24T06:11:04.198180+0000","last_clean":"2023-11-24T06:11:04.198180+0000","last_became_active":"2023-11-24T06:11:04.189500+0000","last_became_peered":"2023-11-24T06:11:04.189500+0000","last_unstale":"2023-11-24T06:11:04.198180+0000","last_undegraded":"2023-11-24T06:11:04.198180+0000","last_fullsized":"2023-11-24T06:11:04.198180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:23.340839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206285+0000","last_change":"2023-11-24T06:11:03.186650+0000","last_active":"2023-11-24T06:11:04.206285+0000","last_peered":"2023-11-24T06:11:04.206285+0000","last_clean":"2023-11-24T06:11:04.206285+0000","last_became_active":"2023-11-24T06:11:03.186357+0000","last_became_peered":"2023-11-24T06:11:03.186357+0000","last_unstale":"2023-11-24T06:11:04.206285+0000","last_undegraded":"2023-11-24T06:11:04.206285+0000","last_fullsized":"2023-11-24T06:11:04.206285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:48.119927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197178+0000","last_change":"2023-11-24T06:11:04.192216+0000","last_active":"2023-11-24T06:11:04.197178+0000","last_peered":"2023-11-24T06:11:04.197178+0000","last_clean":"2023-11-24T06:11:04.197178+0000","last_became_active":"2023-11-24T06:11:04.187320+0000","last_became_peered":"2023-11-24T06:11:04.187320+0000","last_unstale":"2023-11-24T06:11:04.197178+0000","last_undegraded":"2023-11-24T06:11:04.197178+0000","last_fullsized":"2023-11-24T06:11:04.197178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:23.129399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201205+0000","last_change":"2023-11-24T06:11:03.182775+0000","last_active":"2023-11-24T06:11:04.201205+0000","last_peered":"2023-11-24T06:11:04.201205+0000","last_clean":"2023-11-24T06:11:04.201205+0000","last_became_active":"2023-11-24T06:11:03.182503+0000","last_became_peered":"2023-11-24T06:11:03.182503+0000","last_unstale":"2023-11-24T06:11:04.201205+0000","last_undegraded":"2023-11-24T06:11:04.201205+0000","last_fullsized":"2023-11-24T06:11:04.201205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:06.913576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201132+0000","last_change":"2023-11-24T06:11:04.193874+0000","last_active":"2023-11-24T06:11:04.201132+0000","last_peered":"2023-11-24T06:11:04.201132+0000","last_clean":"2023-11-24T06:11:04.201132+0000","last_became_active":"2023-11-24T06:11:04.193582+0000","last_became_peered":"2023-11-24T06:11:04.193582+0000","last_unstale":"2023-11-24T06:11:04.201132+0000","last_undegraded":"2023-11-24T06:11:04.201132+0000","last_fullsized":"2023-11-24T06:11:04.201132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:46.252673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.196039+0000","last_change":"2023-11-24T06:11:03.185909+0000","last_active":"2023-11-24T06:11:04.196039+0000","last_peered":"2023-11-24T06:11:04.196039+0000","last_clean":"2023-11-24T06:11:04.196039+0000","last_became_active":"2023-11-24T06:11:03.185627+0000","last_became_peered":"2023-11-24T06:11:03.185627+0000","last_unstale":"2023-11-24T06:11:04.196039+0000","last_undegraded":"2023-11-24T06:11:04.196039+0000","last_fullsized":"2023-11-24T06:11:04.196039+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:22.423162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206155+0000","last_change":"2023-11-24T06:11:04.204365+0000","last_active":"2023-11-24T06:11:04.206155+0000","last_peered":"2023-11-24T06:11:04.206155+0000","last_clean":"2023-11-24T06:11:04.206155+0000","last_became_active":"2023-11-24T06:11:04.203907+0000","last_became_peered":"2023-11-24T06:11:04.203907+0000","last_unstale":"2023-11-24T06:11:04.206155+0000","last_undegraded":"2023-11-24T06:11:04.206155+0000","last_fullsized":"2023-11-24T06:11:04.206155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:33:56.830325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183757+0000","last_change":"2023-11-24T06:11:03.180339+0000","last_active":"2023-11-24T06:11:04.183757+0000","last_peered":"2023-11-24T06:11:04.183757+0000","last_clean":"2023-11-24T06:11:04.183757+0000","last_became_active":"2023-11-24T06:11:03.180010+0000","last_became_peered":"2023-11-24T06:11:03.180010+0000","last_unstale":"2023-11-24T06:11:04.183757+0000","last_undegraded":"2023-11-24T06:11:04.183757+0000","last_fullsized":"2023-11-24T06:11:04.183757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:43.645339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205879+0000","last_change":"2023-11-24T06:11:04.200610+0000","last_active":"2023-11-24T06:11:04.205879+0000","last_peered":"2023-11-24T06:11:04.205879+0000","last_clean":"2023-11-24T06:11:04.205879+0000","last_became_active":"2023-11-24T06:11:04.198352+0000","last_became_peered":"2023-11-24T06:11:04.198352+0000","last_unstale":"2023-11-24T06:11:04.205879+0000","last_undegraded":"2023-11-24T06:11:04.205879+0000","last_fullsized":"2023-11-24T06:11:04.205879+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:57:49.776816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197514+0000","last_change":"2023-11-24T06:11:03.178694+0000","last_active":"2023-11-24T06:11:04.197514+0000","last_peered":"2023-11-24T06:11:04.197514+0000","last_clean":"2023-11-24T06:11:04.197514+0000","last_became_active":"2023-11-24T06:11:03.178418+0000","last_became_peered":"2023-11-24T06:11:03.178418+0000","last_unstale":"2023-11-24T06:11:04.197514+0000","last_undegraded":"2023-11-24T06:11:04.197514+0000","last_fullsized":"2023-11-24T06:11:04.197514+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:33.280235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195741+0000","last_change":"2023-11-24T06:11:04.192942+0000","last_active":"2023-11-24T06:11:04.195741+0000","last_peered":"2023-11-24T06:11:04.195741+0000","last_clean":"2023-11-24T06:11:04.195741+0000","last_became_active":"2023-11-24T06:11:04.192648+0000","last_became_peered":"2023-11-24T06:11:04.192648+0000","last_unstale":"2023-11-24T06:11:04.195741+0000","last_undegraded":"2023-11-24T06:11:04.195741+0000","last_fullsized":"2023-11-24T06:11:04.195741+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:39.158782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176901+0000","last_change":"2023-11-24T06:11:03.184298+0000","last_active":"2023-11-24T06:11:04.176901+0000","last_peered":"2023-11-24T06:11:04.176901+0000","last_clean":"2023-11-24T06:11:04.176901+0000","last_became_active":"2023-11-24T06:11:03.183471+0000","last_became_peered":"2023-11-24T06:11:03.183471+0000","last_unstale":"2023-11-24T06:11:04.176901+0000","last_undegraded":"2023-11-24T06:11:04.176901+0000","last_fullsized":"2023-11-24T06:11:04.176901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:41:05.893832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205600+0000","last_change":"2023-11-24T06:11:04.196544+0000","last_active":"2023-11-24T06:11:04.205600+0000","last_peered":"2023-11-24T06:11:04.205600+0000","last_clean":"2023-11-24T06:11:04.205600+0000","last_became_active":"2023-11-24T06:11:04.194406+0000","last_became_peered":"2023-11-24T06:11:04.194406+0000","last_unstale":"2023-11-24T06:11:04.205600+0000","last_undegraded":"2023-11-24T06:11:04.205600+0000","last_fullsized":"2023-11-24T06:11:04.205600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:10.411069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.173470+0000","last_change":"2023-11-24T06:11:03.189506+0000","last_active":"2023-11-24T06:11:04.173470+0000","last_peered":"2023-11-24T06:11:04.173470+0000","last_clean":"2023-11-24T06:11:04.173470+0000","last_became_active":"2023-11-24T06:11:03.189223+0000","last_became_peered":"2023-11-24T06:11:03.189223+0000","last_unstale":"2023-11-24T06:11:04.173470+0000","last_undegraded":"2023-11-24T06:11:04.173470+0000","last_fullsized":"2023-11-24T06:11:04.173470+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:59.937947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185298+0000","last_change":"2023-11-24T06:11:04.182876+0000","last_active":"2023-11-24T06:11:04.185298+0000","last_peered":"2023-11-24T06:11:04.185298+0000","last_clean":"2023-11-24T06:11:04.185298+0000","last_became_active":"2023-11-24T06:11:04.181971+0000","last_became_peered":"2023-11-24T06:11:04.181971+0000","last_unstale":"2023-11-24T06:11:04.185298+0000","last_undegraded":"2023-11-24T06:11:04.185298+0000","last_fullsized":"2023-11-24T06:11:04.185298+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:09.830311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.181381+0000","last_change":"2023-11-24T06:11:03.183311+0000","last_active":"2023-11-24T06:11:04.181381+0000","last_peered":"2023-11-24T06:11:04.181381+0000","last_clean":"2023-11-24T06:11:04.181381+0000","last_became_active":"2023-11-24T06:11:03.182809+0000","last_became_peered":"2023-11-24T06:11:03.182809+0000","last_unstale":"2023-11-24T06:11:04.181381+0000","last_undegraded":"2023-11-24T06:11:04.181381+0000","last_fullsized":"2023-11-24T06:11:04.181381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:37:59.937947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199596+0000","last_change":"2023-11-24T06:11:04.197502+0000","last_active":"2023-11-24T06:11:04.199596+0000","last_peered":"2023-11-24T06:11:04.199596+0000","last_clean":"2023-11-24T06:11:04.199596+0000","last_became_active":"2023-11-24T06:11:04.197194+0000","last_became_peered":"2023-11-24T06:11:04.197194+0000","last_unstale":"2023-11-24T06:11:04.199596+0000","last_undegraded":"2023-11-24T06:11:04.199596+0000","last_fullsized":"2023-11-24T06:11:04.199596+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:41:06.897965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199837+0000","last_change":"2023-11-24T06:11:03.186637+0000","last_active":"2023-11-24T06:11:04.199837+0000","last_peered":"2023-11-24T06:11:04.199837+0000","last_clean":"2023-11-24T06:11:04.199837+0000","last_became_active":"2023-11-24T06:11:03.186332+0000","last_became_peered":"2023-11-24T06:11:03.186332+0000","last_unstale":"2023-11-24T06:11:04.199837+0000","last_undegraded":"2023-11-24T06:11:04.199837+0000","last_fullsized":"2023-11-24T06:11:04.199837+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:55.994309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.185689+0000","last_change":"2023-11-24T06:11:04.184915+0000","last_active":"2023-11-24T06:11:04.185689+0000","last_peered":"2023-11-24T06:11:04.185689+0000","last_clean":"2023-11-24T06:11:04.185689+0000","last_became_active":"2023-11-24T06:11:04.184674+0000","last_became_peered":"2023-11-24T06:11:04.184674+0000","last_unstale":"2023-11-24T06:11:04.185689+0000","last_undegraded":"2023-11-24T06:11:04.185689+0000","last_fullsized":"2023-11-24T06:11:04.185689+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:01:41.115525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199459+0000","last_change":"2023-11-24T06:11:03.178337+0000","last_active":"2023-11-24T06:11:04.199459+0000","last_peered":"2023-11-24T06:11:04.199459+0000","last_clean":"2023-11-24T06:11:04.199459+0000","last_became_active":"2023-11-24T06:11:03.177980+0000","last_became_peered":"2023-11-24T06:11:03.177980+0000","last_unstale":"2023-11-24T06:11:04.199459+0000","last_undegraded":"2023-11-24T06:11:04.199459+0000","last_fullsized":"2023-11-24T06:11:04.199459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:22.423162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195409+0000","last_change":"2023-11-24T06:11:04.190817+0000","last_active":"2023-11-24T06:11:04.195409+0000","last_peered":"2023-11-24T06:11:04.195409+0000","last_clean":"2023-11-24T06:11:04.195409+0000","last_became_active":"2023-11-24T06:11:04.190482+0000","last_became_peered":"2023-11-24T06:11:04.190482+0000","last_unstale":"2023-11-24T06:11:04.195409+0000","last_undegraded":"2023-11-24T06:11:04.195409+0000","last_fullsized":"2023-11-24T06:11:04.195409+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:03.327256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.208532+0000","last_change":"2023-11-24T06:11:03.188014+0000","last_active":"2023-11-24T06:11:04.208532+0000","last_peered":"2023-11-24T06:11:04.208532+0000","last_clean":"2023-11-24T06:11:04.208532+0000","last_became_active":"2023-11-24T06:11:03.187732+0000","last_became_peered":"2023-11-24T06:11:03.187732+0000","last_unstale":"2023-11-24T06:11:04.208532+0000","last_undegraded":"2023-11-24T06:11:04.208532+0000","last_fullsized":"2023-11-24T06:11:04.208532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:58:02.323123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183357+0000","last_change":"2023-11-24T06:11:04.180194+0000","last_active":"2023-11-24T06:11:04.183357+0000","last_peered":"2023-11-24T06:11:04.183357+0000","last_clean":"2023-11-24T06:11:04.183357+0000","last_became_active":"2023-11-24T06:11:04.179893+0000","last_became_peered":"2023-11-24T06:11:04.179893+0000","last_unstale":"2023-11-24T06:11:04.183357+0000","last_undegraded":"2023-11-24T06:11:04.183357+0000","last_fullsized":"2023-11-24T06:11:04.183357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:06.446082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206246+0000","last_change":"2023-11-24T06:11:03.186245+0000","last_active":"2023-11-24T06:11:04.206246+0000","last_peered":"2023-11-24T06:11:04.206246+0000","last_clean":"2023-11-24T06:11:04.206246+0000","last_became_active":"2023-11-24T06:11:03.185943+0000","last_became_peered":"2023-11-24T06:11:03.185943+0000","last_unstale":"2023-11-24T06:11:04.206246+0000","last_undegraded":"2023-11-24T06:11:04.206246+0000","last_fullsized":"2023-11-24T06:11:04.206246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:58.064530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195164+0000","last_change":"2023-11-24T06:11:04.189331+0000","last_active":"2023-11-24T06:11:04.195164+0000","last_peered":"2023-11-24T06:11:04.195164+0000","last_clean":"2023-11-24T06:11:04.195164+0000","last_became_active":"2023-11-24T06:11:04.189023+0000","last_became_peered":"2023-11-24T06:11:04.189023+0000","last_unstale":"2023-11-24T06:11:04.195164+0000","last_undegraded":"2023-11-24T06:11:04.195164+0000","last_fullsized":"2023-11-24T06:11:04.195164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:59.683946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.179467+0000","last_change":"2023-11-24T06:11:03.177059+0000","last_active":"2023-11-24T06:11:04.179467+0000","last_peered":"2023-11-24T06:11:04.179467+0000","last_clean":"2023-11-24T06:11:04.179467+0000","last_became_active":"2023-11-24T06:11:03.176749+0000","last_became_peered":"2023-11-24T06:11:03.176749+0000","last_unstale":"2023-11-24T06:11:04.179467+0000","last_undegraded":"2023-11-24T06:11:04.179467+0000","last_fullsized":"2023-11-24T06:11:04.179467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:52:14.402838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.191832+0000","last_change":"2023-11-24T06:11:04.189779+0000","last_active":"2023-11-24T06:11:04.191832+0000","last_peered":"2023-11-24T06:11:04.191832+0000","last_clean":"2023-11-24T06:11:04.191832+0000","last_became_active":"2023-11-24T06:11:04.189582+0000","last_became_peered":"2023-11-24T06:11:04.189582+0000","last_unstale":"2023-11-24T06:11:04.191832+0000","last_undegraded":"2023-11-24T06:11:04.191832+0000","last_fullsized":"2023-11-24T06:11:04.191832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:59.683946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.177011+0000","last_change":"2023-11-24T06:11:03.184703+0000","last_active":"2023-11-24T06:11:04.177011+0000","last_peered":"2023-11-24T06:11:04.177011+0000","last_clean":"2023-11-24T06:11:04.177011+0000","last_became_active":"2023-11-24T06:11:03.183731+0000","last_became_peered":"2023-11-24T06:11:03.183731+0000","last_unstale":"2023-11-24T06:11:04.177011+0000","last_undegraded":"2023-11-24T06:11:04.177011+0000","last_fullsized":"2023-11-24T06:11:04.177011+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:33.516342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206085+0000","last_change":"2023-11-24T06:11:04.199456+0000","last_active":"2023-11-24T06:11:04.206085+0000","last_peered":"2023-11-24T06:11:04.206085+0000","last_clean":"2023-11-24T06:11:04.206085+0000","last_became_active":"2023-11-24T06:11:04.199084+0000","last_became_peered":"2023-11-24T06:11:04.199084+0000","last_unstale":"2023-11-24T06:11:04.206085+0000","last_undegraded":"2023-11-24T06:11:04.206085+0000","last_fullsized":"2023-11-24T06:11:04.206085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:32:44.649472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.203882+0000","last_change":"2023-11-24T06:11:03.182763+0000","last_active":"2023-11-24T06:11:04.203882+0000","last_peered":"2023-11-24T06:11:04.203882+0000","last_clean":"2023-11-24T06:11:04.203882+0000","last_became_active":"2023-11-24T06:11:03.182496+0000","last_became_peered":"2023-11-24T06:11:03.182496+0000","last_unstale":"2023-11-24T06:11:04.203882+0000","last_undegraded":"2023-11-24T06:11:04.203882+0000","last_fullsized":"2023-11-24T06:11:04.203882+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:08.826178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.198249+0000","last_change":"2023-11-24T06:11:04.192044+0000","last_active":"2023-11-24T06:11:04.198249+0000","last_peered":"2023-11-24T06:11:04.198249+0000","last_clean":"2023-11-24T06:11:04.198249+0000","last_became_active":"2023-11-24T06:11:04.191633+0000","last_became_peered":"2023-11-24T06:11:04.191633+0000","last_unstale":"2023-11-24T06:11:04.198249+0000","last_undegraded":"2023-11-24T06:11:04.198249+0000","last_fullsized":"2023-11-24T06:11:04.198249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:06.583278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206172+0000","last_change":"2023-11-24T06:11:03.187454+0000","last_active":"2023-11-24T06:11:04.206172+0000","last_peered":"2023-11-24T06:11:04.206172+0000","last_clean":"2023-11-24T06:11:04.206172+0000","last_became_active":"2023-11-24T06:11:03.187186+0000","last_became_peered":"2023-11-24T06:11:03.187186+0000","last_unstale":"2023-11-24T06:11:04.206172+0000","last_undegraded":"2023-11-24T06:11:04.206172+0000","last_fullsized":"2023-11-24T06:11:04.206172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:50:59.733135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183914+0000","last_change":"2023-11-24T06:11:04.182314+0000","last_active":"2023-11-24T06:11:04.183914+0000","last_peered":"2023-11-24T06:11:04.183914+0000","last_clean":"2023-11-24T06:11:04.183914+0000","last_became_active":"2023-11-24T06:11:04.181877+0000","last_became_peered":"2023-11-24T06:11:04.181877+0000","last_unstale":"2023-11-24T06:11:04.183914+0000","last_undegraded":"2023-11-24T06:11:04.183914+0000","last_fullsized":"2023-11-24T06:11:04.183914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:59.683946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199552+0000","last_change":"2023-11-24T06:11:03.187037+0000","last_active":"2023-11-24T06:11:04.199552+0000","last_peered":"2023-11-24T06:11:04.199552+0000","last_clean":"2023-11-24T06:11:04.199552+0000","last_became_active":"2023-11-24T06:11:03.186748+0000","last_became_peered":"2023-11-24T06:11:03.186748+0000","last_unstale":"2023-11-24T06:11:04.199552+0000","last_undegraded":"2023-11-24T06:11:04.199552+0000","last_fullsized":"2023-11-24T06:11:04.199552+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:33.516342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.194622+0000","last_change":"2023-11-24T06:11:04.184957+0000","last_active":"2023-11-24T06:11:04.194622+0000","last_peered":"2023-11-24T06:11:04.194622+0000","last_clean":"2023-11-24T06:11:04.194622+0000","last_became_active":"2023-11-24T06:11:04.184657+0000","last_became_peered":"2023-11-24T06:11:04.184657+0000","last_unstale":"2023-11-24T06:11:04.194622+0000","last_undegraded":"2023-11-24T06:11:04.194622+0000","last_fullsized":"2023-11-24T06:11:04.194622+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:15.784231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.195900+0000","last_change":"2023-11-24T06:11:03.185867+0000","last_active":"2023-11-24T06:11:04.195900+0000","last_peered":"2023-11-24T06:11:04.195900+0000","last_clean":"2023-11-24T06:11:04.195900+0000","last_became_active":"2023-11-24T06:11:03.185518+0000","last_became_peered":"2023-11-24T06:11:03.185518+0000","last_unstale":"2023-11-24T06:11:04.195900+0000","last_undegraded":"2023-11-24T06:11:04.195900+0000","last_fullsized":"2023-11-24T06:11:04.195900+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:53.698042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206099+0000","last_change":"2023-11-24T06:11:04.199502+0000","last_active":"2023-11-24T06:11:04.206099+0000","last_peered":"2023-11-24T06:11:04.206099+0000","last_clean":"2023-11-24T06:11:04.206099+0000","last_became_active":"2023-11-24T06:11:04.198901+0000","last_became_peered":"2023-11-24T06:11:04.198901+0000","last_unstale":"2023-11-24T06:11:04.206099+0000","last_undegraded":"2023-11-24T06:11:04.206099+0000","last_fullsized":"2023-11-24T06:11:04.206099+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:32:54.702175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.170479+0000","last_change":"2023-11-24T06:11:03.188060+0000","last_active":"2023-11-24T06:11:04.170479+0000","last_peered":"2023-11-24T06:11:04.170479+0000","last_clean":"2023-11-24T06:11:04.170479+0000","last_became_active":"2023-11-24T06:11:03.187761+0000","last_became_peered":"2023-11-24T06:11:03.187761+0000","last_unstale":"2023-11-24T06:11:04.170479+0000","last_undegraded":"2023-11-24T06:11:04.170479+0000","last_fullsized":"2023-11-24T06:11:04.170479+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:32.505047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183378+0000","last_change":"2023-11-24T06:11:04.180281+0000","last_active":"2023-11-24T06:11:04.183378+0000","last_peered":"2023-11-24T06:11:04.183378+0000","last_clean":"2023-11-24T06:11:04.183378+0000","last_became_active":"2023-11-24T06:11:04.179961+0000","last_became_peered":"2023-11-24T06:11:04.179961+0000","last_unstale":"2023-11-24T06:11:04.183378+0000","last_undegraded":"2023-11-24T06:11:04.183378+0000","last_fullsized":"2023-11-24T06:11:04.183378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:00.942080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206879+0000","last_change":"2023-11-24T06:11:03.186597+0000","last_active":"2023-11-24T06:11:04.206879+0000","last_peered":"2023-11-24T06:11:04.206879+0000","last_clean":"2023-11-24T06:11:04.206879+0000","last_became_active":"2023-11-24T06:11:03.186316+0000","last_became_peered":"2023-11-24T06:11:03.186316+0000","last_unstale":"2023-11-24T06:11:04.206879+0000","last_undegraded":"2023-11-24T06:11:04.206879+0000","last_fullsized":"2023-11-24T06:11:04.206879+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:13:54.640391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197747+0000","last_change":"2023-11-24T06:11:04.193045+0000","last_active":"2023-11-24T06:11:04.197747+0000","last_peered":"2023-11-24T06:11:04.197747+0000","last_clean":"2023-11-24T06:11:04.197747+0000","last_became_active":"2023-11-24T06:11:04.192792+0000","last_became_peered":"2023-11-24T06:11:04.192792+0000","last_unstale":"2023-11-24T06:11:04.197747+0000","last_undegraded":"2023-11-24T06:11:04.197747+0000","last_fullsized":"2023-11-24T06:11:04.197747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:06:33.622081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176376+0000","last_change":"2023-11-24T06:11:03.180198+0000","last_active":"2023-11-24T06:11:04.176376+0000","last_peered":"2023-11-24T06:11:04.176376+0000","last_clean":"2023-11-24T06:11:04.176376+0000","last_became_active":"2023-11-24T06:11:03.179329+0000","last_became_peered":"2023-11-24T06:11:03.179329+0000","last_unstale":"2023-11-24T06:11:04.176376+0000","last_undegraded":"2023-11-24T06:11:04.176376+0000","last_fullsized":"2023-11-24T06:11:04.176376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:42.310119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.197187+0000","last_change":"2023-11-24T06:11:04.191954+0000","last_active":"2023-11-24T06:11:04.197187+0000","last_peered":"2023-11-24T06:11:04.197187+0000","last_clean":"2023-11-24T06:11:04.197187+0000","last_became_active":"2023-11-24T06:11:04.191670+0000","last_became_peered":"2023-11-24T06:11:04.191670+0000","last_unstale":"2023-11-24T06:11:04.197187+0000","last_undegraded":"2023-11-24T06:11:04.197187+0000","last_fullsized":"2023-11-24T06:11:04.197187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:58.121927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176921+0000","last_change":"2023-11-24T06:11:03.191521+0000","last_active":"2023-11-24T06:11:04.176921+0000","last_peered":"2023-11-24T06:11:04.176921+0000","last_clean":"2023-11-24T06:11:04.176921+0000","last_became_active":"2023-11-24T06:11:03.191264+0000","last_became_peered":"2023-11-24T06:11:03.191264+0000","last_unstale":"2023-11-24T06:11:04.176921+0000","last_undegraded":"2023-11-24T06:11:04.176921+0000","last_fullsized":"2023-11-24T06:11:04.176921+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:05.579145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183458+0000","last_change":"2023-11-24T06:11:04.181054+0000","last_active":"2023-11-24T06:11:04.183458+0000","last_peered":"2023-11-24T06:11:04.183458+0000","last_clean":"2023-11-24T06:11:04.183458+0000","last_became_active":"2023-11-24T06:11:04.180796+0000","last_became_peered":"2023-11-24T06:11:04.180796+0000","last_unstale":"2023-11-24T06:11:04.183458+0000","last_undegraded":"2023-11-24T06:11:04.183458+0000","last_fullsized":"2023-11-24T06:11:04.183458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:09.830311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.203886+0000","last_change":"2023-11-24T06:11:03.190074+0000","last_active":"2023-11-24T06:11:04.203886+0000","last_peered":"2023-11-24T06:11:04.203886+0000","last_clean":"2023-11-24T06:11:04.203886+0000","last_became_active":"2023-11-24T06:11:03.189744+0000","last_became_peered":"2023-11-24T06:11:03.189744+0000","last_unstale":"2023-11-24T06:11:04.203886+0000","last_undegraded":"2023-11-24T06:11:04.203886+0000","last_fullsized":"2023-11-24T06:11:04.203886+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:52:14.402838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.188726+0000","last_change":"2023-11-24T06:11:04.187442+0000","last_active":"2023-11-24T06:11:04.188726+0000","last_peered":"2023-11-24T06:11:04.188726+0000","last_clean":"2023-11-24T06:11:04.188726+0000","last_became_active":"2023-11-24T06:11:04.187162+0000","last_became_peered":"2023-11-24T06:11:04.187162+0000","last_unstale":"2023-11-24T06:11:04.188726+0000","last_undegraded":"2023-11-24T06:11:04.188726+0000","last_fullsized":"2023-11-24T06:11:04.188726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:37:30.803405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.170466+0000","last_change":"2023-11-24T06:11:03.181114+0000","last_active":"2023-11-24T06:11:04.170466+0000","last_peered":"2023-11-24T06:11:04.170466+0000","last_clean":"2023-11-24T06:11:04.170466+0000","last_became_active":"2023-11-24T06:11:03.180828+0000","last_became_peered":"2023-11-24T06:11:03.180828+0000","last_unstale":"2023-11-24T06:11:04.170466+0000","last_undegraded":"2023-11-24T06:11:04.170466+0000","last_fullsized":"2023-11-24T06:11:04.170466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:22.423162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.192661+0000","last_change":"2023-11-24T06:11:04.185379+0000","last_active":"2023-11-24T06:11:04.192661+0000","last_peered":"2023-11-24T06:11:04.192661+0000","last_clean":"2023-11-24T06:11:04.192661+0000","last_became_active":"2023-11-24T06:11:04.185031+0000","last_became_peered":"2023-11-24T06:11:04.185031+0000","last_unstale":"2023-11-24T06:11:04.192661+0000","last_undegraded":"2023-11-24T06:11:04.192661+0000","last_fullsized":"2023-11-24T06:11:04.192661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:52.004170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.182798+0000","last_change":"2023-11-24T06:11:03.177439+0000","last_active":"2023-11-24T06:11:04.182798+0000","last_peered":"2023-11-24T06:11:04.182798+0000","last_clean":"2023-11-24T06:11:04.182798+0000","last_became_active":"2023-11-24T06:11:03.177076+0000","last_became_peered":"2023-11-24T06:11:03.177076+0000","last_unstale":"2023-11-24T06:11:04.182798+0000","last_undegraded":"2023-11-24T06:11:04.182798+0000","last_fullsized":"2023-11-24T06:11:04.182798+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:37:16.886042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.201093+0000","last_change":"2023-11-24T06:11:04.193602+0000","last_active":"2023-11-24T06:11:04.201093+0000","last_peered":"2023-11-24T06:11:04.201093+0000","last_clean":"2023-11-24T06:11:04.201093+0000","last_became_active":"2023-11-24T06:11:04.193316+0000","last_became_peered":"2023-11-24T06:11:04.193316+0000","last_unstale":"2023-11-24T06:11:04.201093+0000","last_undegraded":"2023-11-24T06:11:04.201093+0000","last_fullsized":"2023-11-24T06:11:04.201093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:13.777194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.177260+0000","last_change":"2023-11-24T06:11:03.183951+0000","last_active":"2023-11-24T06:11:04.177260+0000","last_peered":"2023-11-24T06:11:04.177260+0000","last_clean":"2023-11-24T06:11:04.177260+0000","last_became_active":"2023-11-24T06:11:03.183689+0000","last_became_peered":"2023-11-24T06:11:03.183689+0000","last_unstale":"2023-11-24T06:11:04.177260+0000","last_undegraded":"2023-11-24T06:11:04.177260+0000","last_fullsized":"2023-11-24T06:11:04.177260+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:32.505047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.206100+0000","last_change":"2023-11-24T06:11:04.203686+0000","last_active":"2023-11-24T06:11:04.206100+0000","last_peered":"2023-11-24T06:11:04.206100+0000","last_clean":"2023-11-24T06:11:04.206100+0000","last_became_active":"2023-11-24T06:11:04.203407+0000","last_became_peered":"2023-11-24T06:11:04.203407+0000","last_unstale":"2023-11-24T06:11:04.206100+0000","last_undegraded":"2023-11-24T06:11:04.206100+0000","last_fullsized":"2023-11-24T06:11:04.206100+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:57:37.335707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176465+0000","last_change":"2023-11-24T06:11:03.185143+0000","last_active":"2023-11-24T06:11:04.176465+0000","last_peered":"2023-11-24T06:11:04.176465+0000","last_clean":"2023-11-24T06:11:04.176465+0000","last_became_active":"2023-11-24T06:11:03.184866+0000","last_became_peered":"2023-11-24T06:11:03.184866+0000","last_unstale":"2023-11-24T06:11:04.176465+0000","last_undegraded":"2023-11-24T06:11:04.176465+0000","last_fullsized":"2023-11-24T06:11:04.176465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:25.278280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.199557+0000","last_change":"2023-11-24T06:11:04.194739+0000","last_active":"2023-11-24T06:11:04.199557+0000","last_peered":"2023-11-24T06:11:04.199557+0000","last_clean":"2023-11-24T06:11:04.199557+0000","last_became_active":"2023-11-24T06:11:04.194501+0000","last_became_peered":"2023-11-24T06:11:04.194501+0000","last_unstale":"2023-11-24T06:11:04.199557+0000","last_undegraded":"2023-11-24T06:11:04.199557+0000","last_fullsized":"2023-11-24T06:11:04.199557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:06.446082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.173074+0000","last_change":"2023-11-24T06:11:03.188018+0000","last_active":"2023-11-24T06:11:04.173074+0000","last_peered":"2023-11-24T06:11:04.173074+0000","last_clean":"2023-11-24T06:11:04.173074+0000","last_became_active":"2023-11-24T06:11:03.187777+0000","last_became_peered":"2023-11-24T06:11:03.187777+0000","last_unstale":"2023-11-24T06:11:04.173074+0000","last_undegraded":"2023-11-24T06:11:04.173074+0000","last_fullsized":"2023-11-24T06:11:04.173074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:05.441949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.196751+0000","last_change":"2023-11-24T06:11:04.185798+0000","last_active":"2023-11-24T06:11:04.196751+0000","last_peered":"2023-11-24T06:11:04.196751+0000","last_clean":"2023-11-24T06:11:04.196751+0000","last_became_active":"2023-11-24T06:11:04.185535+0000","last_became_peered":"2023-11-24T06:11:04.185535+0000","last_unstale":"2023-11-24T06:11:04.196751+0000","last_undegraded":"2023-11-24T06:11:04.196751+0000","last_fullsized":"2023-11-24T06:11:04.196751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:37.567668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.176942+0000","last_change":"2023-11-24T06:11:03.183029+0000","last_active":"2023-11-24T06:11:04.176942+0000","last_peered":"2023-11-24T06:11:04.176942+0000","last_clean":"2023-11-24T06:11:04.176942+0000","last_became_active":"2023-11-24T06:11:03.182745+0000","last_became_peered":"2023-11-24T06:11:03.182745+0000","last_unstale":"2023-11-24T06:11:04.176942+0000","last_undegraded":"2023-11-24T06:11:04.176942+0000","last_fullsized":"2023-11-24T06:11:04.176942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:10:38.154648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.194175+0000","last_change":"2023-11-24T06:11:04.188063+0000","last_active":"2023-11-24T06:11:04.194175+0000","last_peered":"2023-11-24T06:11:04.194175+0000","last_clean":"2023-11-24T06:11:04.194175+0000","last_became_active":"2023-11-24T06:11:04.187865+0000","last_became_peered":"2023-11-24T06:11:04.187865+0000","last_unstale":"2023-11-24T06:11:04.194175+0000","last_undegraded":"2023-11-24T06:11:04.194175+0000","last_fullsized":"2023-11-24T06:11:04.194175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:23.546146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.202571+0000","last_change":"2023-11-24T06:11:03.188747+0000","last_active":"2023-11-24T06:11:04.202571+0000","last_peered":"2023-11-24T06:11:04.202571+0000","last_clean":"2023-11-24T06:11:04.202571+0000","last_became_active":"2023-11-24T06:11:03.188526+0000","last_became_peered":"2023-11-24T06:11:03.188526+0000","last_unstale":"2023-11-24T06:11:04.202571+0000","last_undegraded":"2023-11-24T06:11:04.202571+0000","last_fullsized":"2023-11-24T06:11:04.202571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:04:29.610240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.205914+0000","last_change":"2023-11-24T06:11:04.202133+0000","last_active":"2023-11-24T06:11:04.205914+0000","last_peered":"2023-11-24T06:11:04.205914+0000","last_clean":"2023-11-24T06:11:04.205914+0000","last_became_active":"2023-11-24T06:11:04.200380+0000","last_became_peered":"2023-11-24T06:11:04.200380+0000","last_unstale":"2023-11-24T06:11:04.205914+0000","last_undegraded":"2023-11-24T06:11:04.205914+0000","last_fullsized":"2023-11-24T06:11:04.205914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:03.159139+0000","last_clean_scrub_stamp":"2023-11-24T06:11:03.159139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:51:04.213597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T06:11:04.183749+0000","last_change":"2023-11-24T06:11:03.182919+0000","last_active":"2023-11-24T06:11:04.183749+0000","last_peered":"2023-11-24T06:11:04.183749+0000","last_clean":"2023-11-24T06:11:04.183749+0000","last_became_active":"2023-11-24T06:11:03.182470+0000","last_became_peered":"2023-11-24T06:11:03.182470+0000","last_unstale":"2023-11-24T06:11:04.183749+0000","last_undegraded":"2023-11-24T06:11:04.183749+0000","last_fullsized":"2023-11-24T06:11:04.183749+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:02.155006+0000","last_clean_scrub_stamp":"2023-11-24T06:11:02.155006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:08.706979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":198,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-11-24T06:11:03.170894+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:11:03.170894+0000","last_peered":"2023-11-24T06:11:03.170894+0000","last_clean":"2023-11-24T06:11:03.170894+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:11:03.170894+0000","last_undegraded":"2023-11-24T06:11:03.170894+0000","last_fullsized":"2023-11-24T06:11:03.170894+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28304,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343536,"statfs":{"total":96636764160,"available":96607780864,"internally_reserved":0,"allocated":1310720,"data_stored":887248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.87,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68000000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.377}]},{"osd":5,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.373,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.41799999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.76800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34200000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.83099999999999996,"15min":0.88900000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.621,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":1.371,"15min":1.371},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66100000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.40300000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.83099999999999996,"15min":0.875},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70299999999999996,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":5.2830000000000004,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70199999999999996,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71599999999999997,"5min":1.407,"15min":1.407},"last":0.628}]},{"osd":4,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78400000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.33600000000000002,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45100000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.25900000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.48599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47399999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.375,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.52600000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.434},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.497,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71299999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.753}]},{"osd":6,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.46800000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.78100000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69599999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81100000000000005,"5min":0.88,"15min":0.88},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.75700000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.79200000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1290240,"data_stored":880313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.72599999999999998,"15min":0.79200000000000004},"min":{"1min":0.224,"5min":0.216,"15min":0.216},"max":{"1min":0.46100000000000002,"5min":12.378,"15min":12.378},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58799999999999997,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.501,"15min":0.52000000000000002},"min":{"1min":0.26200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.54300000000000004,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59899999999999998,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.39500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.41099999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.0920000000000001,"15min":1.169},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97499999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83099999999999996,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93300000000000005,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.621,"5min":1.091,"15min":1.169},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0289999999999999,"5min":15.356,"15min":15.356},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.72899999999999998,"15min":0.75},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83999999999999997,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.752,"15min":0.77400000000000002},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":4.726,"15min":4.726},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.755,"15min":0.77600000000000002},"min":{"1min":0.45100000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":1.129,"15min":1.2090000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":15.388,"15min":15.388},"last":0.69499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.214},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33500000000000002,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.49299999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.251}]},{"osd":1,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.25900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.38900000000000001,"15min":0.39200000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56299999999999994,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.29199999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72899999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.42299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.32200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0580000000000001,"5min":1.173,"15min":1.173},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99099999999999999,"5min":1.339,"15min":1.339},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":0.91100000000000003,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92500000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.018,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.502}]},{"osd":7,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.079,"5min":1.405,"15min":1.405},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95399999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.52100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":296776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.33600000000000002,"15min":0.34100000000000003},"min":{"1min":0.23799999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.42299999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.39400000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.28399999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.432,"5min":1.032,"15min":1.032},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.47899999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.52100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40100000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90100000000000002,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.878,"5min":1.728,"15min":1.728},"last":0.59699999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88500000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.161,"5min":1.222,"15min":1.222},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.499,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63300000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.39200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.79500000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.436},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.47199999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.497},{"interface":"front","average":{"1min":0.626,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":289945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89800000000000002,"5min":3.335,"15min":3.335},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.628}]},{"osd":1,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81299999999999994,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77600000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72299999999999998,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76200000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71899999999999997,"15min":0.73999999999999999},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.80700000000000005,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79300000000000004,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.503}]},{"osd":4,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76200000000000001,"5min":1.536,"15min":1.536},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.59999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82899999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.39400000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77300000000000002,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.42899999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80900000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.35999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:11:08.686 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:11:08.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:11:08.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.843+0000 7f4cb9778700 1 Processor -- start 2023-11-24T06:11:08.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.843+0000 7f4cb9778700 1 -- start start 2023-11-24T06:11:08.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb9778700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4cb415e140 0x7f4cb40d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb9778700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb40b98c0 0x7f4cb40d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb9778700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb40b80e0 con 0x7f4cb40d39d0 2023-11-24T06:11:08.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb9778700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb415e520 con 0x7f4cb415e140 2023-11-24T06:11:08.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb9778700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40d2930 con 0x7f4cb40b98c0 2023-11-24T06:11:08.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb27fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4cb40d39d0 0x7f4cb40b59b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41596/0 (socket says 172.21.15.97:41596) 2023-11-24T06:11:08.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb27fc700 1 -- 172.21.15.97:0/1836170807 learned_addr learned my addr 172.21.15.97:0/1836170807 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:08.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb2ffd700 1 --2- 172.21.15.97:0/1836170807 >> v2:172.21.15.183:3300/0 conn(0x7f4cb415e140 0x7f4cb40d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb37fe700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb40b98c0 0x7f4cb40d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940425414 0 0) 0x7f4ca4001110 con 0x7f4cb40d39d0 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.844+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c9c0035b0 con 0x7f4cb40d39d0 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4059736083 0 0) 0x7f4c9c0035b0 con 0x7f4cb40d39d0 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb40b98c0 msgr2=0x7f4cb40d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb40b98c0 0x7f4cb40d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 >> v2:172.21.15.183:3300/0 conn(0x7f4cb415e140 msgr2=0x7f4cb40d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 --2- 172.21.15.97:0/1836170807 >> v2:172.21.15.183:3300/0 conn(0x7f4cb415e140 0x7f4cb40d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb40d2c10 con 0x7f4cb40d39d0 2023-11-24T06:11:08.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f4ca4001800 con 0x7f4cb40d39d0 2023-11-24T06:11:08.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb2ffd700 1 --2- 172.21.15.97:0/1836170807 >> v2:172.21.15.183:3300/0 conn(0x7f4cb415e140 0x7f4cb40d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:08.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 >> v1:172.21.15.97:6789/0 conn(0x7f4cb40d39d0 legacy=0x7f4cb40b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb40b98c0 0x7f4c9c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c9c003f80 0x7f4c9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9c004d90 0x7f4c9c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb415e520 con 0x7f4c9c004d90 2023-11-24T06:11:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40d2930 con 0x7f4c9c003f80 2023-11-24T06:11:08.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c00ba10 con 0x7f4cb40b98c0 2023-11-24T06:11:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb37fe700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9c004d90 0x7f4c9c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb2ffd700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb40b98c0 0x7f4c9c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb27fc700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c9c003f80 0x7f4c9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb37fe700 1 -- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb40b98c0 msgr2=0x7f4c9c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb37fe700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb40b98c0 0x7f4c9c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb37fe700 1 -- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c9c003f80 msgr2=0x7f4c9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb37fe700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c9c003f80 0x7f4c9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb37fe700 1 -- 172.21.15.97:0/1836170807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0035b0 con 0x7f4c9c004d90 2023-11-24T06:11:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb37fe700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9c004d90 0x7f4c9c005210 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009fc0 tx=0x7f4c9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e88d9c47b0039ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00ebd0 con 0x7f4c9c004d90 2023-11-24T06:11:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.845+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac00f040 con 0x7f4c9c004d90 2023-11-24T06:11:08.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.846+0000 7f4cb1ffb700 1 -- 172.21.15.97:0/1836170807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac014480 con 0x7f4c9c004d90 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.846+0000 7f4cb9778700 1 -- 172.21.15.97:0/1836170807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9c004d90 msgr2=0x7f4c9c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.846+0000 7f4cb9778700 1 --2- 172.21.15.97:0/1836170807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c9c004d90 0x7f4c9c005210 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009fc0 tx=0x7f4c9c005880 comp rx=0 tx=0).stop 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.846+0000 7f4cb37fe700 1 -- 172.21.15.97:0/1836170807 reap_dead start 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.846+0000 7f4cb9778700 1 -- 172.21.15.97:0/1836170807 shutdown_connections 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.846+0000 7f4cb9778700 1 -- 172.21.15.97:0/1836170807 >> 172.21.15.97:0/1836170807 conn(0x7f4cb406b410 msgr2=0x7f4cb406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.846+0000 7f4cb9778700 1 -- 172.21.15.97:0/1836170807 shutdown_connections 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.846+0000 7f4cb9778700 1 -- 172.21.15.97:0/1836170807 wait complete. 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.847+0000 7f4cb9778700 1 Processor -- start 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.847+0000 7f4cb9778700 1 -- start start 2023-11-24T06:11:08.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb9778700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb415a6c0 0x7f4cb40aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb9778700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb40b49c0 0x7f4cb40b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb9778700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb40cb000 0x7f4cb40cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb9778700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40b5f00 con 0x7f4cb40cb000 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb9778700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40b5380 con 0x7f4cb40b49c0 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb9778700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb415aaa0 con 0x7f4cb415a6c0 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb40cb000 0x7f4cb40cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb40cb000 0x7f4cb40cb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49158/0 (socket says 172.21.15.97:49158) 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb37fe700 1 -- 172.21.15.97:0/4067912433 learned_addr learned my addr 172.21.15.97:0/4067912433 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb2ffd700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb415a6c0 0x7f4cb40aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb27fc700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb40b49c0 0x7f4cb40b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb37fe700 1 -- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb415a6c0 msgr2=0x7f4cb40aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb37fe700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb415a6c0 0x7f4cb40aa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb37fe700 1 -- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb40b49c0 msgr2=0x7f4cb40b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:08.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb37fe700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb40b49c0 0x7f4cb40b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb37fe700 1 -- 172.21.15.97:0/4067912433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb40cb9c0 con 0x7f4cb40cb000 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb2ffd700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb415a6c0 0x7f4cb40aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.848+0000 7f4cb27fc700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb40b49c0 0x7f4cb40b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.849+0000 7f4cb37fe700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb40cb000 0x7f4cb40cb480 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009f60 tx=0x7f4cac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.849+0000 7f4c9b7fe700 1 -- 172.21.15.97:0/4067912433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac007000 con 0x7f4cb40cb000 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.849+0000 7f4c9b7fe700 1 -- 172.21.15.97:0/4067912433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac003e70 con 0x7f4cb40cb000 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.849+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb40cbc30 con 0x7f4cb40cb000 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.849+0000 7f4c9b7fe700 1 -- 172.21.15.97:0/4067912433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac015b40 con 0x7f4cb40cb000 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.849+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb40cc580 con 0x7f4cb40cb000 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.850+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb4068fb0 con 0x7f4cb40cb000 2023-11-24T06:11:08.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.850+0000 7f4c9b7fe700 1 -- 172.21.15.97:0/4067912433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f4cac024070 con 0x7f4cb40cb000 2023-11-24T06:11:08.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.851+0000 7f4c9b7fe700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ca006b140 0x7f4ca006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:08.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.851+0000 7f4cb2ffd700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ca006b140 0x7f4ca006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:08.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.851+0000 7f4c9b7fe700 1 -- 172.21.15.97:0/4067912433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cac08b6a0 con 0x7f4cb40cb000 2023-11-24T06:11:08.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.851+0000 7f4cb2ffd700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ca006b140 0x7f4ca006d600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009d60 tx=0x7f4ca8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:08.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:08.855+0000 7f4c9b7fe700 1 -- 172.21.15.97:0/4067912433 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4cac0565a0 con 0x7f4cb40cb000 2023-11-24T06:11:09.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.123+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4cb41610d0 con 0x7f4cb40cb000 2023-11-24T06:11:09.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.125+0000 7f4c9b7fe700 1 -- 172.21.15.97:0/4067912433 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4cac0180b0 con 0x7f4cb40cb000 2023-11-24T06:11:09.129 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-11-24T06:11:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA== 2023-11-24T06:11:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-11-24T06:11:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-11-24T06:11:09.129 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:11:09.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ca006b140 msgr2=0x7f4ca006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ca006b140 0x7f4ca006d600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ca8009d60 tx=0x7f4ca8008040 comp rx=0 tx=0).stop 2023-11-24T06:11:09.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb40cb000 msgr2=0x7f4cb40cb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb40cb000 0x7f4cb40cb480 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4cac009f60 tx=0x7f4cac009750 comp rx=0 tx=0).stop 2023-11-24T06:11:09.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 shutdown_connections 2023-11-24T06:11:09.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cb40cb000 0x7f4cb40cb480 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ca006b140 0x7f4ca006d600 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4cb415a6c0 0x7f4cb40aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 --2- 172.21.15.97:0/4067912433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4cb40b49c0 0x7f4cb40b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 >> 172.21.15.97:0/4067912433 conn(0x7f4cb406b410 msgr2=0x7f4cb40b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:09.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 shutdown_connections 2023-11-24T06:11:09.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.127+0000 7f4cb9778700 1 -- 172.21.15.97:0/4067912433 wait complete. 2023-11-24T06:11:09.132 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-11-24T06:11:09.143 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:09.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.261+0000 7f10c26e0700 1 Processor -- start 2023-11-24T06:11:09.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.261+0000 7f10c26e0700 1 -- start start 2023-11-24T06:11:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10c26e0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc15e080 0x7f10bc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10c26e0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10bc0af4d0 0x7f10bc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10c26e0700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10bc06c000 0x7f10bc06c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10c26e0700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc0af0b0 con 0x7f10bc0af4d0 2023-11-24T06:11:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10c26e0700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc06c9f0 con 0x7f10bc06c000 2023-11-24T06:11:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10c26e0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0b05a0 con 0x7f10bc15e080 2023-11-24T06:11:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10bc0af4d0 0x7f10bc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10bc0af4d0 0x7f10bc0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49164/0 (socket says 172.21.15.97:49164) 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 -- 172.21.15.97:0/150551185 learned_addr learned my addr 172.21.15.97:0/150551185 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10c0c7d700 1 --2- 172.21.15.97:0/150551185 >> v2:172.21.15.183:3300/0 conn(0x7f10bc06c000 0x7f10bc06c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bbfff700 1 --2- 172.21.15.97:0/150551185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc15e080 0x7f10bc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 -- 172.21.15.97:0/150551185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc15e080 msgr2=0x7f10bc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 --2- 172.21.15.97:0/150551185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc15e080 0x7f10bc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 -- 172.21.15.97:0/150551185 >> v2:172.21.15.183:3300/0 conn(0x7f10bc06c000 msgr2=0x7f10bc06c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 --2- 172.21.15.97:0/150551185 >> v2:172.21.15.183:3300/0 conn(0x7f10bc06c000 0x7f10bc06c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 -- 172.21.15.97:0/150551185 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc0ae430 con 0x7f10bc0af4d0 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10c0c7d700 1 --2- 172.21.15.97:0/150551185 >> v2:172.21.15.183:3300/0 conn(0x7f10bc06c000 0x7f10bc06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10bb7fe700 1 --2- 172.21.15.97:0/150551185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc0af4d0 0x7f10bc0afb90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f10bc09eea0 tx=0x7f10b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d50b5f3c69f3ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10baffd700 1 -- 172.21.15.97:0/150551185 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b000b860 con 0x7f10bc0af4d0 2023-11-24T06:11:09.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10baffd700 1 -- 172.21.15.97:0/150551185 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b000b9c0 con 0x7f10bc0af4d0 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.262+0000 7f10baffd700 1 -- 172.21.15.97:0/150551185 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b000bd50 con 0x7f10bc0af4d0 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 -- 172.21.15.97:0/150551185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc0af4d0 msgr2=0x7f10bc0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150551185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc0af4d0 0x7f10bc0afb90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f10bc09eea0 tx=0x7f10b0009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 -- 172.21.15.97:0/150551185 shutdown_connections 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150551185 >> v2:172.21.15.183:3300/0 conn(0x7f10bc06c000 0x7f10bc06c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150551185 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc15e080 0x7f10bc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150551185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc0af4d0 0x7f10bc0afb90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 -- 172.21.15.97:0/150551185 >> 172.21.15.97:0/150551185 conn(0x7f10bc071b90 msgr2=0x7f10bc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 -- 172.21.15.97:0/150551185 shutdown_connections 2023-11-24T06:11:09.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.263+0000 7f10c26e0700 1 -- 172.21.15.97:0/150551185 wait complete. 2023-11-24T06:11:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.264+0000 7f10c26e0700 1 Processor -- start 2023-11-24T06:11:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.264+0000 7f10c26e0700 1 -- start start 2023-11-24T06:11:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c26e0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc06c000 0x7f10bc0cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c26e0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10bc0af4d0 0x7f10bc153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c26e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc15e080 0x7f10bc15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c26e0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc15f360 con 0x7f10bc15e080 2023-11-24T06:11:09.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c26e0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc15f630 con 0x7f10bc0af4d0 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c26e0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc15f910 con 0x7f10bc06c000 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c0c7d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc15e080 0x7f10bc15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c0c7d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc15e080 0x7f10bc15ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49174/0 (socket says 172.21.15.97:49174) 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c0c7d700 1 -- 172.21.15.97:0/150956382 learned_addr learned my addr 172.21.15.97:0/150956382 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10bbfff700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc06c000 0x7f10bc0cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10bb7fe700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10bc0af4d0 0x7f10bc153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.265+0000 7f10c0c7d700 1 -- 172.21.15.97:0/150956382 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc06c000 msgr2=0x7f10bc0cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10c0c7d700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc06c000 0x7f10bc0cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10c0c7d700 1 -- 172.21.15.97:0/150956382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10bc0af4d0 msgr2=0x7f10bc153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10c0c7d700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10bc0af4d0 0x7f10bc153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10c0c7d700 1 -- 172.21.15.97:0/150956382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0009550 con 0x7f10bc15e080 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10c0c7d700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc15e080 0x7f10bc15ee20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f10b4007e10 tx=0x7f10b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10b8ff9700 1 -- 172.21.15.97:0/150956382 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b400dd20 con 0x7f10bc15e080 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10b8ff9700 1 -- 172.21.15.97:0/150956382 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b400f040 con 0x7f10bc15e080 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc15fbf0 con 0x7f10bc15e080 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10b8ff9700 1 -- 172.21.15.97:0/150956382 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4003c60 con 0x7f10bc15e080 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.266+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc1601b0 con 0x7f10bc15e080 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.267+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc0753f0 con 0x7f10bc15e080 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.267+0000 7f10b8ff9700 1 -- 172.21.15.97:0/150956382 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f10b4004030 con 0x7f10bc15e080 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.268+0000 7f10b8ff9700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f10a806b110 0x7f10a806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.268+0000 7f10b8ff9700 1 -- 172.21.15.97:0/150956382 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10b4089a50 con 0x7f10bc15e080 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.268+0000 7f10bbfff700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f10a806b110 0x7f10a806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:09.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.268+0000 7f10bbfff700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f10a806b110 0x7f10a806d5d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f10ac006b20 tx=0x7f10ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:09.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.272+0000 7f10b8ff9700 1 -- 172.21.15.97:0/150956382 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10b4054720 con 0x7f10bc15e080 2023-11-24T06:11:09.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.505+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10bc072cf0 con 0x7f10bc15e080 2023-11-24T06:11:09.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.506+0000 7f10b8ff9700 1 -- 172.21.15.97:0/150956382 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f10b401a090 con 0x7f10bc15e080 2023-11-24T06:11:09.509 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:09.509 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3508641674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3508641674},{"type":"v1","addr":"172.21.15.183:6833","nonce":3508641674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5557,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/116508824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":116508824},{"type":"v1","addr":"172.21.15.183:6835","nonce":116508824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2785455707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2785455707},{"type":"v1","addr":"172.21.15.97:6837","nonce":2785455707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:03.572274+0000","modified":"2023-11-24T06:11:06.595718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3662495999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3662495999},{"type":"v1","addr":"172.21.15.97:6836","nonce":3662495999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:11:09.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.507+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f10a806b110 msgr2=0x7f10a806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.507+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f10a806b110 0x7f10a806d5d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f10ac006b20 tx=0x7f10ac008040 comp rx=0 tx=0).stop 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.507+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc15e080 msgr2=0x7f10bc15ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.507+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc15e080 0x7f10bc15ee20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f10b4007e10 tx=0x7f10b400d3b0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.508+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 shutdown_connections 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.508+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10bc15e080 0x7f10bc15ee20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.508+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f10a806b110 0x7f10a806d5d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.508+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10bc06c000 0x7f10bc0cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.508+0000 7f10c26e0700 1 --2- 172.21.15.97:0/150956382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10bc0af4d0 0x7f10bc153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.508+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 >> 172.21.15.97:0/150956382 conn(0x7f10bc071b90 msgr2=0x7f10bc073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.508+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 shutdown_connections 2023-11-24T06:11:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:09.508+0000 7f10c26e0700 1 -- 172.21.15.97:0/150956382 wait complete. 2023-11-24T06:11:09.513 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 36 2023-11-24T06:11:09.926 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:11:03.572274+0000', 'modified': '2023-11-24T06:11:06.595718+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6003}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6003': {'gid': 6003, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6836/3662495999', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 3662495999}, {'type': 'v1', 'addr': '172.21.15.97:6836', 'nonce': 3662495999}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:11:09.926 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:11:09.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T06:11:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.046+0000 7f09a0358700 1 Processor -- start 2023-11-24T06:11:10.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.046+0000 7f09a0358700 1 -- start start 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.047+0000 7f09a0358700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f09980cf2a0 0x7f0998168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.047+0000 7f09a0358700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099806bc00 con 0x7f09980b2b60 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.047+0000 7f09a0358700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099815e4d0 con 0x7f099815e0f0 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.047+0000 7f09a0358700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099806aff0 con 0x7f09980cf2a0 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.047+0000 7f099d8f3700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f09980b2b60 0x7f09980ceb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41632/0 (socket says 172.21.15.97:41632) 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.047+0000 7f099d8f3700 1 -- 172.21.15.97:0/2749838825 learned_addr learned my addr 172.21.15.97:0/2749838825 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099e8f5700 1 --2- 172.21.15.97:0/2749838825 >> v2:172.21.15.183:3300/0 conn(0x7f09980cf2a0 0x7f0998168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099d0f2700 1 -- 172.21.15.97:0/2749838825 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1979557747 0 0) 0x7f099806bc00 con 0x7f09980b2b60 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099d0f2700 1 -- 172.21.15.97:0/2749838825 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09800035b0 con 0x7f09980b2b60 2023-11-24T06:11:10.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099e8f5700 1 -- 172.21.15.97:0/2749838825 >> v1:172.21.15.183:6789/0 conn(0x7f099815e0f0 legacy=0x7f09980b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099e8f5700 1 -- 172.21.15.97:0/2749838825 >> v1:172.21.15.97:6789/0 conn(0x7f09980b2b60 legacy=0x7f09980ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099e8f5700 1 -- 172.21.15.97:0/2749838825 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099806b2d0 con 0x7f09980cf2a0 2023-11-24T06:11:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099e8f5700 1 --2- 172.21.15.97:0/2749838825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980cf2a0 0x7f0998168f10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0988009840 tx=0x7f0988009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d91ad94d7ef8e816 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099d0f2700 1 -- 172.21.15.97:0/2749838825 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988011830 con 0x7f09980cf2a0 2023-11-24T06:11:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099d0f2700 1 -- 172.21.15.97:0/2749838825 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0988011990 con 0x7f09980cf2a0 2023-11-24T06:11:10.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.048+0000 7f099d0f2700 1 -- 172.21.15.97:0/2749838825 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988011d20 con 0x7f09980cf2a0 2023-11-24T06:11:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.049+0000 7f09a0358700 1 -- 172.21.15.97:0/2749838825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980cf2a0 msgr2=0x7f0998168f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.049+0000 7f09a0358700 1 --2- 172.21.15.97:0/2749838825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980cf2a0 0x7f0998168f10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0988009840 tx=0x7f0988009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.049+0000 7f09a0358700 1 -- 172.21.15.97:0/2749838825 shutdown_connections 2023-11-24T06:11:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.049+0000 7f09a0358700 1 --2- 172.21.15.97:0/2749838825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09980cf2a0 0x7f0998168f10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.049+0000 7f09a0358700 1 -- 172.21.15.97:0/2749838825 >> 172.21.15.97:0/2749838825 conn(0x7f09980b5ce0 msgr2=0x7f09980b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.049+0000 7f09a0358700 1 -- 172.21.15.97:0/2749838825 shutdown_connections 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.049+0000 7f09a0358700 1 -- 172.21.15.97:0/2749838825 wait complete. 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.050+0000 7f09a0358700 1 Processor -- start 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.050+0000 7f09a0358700 1 -- start start 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f09a0358700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09980b2b60 0x7f0998158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f09a0358700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf2a0 0x7f099815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f09a0358700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099815e0f0 0x7f099815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f09a0358700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0998159050 con 0x7f09980b2b60 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f09a0358700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0998153140 con 0x7f099815e0f0 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f09a0358700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099815fc30 con 0x7f09980cf2a0 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09980b2b60 0x7f0998158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09980b2b60 0x7f0998158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49184/0 (socket says 172.21.15.97:49184) 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.051+0000 7f099e0f4700 1 -- 172.21.15.97:0/127344362 learned_addr learned my addr 172.21.15.97:0/127344362 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:10.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099d8f3700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf2a0 0x7f099815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099e8f5700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099815e0f0 0x7f099815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099e0f4700 1 -- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf2a0 msgr2=0x7f099815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099e0f4700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf2a0 0x7f099815ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099e0f4700 1 -- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099815e0f0 msgr2=0x7f099815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099e0f4700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099815e0f0 0x7f099815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099e0f4700 1 -- 172.21.15.97:0/127344362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09880094a0 con 0x7f09980b2b60 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099e8f5700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099815e0f0 0x7f099815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099d8f3700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf2a0 0x7f099815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f099e0f4700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09980b2b60 0x7f0998158b10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f099000cbd0 tx=0x7f099000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f0996ffd700 1 -- 172.21.15.97:0/127344362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990003c00 con 0x7f09980b2b60 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f0996ffd700 1 -- 172.21.15.97:0/127344362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0990003d60 con 0x7f09980b2b60 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f09a0358700 1 -- 172.21.15.97:0/127344362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099815ff10 con 0x7f09980b2b60 2023-11-24T06:11:10.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f0996ffd700 1 -- 172.21.15.97:0/127344362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099000abf0 con 0x7f09980b2b60 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.052+0000 7f09a0358700 1 -- 172.21.15.97:0/127344362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09981604d0 con 0x7f09980b2b60 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.053+0000 7f0996ffd700 1 -- 172.21.15.97:0/127344362 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f099000ad50 con 0x7f09980b2b60 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.054+0000 7f0996ffd700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f098406b110 0x7f098406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.054+0000 7f0996ffd700 1 -- 172.21.15.97:0/127344362 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0990089a90 con 0x7f09980b2b60 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.054+0000 7f099d8f3700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f098406b110 0x7f098406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.054+0000 7f099d8f3700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f098406b110 0x7f098406d5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f098c0011e0 tx=0x7f098c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.056+0000 7f09ac551b80 1 Processor -- start 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.056+0000 7f09ac551b80 1 -- start start 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c4080 0x558bcf70a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf70ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c1490 0x558bcf6ede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f09ac551b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bcf637770 con 0x558bcf6c4080 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f09ac551b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bcf61e750 con 0x558bcf6c1490 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f09ac551b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bcf61c070 con 0x558bcf6d9ed0 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c4080 0x558bcf70a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c4080 0x558bcf70a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49192/0 (socket says 172.21.15.97:49192) 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 -- 172.21.15.97:0/2371733692 learned_addr learned my addr 172.21.15.97:0/2371733692 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099d8f3700 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf70ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e8f5700 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c1490 0x558bcf6ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 -- 172.21.15.97:0/2371733692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 msgr2=0x558bcf70ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf70ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 -- 172.21.15.97:0/2371733692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c1490 msgr2=0x558bcf6ede70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c1490 0x558bcf6ede70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 -- 172.21.15.97:0/2371733692 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bcf4945e0 con 0x558bcf6c4080 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.057+0000 7f099e0f4700 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c4080 0x558bcf70a840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0990054a30 tx=0x7f0990058c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476e2d5a0711872 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f097ffff700 1 -- 172.21.15.97:0/2371733692 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990060830 con 0x558bcf6c4080 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f097ffff700 1 -- 172.21.15.97:0/2371733692 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0990058790 con 0x558bcf6c4080 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f097ffff700 1 -- 172.21.15.97:0/2371733692 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990057a20 con 0x558bcf6c4080 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 -- 172.21.15.97:0/2371733692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c4080 msgr2=0x558bcf70a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c4080 0x558bcf70a840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0990054a30 tx=0x7f0990058c30 comp rx=0 tx=0).stop 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 -- 172.21.15.97:0/2371733692 shutdown_connections 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c1490 0x558bcf6ede70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c4080 0x558bcf70a840 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 --2- 172.21.15.97:0/2371733692 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf70ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 -- 172.21.15.97:0/2371733692 >> 172.21.15.97:0/2371733692 conn(0x558bcf5f2310 msgr2=0x558bcf708e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 -- 172.21.15.97:0/2371733692 shutdown_connections 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.058+0000 7f09ac551b80 1 -- 172.21.15.97:0/2371733692 wait complete. 2023-11-24T06:11:10.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.059+0000 7f09ac551b80 1 -- start start 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.059+0000 7f09ac551b80 1 Processor -- start 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf6d9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf701a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf701f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f09ac551b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bcf617960 con 0x558bcf6c1490 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f09ac551b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bcf5d1510 con 0x558bcf6c4080 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f09ac551b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bcf5d11b0 con 0x558bcf6d9ed0 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f09ac551b80 10 client.0 _ll_register_callbacks cb 0x558bcf4fca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf6d9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf6d9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49198/0 (socket says 172.21.15.97:49198) 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e0f4700 1 -- 172.21.15.97:0/1988572426 learned_addr learned my addr 172.21.15.97:0/1988572426 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:10.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099d8f3700 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf701a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e8f5700 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf701f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e0f4700 1 -- 172.21.15.97:0/1988572426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 msgr2=0x558bcf701f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e0f4700 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf701f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e0f4700 1 -- 172.21.15.97:0/1988572426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 msgr2=0x558bcf701a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e0f4700 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf701a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.060+0000 7f099e0f4700 1 -- 172.21.15.97:0/1988572426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09981604d0 con 0x558bcf6c1490 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f099e0f4700 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf6d9840 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0990058470 tx=0x7f0990011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3409b20180e4f19c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f097d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f097d7fa700 1 -- 172.21.15.97:0/1988572426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990057c20 con 0x558bcf6c1490 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f097d7fa700 1 -- 172.21.15.97:0/1988572426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0990078030 con 0x558bcf6c1490 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 1 -- 172.21.15.97:0/1988572426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bcf6b6a40 con 0x558bcf6c1490 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f097d7fa700 1 -- 172.21.15.97:0/1988572426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09900603a0 con 0x558bcf6c1490 2023-11-24T06:11:10.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 10 client.6141 fetch_fsmap learned FSMap version 36 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 1 -- 172.21.15.97:0/1988572426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x558bcf440d90 con 0x558bcf6c1490 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f097d7fa700 1 -- 172.21.15.97:0/1988572426 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558bcf6b6a40 con 0x558bcf6c1490 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f097d7fa700 1 -- 172.21.15.97:0/1988572426 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f099005c170 con 0x558bcf6c1490 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 10 client.6141 fetch_fsmap finished waiting for FSMap version 36 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 10 client.6141 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 20 client.6141 populate_metadata read hostname 'smithi097' 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] conn(0x558bcf6f0360 0x558bcf6f0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 4 client.6141 mds_command: new command op to 6003 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f09ac551b80 1 -- 172.21.15.97:0/1988572426 --> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bcf692840 con 0x558bcf6f0360 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f099d8f3700 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] conn(0x558bcf6f0360 0x558bcf6f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f097d7fa700 10 client.6141 ms_handle_connect on v2:172.21.15.97:6834/3662495999 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.061+0000 7f099d8f3700 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] conn(0x558bcf6f0360 0x558bcf6f0740 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd23ef234fdb06f server_cookie=aacb1f964fe39fbd in_seq=0 out_seq=0 2023-11-24T06:11:10.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f097d7fa700 1 -- 172.21.15.97:0/1988572426 <== mds.0 v2:172.21.15.97:6834/3662495999 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558bcf692840 con 0x558bcf6f0360 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f097d7fa700 10 client.6141 handle_command_reply: tid=0 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 client.6141 shutdown 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 -- 172.21.15.97:0/1988572426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 msgr2=0x558bcf6d9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf6d9840 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0990058470 tx=0x7f0990011000 comp rx=0 tx=0).stop 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 -- 172.21.15.97:0/1988572426 shutdown_connections 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] conn(0x558bcf6f0360 0x558bcf6f0740 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf701f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf6d9840 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f097d7fa700 0 client.6141 ms_handle_reset on v2:172.21.15.97:6834/3662495999 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 --2- 172.21.15.97:0/1988572426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf701a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 -- 172.21.15.97:0/1988572426 >> 172.21.15.97:0/1988572426 conn(0x558bcf5f2310 msgr2=0x558bcf6be470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 -- 172.21.15.97:0/1988572426 shutdown_connections 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.062+0000 7f09ac551b80 1 -- 172.21.15.97:0/1988572426 wait complete. 2023-11-24T06:11:10.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.063+0000 7f09ac551b80 20 client.6141 trim_cache size 0 max 16384 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.080+0000 7f09ac551b80 1 Processor -- start 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.080+0000 7f09ac551b80 1 -- start start 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf7027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf70c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf6f0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f09ac551b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bcf4968d0 con 0x558bcf6c1490 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f09ac551b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bcf6f14b0 con 0x558bcf6c4080 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f09ac551b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bcf705320 con 0x558bcf6d9ed0 2023-11-24T06:11:10.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf7027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf7027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49206/0 (socket says 172.21.15.97:49206) 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 -- 172.21.15.97:0/691079654 learned_addr learned my addr 172.21.15.97:0/691079654 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 -- 172.21.15.97:0/691079654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 msgr2=0x558bcf6f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 --2- 172.21.15.97:0/691079654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf6f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 -- 172.21.15.97:0/691079654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 msgr2=0x558bcf70c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 --2- 172.21.15.97:0/691079654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf70c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 -- 172.21.15.97:0/691079654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bcf429c10 con 0x558bcf6c1490 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.081+0000 7f099e0f4700 1 --2- 172.21.15.97:0/691079654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf7027f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0990056880 tx=0x7f0990083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f70d5f9f41e82d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f0994d82700 1 -- 172.21.15.97:0/691079654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990058530 con 0x558bcf6c1490 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f0994d82700 1 -- 172.21.15.97:0/691079654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0990001cb0 con 0x558bcf6c1490 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f0994d82700 1 -- 172.21.15.97:0/691079654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0990057a20 con 0x558bcf6c1490 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 -- 172.21.15.97:0/691079654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 msgr2=0x558bcf7027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 --2- 172.21.15.97:0/691079654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf7027f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0990056880 tx=0x7f0990083040 comp rx=0 tx=0).stop 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 -- 172.21.15.97:0/691079654 shutdown_connections 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 --2- 172.21.15.97:0/691079654 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf6f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 --2- 172.21.15.97:0/691079654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf7027f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 --2- 172.21.15.97:0/691079654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf70c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 -- 172.21.15.97:0/691079654 >> 172.21.15.97:0/691079654 conn(0x558bcf5f2310 msgr2=0x558bcf702f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 -- 172.21.15.97:0/691079654 shutdown_connections 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.082+0000 7f09ac551b80 1 -- 172.21.15.97:0/691079654 wait complete. 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.083+0000 7f09ac551b80 1 -- start start 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.083+0000 7f09ac551b80 1 Processor -- start 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf717e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf705a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf705fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bcf7732f0 con 0x558bcf6c1490 2023-11-24T06:11:10.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bcf703f90 con 0x558bcf6c4080 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bcf7015d0 con 0x558bcf6d9ed0 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 10 client.0 _ll_register_callbacks cb 0x558bcf4fca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf717e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf717e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49212/0 (socket says 172.21.15.97:49212) 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 -- 172.21.15.97:0/143986346 learned_addr learned my addr 172.21.15.97:0/143986346 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099d8f3700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf705a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e8f5700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf705fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 -- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 msgr2=0x558bcf705fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf705fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 -- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 msgr2=0x558bcf705a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf705a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 -- 172.21.15.97:0/143986346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bcf440d90 con 0x558bcf6c1490 2023-11-24T06:11:10.101 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099d8f3700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf705a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e8f5700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf705fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f099e0f4700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf717e40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f09900586f0 tx=0x7f09900549a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b1d8d6f55f1656 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f097d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f097d7fa700 1 -- 172.21.15.97:0/143986346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099005c170 con 0x558bcf6c1490 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f097d7fa700 1 -- 172.21.15.97:0/143986346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0990078040 con 0x558bcf6c1490 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 1 -- 172.21.15.97:0/143986346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bcf5ddf80 con 0x558bcf6c1490 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f097d7fa700 1 -- 172.21.15.97:0/143986346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f099005c2d0 con 0x558bcf6c1490 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.084+0000 7f09ac551b80 10 client.6147 fetch_fsmap learned FSMap version 36 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 1 -- 172.21.15.97:0/143986346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x558bcf467200 con 0x558bcf6c1490 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f097d7fa700 1 -- 172.21.15.97:0/143986346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558bcf5ddf80 con 0x558bcf6c1490 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f097d7fa700 1 -- 172.21.15.97:0/143986346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0990057a20 con 0x558bcf6c1490 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 10 client.6147 fetch_fsmap finished waiting for FSMap version 36 2023-11-24T06:11:10.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 10 client.6147 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 20 client.6147 populate_metadata read hostname 'smithi097' 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] conn(0x558bcf6ee4f0 0x558bcf772120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 4 client.6147 mds_command: new command op to 6003 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 1 -- 172.21.15.97:0/143986346 --> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558bcf5d0440 con 0x558bcf6ee4f0 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f099d8f3700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] conn(0x558bcf6ee4f0 0x558bcf772120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f097d7fa700 10 client.6147 ms_handle_connect on v2:172.21.15.97:6834/3662495999 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f099d8f3700 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] conn(0x558bcf6ee4f0 0x558bcf772120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12f73b99ff2b6c5 server_cookie=d78cae414bd95106 in_seq=0 out_seq=0 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f097d7fa700 1 -- 172.21.15.97:0/143986346 <== mds.0 v2:172.21.15.97:6834/3662495999 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x558bcf5d0440 con 0x558bcf6ee4f0 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f097d7fa700 10 client.6147 handle_command_reply: tid=0 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 1 client.6147 shutdown 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 1 -- 172.21.15.97:0/143986346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 msgr2=0x558bcf717e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.085+0000 7f09ac551b80 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf717e40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f09900586f0 tx=0x7f09900549a0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 1 -- 172.21.15.97:0/143986346 shutdown_connections 2023-11-24T06:11:10.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.97:6834/3662495999,v1:172.21.15.97:6836/3662495999] conn(0x558bcf6ee4f0 0x558bcf772120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x558bcf6d9ed0 0x558bcf705fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bcf6c1490 0x558bcf717e40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 1 --2- 172.21.15.97:0/143986346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bcf6c4080 0x558bcf705a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f097d7fa700 0 client.6147 ms_handle_reset on v2:172.21.15.97:6834/3662495999 2023-11-24T06:11:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 1 -- 172.21.15.97:0/143986346 >> 172.21.15.97:0/143986346 conn(0x558bcf5f2310 msgr2=0x558bcf70ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 1 -- 172.21.15.97:0/143986346 shutdown_connections 2023-11-24T06:11:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 1 -- 172.21.15.97:0/143986346 wait complete. 2023-11-24T06:11:10.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09ac551b80 20 client.6147 trim_cache size 0 max 16384 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9", 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6003, 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.9199752099999996, 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 36, 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 60, 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 60, 2023-11-24T06:11:10.105 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.875218626000001 2023-11-24T06:11:10.106 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:11:10.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 -- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f098406b110 msgr2=0x7f098406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f098406b110 0x7f098406d5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f098c0011e0 tx=0x7f098c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:10.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 -- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09980b2b60 msgr2=0x7f0998158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09980b2b60 0x7f0998158b10 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f099000cbd0 tx=0x7f099000cf90 comp rx=0 tx=0).stop 2023-11-24T06:11:10.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 -- 172.21.15.97:0/127344362 shutdown_connections 2023-11-24T06:11:10.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f099815e0f0 0x7f099815b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09980b2b60 0x7f0998158b10 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f098406b110 0x7f098406d5d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 --2- 172.21.15.97:0/127344362 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09980cf2a0 0x7f099815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 -- 172.21.15.97:0/127344362 >> 172.21.15.97:0/127344362 conn(0x7f09980b5ce0 msgr2=0x7f09980d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 -- 172.21.15.97:0/127344362 shutdown_connections 2023-11-24T06:11:10.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.086+0000 7f09a0358700 1 -- 172.21.15.97:0/127344362 wait complete. 2023-11-24T06:11:10.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:11:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.225+0000 7f09f6f3d700 1 Processor -- start 2023-11-24T06:11:10.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.226+0000 7f09f6f3d700 1 -- start start 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.226+0000 7f09f6f3d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f0145510 0x7f09f0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.226+0000 7f09f6f3d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f09f00ceb70 0x7f09f006eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.226+0000 7f09f6f3d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f015fb30 con 0x7f09f00744d0 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.226+0000 7f09f6f3d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f0074960 con 0x7f09f0145510 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.226+0000 7f09f6f3d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f015fd80 con 0x7f09f00ceb70 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.226+0000 7f09f4cd9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f0145510 0x7f09f0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09effff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f09f00744d0 0x7f09f00ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41672/0 (socket says 172.21.15.97:41672) 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09f4cd9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f09f0145510 0x7f09f0073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49220/0 (socket says 172.21.15.97:49220) 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09effff700 1 -- 172.21.15.97:0/3005725481 learned_addr learned my addr 172.21.15.97:0/3005725481 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09f4cd9700 1 -- 172.21.15.97:0/3005725481 >> v2:172.21.15.183:3300/0 conn(0x7f09f00ceb70 msgr2=0x7f09f006eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09f4cd9700 1 --2- 172.21.15.97:0/3005725481 >> v2:172.21.15.183:3300/0 conn(0x7f09f00ceb70 0x7f09f006eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09f4cd9700 1 -- 172.21.15.97:0/3005725481 >> v1:172.21.15.97:6789/0 conn(0x7f09f00744d0 legacy=0x7f09f00ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09f4cd9700 1 -- 172.21.15.97:0/3005725481 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f0160060 con 0x7f09f0145510 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09ef7fe700 1 -- 172.21.15.97:0/3005725481 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685461892 0 0) 0x7f09f015fb30 con 0x7f09f00744d0 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09f4cd9700 1 --2- 172.21.15.97:0/3005725481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f0145510 0x7f09f0073f90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f09f00d00d0 tx=0x7f09e4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8371175cdda7689b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09ef7fe700 1 -- 172.21.15.97:0/3005725481 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4011830 con 0x7f09f0145510 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09ef7fe700 1 -- 172.21.15.97:0/3005725481 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e4011990 con 0x7f09f0145510 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.227+0000 7f09ef7fe700 1 -- 172.21.15.97:0/3005725481 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4011d20 con 0x7f09f0145510 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.228+0000 7f09f6f3d700 1 -- 172.21.15.97:0/3005725481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f0145510 msgr2=0x7f09f0073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.228+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/3005725481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f0145510 0x7f09f0073f90 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f09f00d00d0 tx=0x7f09e4009b20 comp rx=0 tx=0).stop 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.228+0000 7f09f6f3d700 1 -- 172.21.15.97:0/3005725481 shutdown_connections 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.228+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/3005725481 >> v2:172.21.15.183:3300/0 conn(0x7f09f00ceb70 0x7f09f006eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.228+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/3005725481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f0145510 0x7f09f0073f90 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.228+0000 7f09f6f3d700 1 -- 172.21.15.97:0/3005725481 >> 172.21.15.97:0/3005725481 conn(0x7f09f0072990 msgr2=0x7f09f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.228+0000 7f09f6f3d700 1 -- 172.21.15.97:0/3005725481 shutdown_connections 2023-11-24T06:11:10.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.228+0000 7f09f6f3d700 1 -- 172.21.15.97:0/3005725481 wait complete. 2023-11-24T06:11:10.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.229+0000 7f09f6f3d700 1 Processor -- start 2023-11-24T06:11:10.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.229+0000 7f09f6f3d700 1 -- start start 2023-11-24T06:11:10.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f6f3d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09f00744d0 0x7f09f01e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f6f3d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f00ceb70 0x7f09f01e7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f6f3d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09f0145510 0x7f09f013c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f6f3d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f0140e70 con 0x7f09f00ceb70 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f6f3d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f0140b90 con 0x7f09f0145510 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f6f3d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f01e6300 con 0x7f09f00744d0 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f00ceb70 0x7f09f01e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f00ceb70 0x7f09f01e7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49228/0 (socket says 172.21.15.97:49228) 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 -- 172.21.15.97:0/2650016855 learned_addr learned my addr 172.21.15.97:0/2650016855 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f54da700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09f0145510 0x7f09f013c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 -- 172.21.15.97:0/2650016855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09f00744d0 msgr2=0x7f09f01e6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09f00744d0 0x7f09f01e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 -- 172.21.15.97:0/2650016855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09f0145510 msgr2=0x7f09f013c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09f0145510 0x7f09f013c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 -- 172.21.15.97:0/2650016855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e40094a0 con 0x7f09f00ceb70 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09effff700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f00ceb70 0x7f09f01e7670 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c550 tx=0x7f09dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09ed7fa700 1 -- 172.21.15.97:0/2650016855 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc004c80 con 0x7f09f00ceb70 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09ed7fa700 1 -- 172.21.15.97:0/2650016855 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc004e00 con 0x7f09f00ceb70 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f01e65e0 con 0x7f09f00ceb70 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09ed7fa700 1 -- 172.21.15.97:0/2650016855 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc007310 con 0x7f09f00ceb70 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.230+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f01e7bb0 con 0x7f09f00ceb70 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.231+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f01e6c30 con 0x7f09f00ceb70 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.232+0000 7f09ed7fa700 1 -- 172.21.15.97:0/2650016855 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f09dc007470 con 0x7f09f00ceb70 2023-11-24T06:11:10.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.232+0000 7f09ed7fa700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f09d406b110 0x7f09d406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:10.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.232+0000 7f09ed7fa700 1 -- 172.21.15.97:0/2650016855 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09dc089690 con 0x7f09f00ceb70 2023-11-24T06:11:10.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.232+0000 7f09f4cd9700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f09d406b110 0x7f09d406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:10.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.233+0000 7f09f4cd9700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f09d406b110 0x7f09d406d5d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f09e4000c00 tx=0x7f09e4010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:10.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.237+0000 7f09ed7fa700 1 -- 172.21.15.97:0/2650016855 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09dc054470 con 0x7f09f00ceb70 2023-11-24T06:11:10.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.468+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f09f0067f60 con 0x7f09f00ceb70 2023-11-24T06:11:10.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.469+0000 7f09ed7fa700 1 -- 172.21.15.97:0/2650016855 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) v1 ==== 53+0+73 (secure 0 0 0) 0x7f09dc057a90 con 0x7f09f00ceb70 2023-11-24T06:11:10.473 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f09d406b110 msgr2=0x7f09d406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f09d406b110 0x7f09d406d5d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f09e4000c00 tx=0x7f09e4010000 comp rx=0 tx=0).stop 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f00ceb70 msgr2=0x7f09f01e7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f00ceb70 0x7f09f01e7670 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f09dc00c550 tx=0x7f09dc00c910 comp rx=0 tx=0).stop 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 shutdown_connections 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09f0145510 0x7f09f013c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f09d406b110 0x7f09d406d5d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f09f00744d0 0x7f09f01e6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 --2- 172.21.15.97:0/2650016855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09f00ceb70 0x7f09f01e7670 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 >> 172.21.15.97:0/2650016855 conn(0x7f09f0072990 msgr2=0x7f09f006bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:10.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 shutdown_connections 2023-11-24T06:11:10.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:10.471+0000 7f09f6f3d700 1 -- 172.21.15.97:0/2650016855 wait complete. 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:11:10.487 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:10.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:10.488 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-24T06:11:10.504 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:11:10.505 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:11:10.505 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:11:10.505 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:10.505 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:11:10.505 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:10.505 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:10.505 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42066sec preferred_lft 42066sec 2023-11-24T06:11:10.506 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c9d4:801c:12b3:b866/64 scope link noprefixroute 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::7c25:b1ff:fe16:36f1/64 scope link 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether a2:28:0d:2d:82:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a028:dff:fe2d:822b/64 scope link 2023-11-24T06:11:10.507 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:10.508 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:10.508 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:11:10.508 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::1875:dff:fec0:4af7/64 scope link 2023-11-24T06:11:10.508 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:10.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:10.508 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:11:10.569 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-24T06:11:10.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:10.571 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:11:10.632 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:11:10.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:10.633 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:11:10.633 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:11:10.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:11:10.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:11:10.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:11:10.633 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:11:10.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:10.737 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:11:10.753 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:11:10.753 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:11:10.754 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:10.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:10.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-24T06:11:10.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:10.820 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:10.877 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:10.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:10.877 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:10.934 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:11:10.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:10.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:11.020 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:11.021 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:11.047 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:11.047 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:11.048 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:11:11.048 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:11:11.048 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:11:11.048 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:11:11.048 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:11:11.048 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:11:11.085 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:11.085 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:11.085 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:11:11.085 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:11:11.085 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:11:11.086 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:11:11.086 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:11:11.086 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:11:11.086 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:11:11.086 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:11:11.087 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:11:11.091 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:11:11.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:11.092 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:11.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:11.123 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:11:11.209 INFO:teuthology.orchestra.run.smithi097.stdout:client5665 2023-11-24T06:11:11.209 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:11.209 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:11.227 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:11:11.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.384+0000 7ffb19362700 1 Processor -- start 2023-11-24T06:11:11.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.384+0000 7ffb19362700 1 -- start start 2023-11-24T06:11:11.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb19362700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb1415c140 0x7ffb140726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb19362700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ffb140af780 0x7ffb14072c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb19362700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb140adfd0 con 0x7ffb140622a0 2023-11-24T06:11:11.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb19362700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb140afbf0 con 0x7ffb140af780 2023-11-24T06:11:11.390 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb19362700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140ae2b0 con 0x7ffb1415c140 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb12ffd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb1415c140 0x7ffb140726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb137fe700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ffb140af780 0x7ffb14072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb127fc700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7ffb140622a0 0x7ffb140af070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51012/0 (socket says 172.21.15.97:51012) 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb127fc700 1 -- 172.21.15.97:0/1903034038 learned_addr learned my addr 172.21.15.97:0/1903034038 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.385+0000 7ffb12ffd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb1415c140 0x7ffb140726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:36274/0 (socket says 172.21.15.97:36274) 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb12ffd700 1 -- 172.21.15.97:0/1903034038 >> v2:172.21.15.183:3300/0 conn(0x7ffb140af780 msgr2=0x7ffb14072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb12ffd700 1 --2- 172.21.15.97:0/1903034038 >> v2:172.21.15.183:3300/0 conn(0x7ffb140af780 0x7ffb14072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb12ffd700 1 -- 172.21.15.97:0/1903034038 >> v1:172.21.15.183:6789/0 conn(0x7ffb140622a0 legacy=0x7ffb140af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb12ffd700 1 -- 172.21.15.97:0/1903034038 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140ae590 con 0x7ffb1415c140 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb137fe700 1 --2- 172.21.15.97:0/1903034038 >> v2:172.21.15.183:3300/0 conn(0x7ffb140af780 0x7ffb14072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb12ffd700 1 --2- 172.21.15.97:0/1903034038 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb1415c140 0x7ffb140726e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009840 tx=0x7ffb08009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa8e1e98a6c0f65f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb11ffb700 1 -- 172.21.15.97:0/1903034038 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08011830 con 0x7ffb1415c140 2023-11-24T06:11:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb11ffb700 1 -- 172.21.15.97:0/1903034038 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb08011990 con 0x7ffb1415c140 2023-11-24T06:11:11.392 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.386+0000 7ffb11ffb700 1 -- 172.21.15.97:0/1903034038 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08011d20 con 0x7ffb1415c140 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.387+0000 7ffb19362700 1 -- 172.21.15.97:0/1903034038 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb1415c140 msgr2=0x7ffb140726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.387+0000 7ffb19362700 1 --2- 172.21.15.97:0/1903034038 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb1415c140 0x7ffb140726e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009840 tx=0x7ffb08009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.387+0000 7ffb19362700 1 -- 172.21.15.97:0/1903034038 shutdown_connections 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.387+0000 7ffb19362700 1 --2- 172.21.15.97:0/1903034038 >> v2:172.21.15.183:3300/0 conn(0x7ffb140af780 0x7ffb14072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.387+0000 7ffb19362700 1 --2- 172.21.15.97:0/1903034038 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb1415c140 0x7ffb140726e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.387+0000 7ffb19362700 1 -- 172.21.15.97:0/1903034038 >> 172.21.15.97:0/1903034038 conn(0x7ffb140b57b0 msgr2=0x7ffb140b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.387+0000 7ffb19362700 1 -- 172.21.15.97:0/1903034038 shutdown_connections 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.387+0000 7ffb19362700 1 -- 172.21.15.97:0/1903034038 wait complete. 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.388+0000 7ffb19362700 1 Processor -- start 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.388+0000 7ffb19362700 1 -- start start 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb19362700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb140622a0 0x7ffb140cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb19362700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb140af780 0x7ffb14149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb19362700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb1415c140 0x7ffb140b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb19362700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14149da0 con 0x7ffb1415c140 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb19362700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415e080 con 0x7ffb140af780 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb19362700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415dd80 con 0x7ffb140622a0 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb1415c140 0x7ffb140b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb1415c140 0x7ffb140b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49248/0 (socket says 172.21.15.97:49248) 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb137fe700 1 -- 172.21.15.97:0/304658895 learned_addr learned my addr 172.21.15.97:0/304658895 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb12ffd700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb140622a0 0x7ffb140cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb127fc700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb140af780 0x7ffb14149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb137fe700 1 -- 172.21.15.97:0/304658895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb140622a0 msgr2=0x7ffb140cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb137fe700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb140622a0 0x7ffb140cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.396 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb137fe700 1 -- 172.21.15.97:0/304658895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb140af780 msgr2=0x7ffb14149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb137fe700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb140af780 0x7ffb14149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.389+0000 7ffb137fe700 1 -- 172.21.15.97:0/304658895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c009a70 con 0x7ffb1415c140 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.390+0000 7ffb137fe700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb1415c140 0x7ffb140b3f30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009f00 tx=0x7ffb0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.390+0000 7ffafb7fe700 1 -- 172.21.15.97:0/304658895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c009d70 con 0x7ffb1415c140 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.390+0000 7ffafb7fe700 1 -- 172.21.15.97:0/304658895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c00f040 con 0x7ffb1415c140 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.390+0000 7ffafb7fe700 1 -- 172.21.15.97:0/304658895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c00cd70 con 0x7ffb1415c140 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.390+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb080094a0 con 0x7ffb1415c140 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.390+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb140b5010 con 0x7ffb1415c140 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.391+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1415dbe0 con 0x7ffb1415c140 2023-11-24T06:11:11.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.391+0000 7ffafb7fe700 1 -- 172.21.15.97:0/304658895 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ffb0c014480 con 0x7ffb1415c140 2023-11-24T06:11:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.392+0000 7ffafb7fe700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb0006b110 0x7ffb0006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.392+0000 7ffb12ffd700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb0006b110 0x7ffb0006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.392+0000 7ffafb7fe700 1 -- 172.21.15.97:0/304658895 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb0c08a130 con 0x7ffb1415c140 2023-11-24T06:11:11.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.392+0000 7ffb12ffd700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb0006b110 0x7ffb0006d5d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffb08000c00 tx=0x7ffb08008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:11.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.396+0000 7ffafb7fe700 1 -- 172.21.15.97:0/304658895 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb1415dbe0 con 0x7ffb1415c140 2023-11-24T06:11:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.632+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb1415e230 con 0x7ffb1415c140 2023-11-24T06:11:11.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.632+0000 7ffafb7fe700 1 -- 172.21.15.97:0/304658895 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb1415e230 con 0x7ffb1415c140 2023-11-24T06:11:11.636 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:11:11.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb0006b110 msgr2=0x7ffb0006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb0006b110 0x7ffb0006d5d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ffb08000c00 tx=0x7ffb08008030 comp rx=0 tx=0).stop 2023-11-24T06:11:11.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb1415c140 msgr2=0x7ffb140b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:11.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb1415c140 0x7ffb140b3f30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009f00 tx=0x7ffb0c00c3a0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 shutdown_connections 2023-11-24T06:11:11.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb1415c140 0x7ffb140b3f30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ffb0006b110 0x7ffb0006d5d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ffb140622a0 0x7ffb140cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 --2- 172.21.15.97:0/304658895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb140af780 0x7ffb14149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:11.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 >> 172.21.15.97:0/304658895 conn(0x7ffb140b57b0 msgr2=0x7ffb140d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:11.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 shutdown_connections 2023-11-24T06:11:11.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:11.634+0000 7ffb19362700 1 -- 172.21.15.97:0/304658895 wait complete. 2023-11-24T06:11:11.650 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:11:11.650 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:11:11.650 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-11-24T06:11:11.651 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:11:11.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:11.651 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:11:11.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:11:11.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:11:11.651 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:11.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:11.651 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:11.673 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42057sec preferred_lft 42057sec 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::5382:2bb9:1085:5950/64 scope link noprefixroute 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::b8cd:8aff:fe67:b24d/64 scope link 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether fe:4f:a7:e6:1c:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::fc4f:a7ff:fee6:1c7f/64 scope link 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::4034:29ff:fe35:b2a5/64 scope link 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:11.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:11.676 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:11:11.744 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-11-24T06:11:11.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:11.745 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-11-24T06:11:11.807 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-11-24T06:11:11.807 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:11:11.807 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:11:11.807 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:11:11.808 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:11:11.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:11.808 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:11:11.808 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:11:11.808 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:11:11.809 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:11:11.809 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:11:11.809 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:11:11.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:11.918 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:11:11.935 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:11:11.935 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:11:11.935 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:11.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:11.936 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2023-11-24T06:11:12.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:12.004 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:12.061 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:12.061 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:12.061 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:12.117 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:11:12.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:12.117 DEBUG:teuthology.orchestra.run.smithi183:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:12.203 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:12.204 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:12.227 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:12.227 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:12.227 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:11:12.227 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:11:12.228 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:11:12.228 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:11:12.228 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:11:12.228 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:11:12.296 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:11:12.297 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:11:12.297 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:11:12.298 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:11:12.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:12.299 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:12.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:12.325 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:11:12.397 INFO:teuthology.orchestra.run.smithi183.stdout:client5267 2023-11-24T06:11:12.398 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:12.398 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:12.454 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:11:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.586+0000 7ff4f3857700 1 Processor -- start 2023-11-24T06:11:12.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.586+0000 7ff4f3857700 1 -- start start 2023-11-24T06:11:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f3857700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff4ec15e0c0 0x7ff4ec0cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f3857700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4ec071d20 0x7ff4ec0b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f3857700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec0d29e0 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f3857700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec0d2c80 con 0x7ff4ec071d20 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f3857700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec0d2f60 con 0x7ff4ec15e0c0 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f1df4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4ec071d20 0x7ff4ec0b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f0df2700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff4ec0d3a40 0x7ff4ec071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41704/0 (socket says 172.21.15.97:41704) 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f0df2700 1 -- 172.21.15.97:0/3060816831 learned_addr learned my addr 172.21.15.97:0/3060816831 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f15f3700 1 --2- 172.21.15.97:0/3060816831 >> v2:172.21.15.183:3300/0 conn(0x7ff4ec15e0c0 0x7ff4ec0cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f1df4700 1 -- 172.21.15.97:0/3060816831 >> v2:172.21.15.183:3300/0 conn(0x7ff4ec15e0c0 msgr2=0x7ff4ec0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f1df4700 1 --2- 172.21.15.97:0/3060816831 >> v2:172.21.15.183:3300/0 conn(0x7ff4ec15e0c0 0x7ff4ec0cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f1df4700 1 -- 172.21.15.97:0/3060816831 >> v1:172.21.15.97:6789/0 conn(0x7ff4ec0d3a40 legacy=0x7ff4ec071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4e3fff700 1 -- 172.21.15.97:0/3060816831 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626802396 0 0) 0x7ff4ec0d29e0 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f1df4700 1 -- 172.21.15.97:0/3060816831 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0d3240 con 0x7ff4ec071d20 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.587+0000 7ff4f15f3700 1 --2- 172.21.15.97:0/3060816831 >> v2:172.21.15.183:3300/0 conn(0x7ff4ec15e0c0 0x7ff4ec0cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:12.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4f1df4700 1 --2- 172.21.15.97:0/3060816831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec071d20 0x7ff4ec0b6eb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0b6960 tx=0x7ff4e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa40569022f9e83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4e3fff700 1 -- 172.21.15.97:0/3060816831 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400e070 con 0x7ff4ec071d20 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4e3fff700 1 -- 172.21.15.97:0/3060816831 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e400bbd0 con 0x7ff4ec071d20 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4e3fff700 1 -- 172.21.15.97:0/3060816831 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4012430 con 0x7ff4ec071d20 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4f3857700 1 -- 172.21.15.97:0/3060816831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec071d20 msgr2=0x7ff4ec0b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4f3857700 1 --2- 172.21.15.97:0/3060816831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec071d20 0x7ff4ec0b6eb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0b6960 tx=0x7ff4e400b330 comp rx=0 tx=0).stop 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4f3857700 1 -- 172.21.15.97:0/3060816831 shutdown_connections 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4f3857700 1 --2- 172.21.15.97:0/3060816831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec071d20 0x7ff4ec0b6eb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4f3857700 1 --2- 172.21.15.97:0/3060816831 >> v2:172.21.15.183:3300/0 conn(0x7ff4ec15e0c0 0x7ff4ec0cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4f3857700 1 -- 172.21.15.97:0/3060816831 >> 172.21.15.97:0/3060816831 conn(0x7ff4ec06b620 msgr2=0x7ff4ec0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.588+0000 7ff4f3857700 1 -- 172.21.15.97:0/3060816831 shutdown_connections 2023-11-24T06:11:12.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.589+0000 7ff4f3857700 1 -- 172.21.15.97:0/3060816831 wait complete. 2023-11-24T06:11:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.590+0000 7ff4f3857700 1 Processor -- start 2023-11-24T06:11:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.590+0000 7ff4f3857700 1 -- start start 2023-11-24T06:11:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.590+0000 7ff4f3857700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4ec071d20 0x7ff4ec0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.590+0000 7ff4f3857700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec0d3a40 0x7ff4ec0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.590+0000 7ff4f3857700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec15e0c0 0x7ff4ec14b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.590+0000 7ff4f3857700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec14b790 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.590+0000 7ff4f3857700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0cbf70 con 0x7ff4ec15e0c0 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.590+0000 7ff4f3857700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0cc250 con 0x7ff4ec071d20 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec0d3a40 0x7ff4ec0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec0d3a40 0x7ff4ec0c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49260/0 (socket says 172.21.15.97:49260) 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 -- 172.21.15.97:0/1075716662 learned_addr learned my addr 172.21.15.97:0/1075716662 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f1df4700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec15e0c0 0x7ff4ec14b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f15f3700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4ec071d20 0x7ff4ec0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 -- 172.21.15.97:0/1075716662 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4ec071d20 msgr2=0x7ff4ec0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4ec071d20 0x7ff4ec0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 -- 172.21.15.97:0/1075716662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec15e0c0 msgr2=0x7ff4ec14b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec15e0c0 0x7ff4ec14b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 -- 172.21.15.97:0/1075716662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e4009a70 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f0df2700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec0d3a40 0x7ff4ec0c9a20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80011e0 tx=0x7ff4e800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4e1ffb700 1 -- 172.21.15.97:0/1075716662 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8004c80 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4e1ffb700 1 -- 172.21.15.97:0/1075716662 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e8004e00 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec15e620 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4e1ffb700 1 -- 172.21.15.97:0/1075716662 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8007310 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.591+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec0cc810 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.592+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec067f60 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.593+0000 7ff4e1ffb700 1 -- 172.21.15.97:0/1075716662 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff4e8007470 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.593+0000 7ff4e1ffb700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4d006b110 0x7ff4d006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.593+0000 7ff4e1ffb700 1 -- 172.21.15.97:0/1075716662 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4e8089700 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.593+0000 7ff4f15f3700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4d006b110 0x7ff4d006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:12.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.593+0000 7ff4f15f3700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4d006b110 0x7ff4d006d5d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc006fd0 tx=0x7ff4dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:12.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.597+0000 7ff4e1ffb700 1 -- 172.21.15.97:0/1075716662 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4e8054450 con 0x7ff4ec0d3a40 2023-11-24T06:11:12.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:12.832+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff4ec0c73c0 con 0x7ff4ec0d3a40 2023-11-24T06:11:13.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.506+0000 7ff4e1ffb700 1 -- 172.21.15.97:0/1075716662 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff4e8057a70 con 0x7ff4ec0d3a40 2023-11-24T06:11:13.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4d006b110 msgr2=0x7ff4d006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4d006b110 0x7ff4d006d5d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc006fd0 tx=0x7ff4dc008040 comp rx=0 tx=0).stop 2023-11-24T06:11:13.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec0d3a40 msgr2=0x7ff4ec0c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec0d3a40 0x7ff4ec0c9a20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80011e0 tx=0x7ff4e800ca30 comp rx=0 tx=0).stop 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 shutdown_connections 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4ec15e0c0 0x7ff4ec14b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4d006b110 0x7ff4d006d5d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4ec071d20 0x7ff4ec0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 --2- 172.21.15.97:0/1075716662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4ec0d3a40 0x7ff4ec0c9a20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 >> 172.21.15.97:0/1075716662 conn(0x7ff4ec06b620 msgr2=0x7ff4ec0b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 shutdown_connections 2023-11-24T06:11:13.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.509+0000 7ff4f3857700 1 -- 172.21.15.97:0/1075716662 wait complete. 2023-11-24T06:11:13.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:11:13.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.651+0000 7f2f7568d700 1 Processor -- start 2023-11-24T06:11:13.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.651+0000 7f2f7568d700 1 -- start start 2023-11-24T06:11:13.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f7568d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f7015e090 0x7f2f7006bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f7568d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f700d2250 0x7f2f700b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f7568d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f700d0ee0 con 0x7f2f7006c090 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f7568d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f700d02d0 con 0x7f2f700d2250 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f7568d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700d05b0 con 0x7f2f7015e090 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6f7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f700d2250 0x7f2f700b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6e7fc700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2f7006c090 0x7f2f700d1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41716/0 (socket says 172.21.15.97:41716) 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6e7fc700 1 -- 172.21.15.97:0/1647958856 learned_addr learned my addr 172.21.15.97:0/1647958856 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6f7fe700 1 -- 172.21.15.97:0/1647958856 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f7015e090 msgr2=0x7f2f7006bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:13.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6effd700 1 --2- 172.21.15.97:0/1647958856 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f7015e090 0x7f2f7006bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6dffb700 1 -- 172.21.15.97:0/1647958856 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027519820 0 0) 0x7f2f700d0ee0 con 0x7f2f7006c090 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6f7fe700 1 --2- 172.21.15.97:0/1647958856 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f7015e090 0x7f2f7006bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6f7fe700 1 -- 172.21.15.97:0/1647958856 >> v1:172.21.15.97:6789/0 conn(0x7f2f7006c090 legacy=0x7f2f700d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6f7fe700 1 -- 172.21.15.97:0/1647958856 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f700d0890 con 0x7f2f700d2250 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6f7fe700 1 --2- 172.21.15.97:0/1647958856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f700d2250 0x7f2f700b15c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009d90 tx=0x7f2f6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b29edf50ee4ee66 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6dffb700 1 -- 172.21.15.97:0/1647958856 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6800e070 con 0x7f2f700d2250 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6dffb700 1 -- 172.21.15.97:0/1647958856 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6800bbd0 con 0x7f2f700d2250 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.652+0000 7f2f6dffb700 1 -- 172.21.15.97:0/1647958856 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68012430 con 0x7f2f700d2250 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.653+0000 7f2f7568d700 1 -- 172.21.15.97:0/1647958856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f700d2250 msgr2=0x7f2f700b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.653+0000 7f2f7568d700 1 --2- 172.21.15.97:0/1647958856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f700d2250 0x7f2f700b15c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009d90 tx=0x7f2f6800b330 comp rx=0 tx=0).stop 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.653+0000 7f2f7568d700 1 -- 172.21.15.97:0/1647958856 shutdown_connections 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.653+0000 7f2f7568d700 1 --2- 172.21.15.97:0/1647958856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f700d2250 0x7f2f700b15c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.653+0000 7f2f7568d700 1 --2- 172.21.15.97:0/1647958856 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f7015e090 0x7f2f7006bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.653+0000 7f2f7568d700 1 -- 172.21.15.97:0/1647958856 >> 172.21.15.97:0/1647958856 conn(0x7f2f700731a0 msgr2=0x7f2f7009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:13.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.653+0000 7f2f7568d700 1 -- 172.21.15.97:0/1647958856 shutdown_connections 2023-11-24T06:11:13.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.653+0000 7f2f7568d700 1 -- 172.21.15.97:0/1647958856 wait complete. 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.654+0000 7f2f7568d700 1 Processor -- start 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.654+0000 7f2f7568d700 1 -- start start 2023-11-24T06:11:13.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f7568d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f7006c090 0x7f2f70157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f7568d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f700d2250 0x7f2f7015fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f7568d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f7015e090 0x7f2f700c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f7568d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7015f710 con 0x7f2f7015e090 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f7568d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70157790 con 0x7f2f7006c090 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f7568d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7015eb00 con 0x7f2f700d2250 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f6f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f7015e090 0x7f2f700c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f6f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f7015e090 0x7f2f700c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49272/0 (socket says 172.21.15.97:49272) 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f6f7fe700 1 -- 172.21.15.97:0/3856848739 learned_addr learned my addr 172.21.15.97:0/3856848739 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f6e7fc700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f700d2250 0x7f2f7015fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.655+0000 7f2f6effd700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f7006c090 0x7f2f70157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f6f7fe700 1 -- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f700d2250 msgr2=0x7f2f7015fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f6f7fe700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f700d2250 0x7f2f7015fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f6f7fe700 1 -- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f7006c090 msgr2=0x7f2f70157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f6f7fe700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f7006c090 0x7f2f70157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f6f7fe700 1 -- 172.21.15.97:0/3856848739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68009a70 con 0x7f2f7015e090 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f6e7fc700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f700d2250 0x7f2f7015fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f6f7fe700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f7015e090 0x7f2f700c7f00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2f68013000 tx=0x7f2f68004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f577fe700 1 -- 172.21.15.97:0/3856848739 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68012430 con 0x7f2f7015e090 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f577fe700 1 -- 172.21.15.97:0/3856848739 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f68020070 con 0x7f2f7015e090 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7015ede0 con 0x7f2f7015e090 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f577fe700 1 -- 172.21.15.97:0/3856848739 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68003ea0 con 0x7f2f7015e090 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.656+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7015f3a0 con 0x7f2f7015e090 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.657+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f700751e0 con 0x7f2f7015e090 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.657+0000 7f2f577fe700 1 -- 172.21.15.97:0/3856848739 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2f68004280 con 0x7f2f7015e090 2023-11-24T06:11:13.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.658+0000 7f2f577fe700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f5c06b110 0x7f2f5c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:13.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.658+0000 7f2f577fe700 1 -- 172.21.15.97:0/3856848739 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f68022070 con 0x7f2f7015e090 2023-11-24T06:11:13.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.658+0000 7f2f6effd700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f5c06b110 0x7f2f5c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:13.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.658+0000 7f2f6effd700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f5c06b110 0x7f2f5c06d5d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000a950 tx=0x7f2f60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.662+0000 7f2f577fe700 1 -- 172.21.15.97:0/3856848739 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f68056a60 con 0x7f2f7015e090 2023-11-24T06:11:13.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.881+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f7015e470 con 0x7f2f5c06b110 2023-11-24T06:11:13.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.882+0000 7f2f577fe700 1 -- 172.21.15.97:0/3856848739 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2f6000c050 con 0x7f2f5c06b110 2023-11-24T06:11:13.885 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:11:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephfs" 2023-11-24T06:11:13.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:13.886 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:11:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f5c06b110 msgr2=0x7f2f5c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f5c06b110 0x7f2f5c06d5d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000a950 tx=0x7f2f60008040 comp rx=0 tx=0).stop 2023-11-24T06:11:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f7015e090 msgr2=0x7f2f700c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f7015e090 0x7f2f700c7f00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2f68013000 tx=0x7f2f68004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 shutdown_connections 2023-11-24T06:11:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f7015e090 0x7f2f700c7f00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f7006c090 0x7f2f70157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f5c06b110 0x7f2f5c06d5d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 --2- 172.21.15.97:0/3856848739 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f700d2250 0x7f2f7015fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 >> 172.21.15.97:0/3856848739 conn(0x7f2f700731a0 msgr2=0x7f2f700b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 shutdown_connections 2023-11-24T06:11:13.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:13.884+0000 7f2f7568d700 1 -- 172.21.15.97:0/3856848739 wait complete. 2023-11-24T06:11:13.899 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:11:14.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.019+0000 7f0b71806700 1 Processor -- start 2023-11-24T06:11:14.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.019+0000 7f0b71806700 1 -- start start 2023-11-24T06:11:14.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b71806700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0b6c15e140 0x7f0b6c0d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b71806700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b6c0d39d0 0x7f0b6c0d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b71806700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c0b80e0 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b71806700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6c15e520 con 0x7f0b6c0d39d0 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b71806700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6c0d2930 con 0x7f0b6c15e140 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6a7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b6c0d39d0 0x7f0b6c0d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6a7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b6c0d39d0 0x7f0b6c0d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49280/0 (socket says 172.21.15.97:49280) 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6a7fc700 1 -- 172.21.15.97:0/3438477124 learned_addr learned my addr 172.21.15.97:0/3438477124 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6affd700 1 --2- 172.21.15.97:0/3438477124 >> v2:172.21.15.183:3300/0 conn(0x7f0b6c15e140 0x7f0b6c0d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6a7fc700 1 -- 172.21.15.97:0/3438477124 >> v2:172.21.15.183:3300/0 conn(0x7f0b6c15e140 msgr2=0x7f0b6c0d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6a7fc700 1 --2- 172.21.15.97:0/3438477124 >> v2:172.21.15.183:3300/0 conn(0x7f0b6c15e140 0x7f0b6c0d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6a7fc700 1 -- 172.21.15.97:0/3438477124 >> v1:172.21.15.183:6789/0 conn(0x7f0b6c0b98c0 legacy=0x7f0b6c0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6a7fc700 1 -- 172.21.15.97:0/3438477124 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c0d2c10 con 0x7f0b6c0d39d0 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.020+0000 7f0b6affd700 1 --2- 172.21.15.97:0/3438477124 >> v2:172.21.15.183:3300/0 conn(0x7f0b6c15e140 0x7f0b6c0d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b6a7fc700 1 --2- 172.21.15.97:0/3438477124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0d39d0 0x7f0b6c0d4400 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0d5030 tx=0x7f0b5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5036627104549bb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b69ffb700 1 -- 172.21.15.97:0/3438477124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c004230 con 0x7f0b6c0d39d0 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b69ffb700 1 -- 172.21.15.97:0/3438477124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c004390 con 0x7f0b6c0d39d0 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b69ffb700 1 -- 172.21.15.97:0/3438477124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0036a0 con 0x7f0b6c0d39d0 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b71806700 1 -- 172.21.15.97:0/3438477124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0d39d0 msgr2=0x7f0b6c0d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b71806700 1 --2- 172.21.15.97:0/3438477124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0d39d0 0x7f0b6c0d4400 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0d5030 tx=0x7f0b5c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b71806700 1 -- 172.21.15.97:0/3438477124 shutdown_connections 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b71806700 1 --2- 172.21.15.97:0/3438477124 >> v2:172.21.15.183:3300/0 conn(0x7f0b6c15e140 0x7f0b6c0d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b71806700 1 --2- 172.21.15.97:0/3438477124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0d39d0 0x7f0b6c0d4400 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b71806700 1 -- 172.21.15.97:0/3438477124 >> 172.21.15.97:0/3438477124 conn(0x7f0b6c06b410 msgr2=0x7f0b6c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b71806700 1 -- 172.21.15.97:0/3438477124 shutdown_connections 2023-11-24T06:11:14.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.021+0000 7f0b71806700 1 -- 172.21.15.97:0/3438477124 wait complete. 2023-11-24T06:11:14.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.022+0000 7f0b71806700 1 Processor -- start 2023-11-24T06:11:14.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.023+0000 7f0b71806700 1 -- start start 2023-11-24T06:11:14.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.023+0000 7f0b71806700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0b98c0 0x7f0b6c1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.023+0000 7f0b71806700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0b6c0d39d0 0x7f0b6c14e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.023+0000 7f0b71806700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b6c15e140 0x7f0b6c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.023+0000 7f0b71806700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c1f5640 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.023+0000 7f0b71806700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c14e8b0 con 0x7f0b6c15e140 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.023+0000 7f0b71806700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c161040 con 0x7f0b6c0d39d0 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.023+0000 7f0b6affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0b98c0 0x7f0b6c1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0b98c0 0x7f0b6c1f4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49284/0 (socket says 172.21.15.97:49284) 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6affd700 1 -- 172.21.15.97:0/204931451 learned_addr learned my addr 172.21.15.97:0/204931451 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6affd700 1 -- 172.21.15.97:0/204931451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0b6c0d39d0 msgr2=0x7f0b6c14e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6b7fe700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b6c15e140 0x7f0b6c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6affd700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0b6c0d39d0 0x7f0b6c14e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6affd700 1 -- 172.21.15.97:0/204931451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b6c15e140 msgr2=0x7f0b6c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6affd700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b6c15e140 0x7f0b6c158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6affd700 1 -- 172.21.15.97:0/204931451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c009550 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6b7fe700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b6c15e140 0x7f0b6c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b6affd700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0b98c0 0x7f0b6c1f4430 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c820 tx=0x7f0b6000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b53fff700 1 -- 172.21.15.97:0/204931451 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60011800 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b53fff700 1 -- 172.21.15.97:0/204931451 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b60011960 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c1f4d10 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b53fff700 1 -- 172.21.15.97:0/204931451 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60011d10 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.024+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c1f52d0 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.025+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6c1f5920 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.025+0000 7f0b53fff700 1 -- 172.21.15.97:0/204931451 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0b60018360 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.026+0000 7f0b53fff700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0b5806b140 0x7f0b5806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.026+0000 7f0b53fff700 1 -- 172.21.15.97:0/204931451 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b60089aa0 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.026+0000 7f0b6a7fc700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0b5806b140 0x7f0b5806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.026+0000 7f0b6a7fc700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0b5806b140 0x7f0b5806d600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c000c00 tx=0x7f0b5c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.030+0000 7f0b53fff700 1 -- 172.21.15.97:0/204931451 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b600546f0 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.222+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0b6c068fb0 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.222+0000 7f0b53fff700 1 -- 172.21.15.97:0/204931451 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0b60057d10 con 0x7f0b6c0b98c0 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0b5806b140 msgr2=0x7f0b5806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0b5806b140 0x7f0b5806d600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c000c00 tx=0x7f0b5c010000 comp rx=0 tx=0).stop 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0b98c0 msgr2=0x7f0b6c1f4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0b98c0 0x7f0b6c1f4430 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c820 tx=0x7f0b6000cbe0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 shutdown_connections 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b6c15e140 0x7f0b6c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b6c0b98c0 0x7f0b6c1f4430 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0b5806b140 0x7f0b5806d600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 --2- 172.21.15.97:0/204931451 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0b6c0d39d0 0x7f0b6c14e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 >> 172.21.15.97:0/204931451 conn(0x7f0b6c06b410 msgr2=0x7f0b6c15d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 shutdown_connections 2023-11-24T06:11:14.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.224+0000 7f0b71806700 1 -- 172.21.15.97:0/204931451 wait complete. 2023-11-24T06:11:14.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol2389 2023-11-24T06:11:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.360+0000 7f1cee87f700 1 Processor -- start 2023-11-24T06:11:14.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.360+0000 7f1cee87f700 1 -- start start 2023-11-24T06:11:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.360+0000 7f1cee87f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1ce815e160 0x7f1ce80b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.360+0000 7f1cee87f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ce80afc90 0x7f1ce8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.360+0000 7f1cee87f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80b3c60 con 0x7f1ce80744d0 2023-11-24T06:11:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.360+0000 7f1cee87f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80b4500 con 0x7f1ce80afc90 2023-11-24T06:11:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.360+0000 7f1cee87f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce8074940 con 0x7f1ce815e160 2023-11-24T06:11:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ce80afc90 0x7f1ce8073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce77fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1ce80afc90 0x7f1ce8073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49292/0 (socket says 172.21.15.97:49292) 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce77fe700 1 -- 172.21.15.97:0/1804620197 learned_addr learned my addr 172.21.15.97:0/1804620197 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce7fff700 1 --2- 172.21.15.97:0/1804620197 >> v2:172.21.15.183:3300/0 conn(0x7f1ce815e160 0x7f1ce80b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce77fe700 1 -- 172.21.15.97:0/1804620197 >> v2:172.21.15.183:3300/0 conn(0x7f1ce815e160 msgr2=0x7f1ce80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce77fe700 1 --2- 172.21.15.97:0/1804620197 >> v2:172.21.15.183:3300/0 conn(0x7f1ce815e160 0x7f1ce80b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce77fe700 1 -- 172.21.15.97:0/1804620197 >> v1:172.21.15.183:6789/0 conn(0x7f1ce80744d0 legacy=0x7f1ce806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce77fe700 1 -- 172.21.15.97:0/1804620197 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce80b58c0 con 0x7f1ce80afc90 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce7fff700 1 --2- 172.21.15.97:0/1804620197 >> v2:172.21.15.183:3300/0 conn(0x7f1ce815e160 0x7f1ce80b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce77fe700 1 --2- 172.21.15.97:0/1804620197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce80afc90 0x7f1ce8073f90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1ce809efa0 tx=0x7f1cdc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3145e22200eb79ab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce6ffd700 1 -- 172.21.15.97:0/1804620197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc004230 con 0x7f1ce80afc90 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce6ffd700 1 -- 172.21.15.97:0/1804620197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cdc004390 con 0x7f1ce80afc90 2023-11-24T06:11:14.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.361+0000 7f1ce6ffd700 1 -- 172.21.15.97:0/1804620197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0036a0 con 0x7f1ce80afc90 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.362+0000 7f1cee87f700 1 -- 172.21.15.97:0/1804620197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce80afc90 msgr2=0x7f1ce8073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.362+0000 7f1cee87f700 1 --2- 172.21.15.97:0/1804620197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce80afc90 0x7f1ce8073f90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1ce809efa0 tx=0x7f1cdc009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.362+0000 7f1cee87f700 1 -- 172.21.15.97:0/1804620197 shutdown_connections 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.362+0000 7f1cee87f700 1 --2- 172.21.15.97:0/1804620197 >> v2:172.21.15.183:3300/0 conn(0x7f1ce815e160 0x7f1ce80b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.362+0000 7f1cee87f700 1 --2- 172.21.15.97:0/1804620197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce80afc90 0x7f1ce8073f90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.362+0000 7f1cee87f700 1 -- 172.21.15.97:0/1804620197 >> 172.21.15.97:0/1804620197 conn(0x7f1ce80d0bf0 msgr2=0x7f1ce80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.362+0000 7f1cee87f700 1 -- 172.21.15.97:0/1804620197 shutdown_connections 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.362+0000 7f1cee87f700 1 -- 172.21.15.97:0/1804620197 wait complete. 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.363+0000 7f1cee87f700 1 Processor -- start 2023-11-24T06:11:14.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.363+0000 7f1cee87f700 1 -- start start 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cee87f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ce80744d0 0x7f1ce81f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cee87f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce80afc90 0x7f1ce8160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cee87f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce815e160 0x7f1ce80cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cee87f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce815ef80 con 0x7f1ce815e160 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cee87f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80c6ba0 con 0x7f1ce80afc90 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cee87f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce815e5e0 con 0x7f1ce80744d0 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cece1c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce815e160 0x7f1ce80cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cece1c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce815e160 0x7f1ce80cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49302/0 (socket says 172.21.15.97:49302) 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cece1c700 1 -- 172.21.15.97:0/176503392 learned_addr learned my addr 172.21.15.97:0/176503392 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1ce77fe700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce80afc90 0x7f1ce8160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1ce7fff700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ce80744d0 0x7f1ce81f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cece1c700 1 -- 172.21.15.97:0/176503392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ce80744d0 msgr2=0x7f1ce81f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cece1c700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ce80744d0 0x7f1ce81f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cece1c700 1 -- 172.21.15.97:0/176503392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce80afc90 msgr2=0x7f1ce8160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cece1c700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce80afc90 0x7f1ce8160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.364+0000 7f1cece1c700 1 -- 172.21.15.97:0/176503392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc009550 con 0x7f1ce815e160 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.365+0000 7f1cece1c700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce815e160 0x7f1ce80cfbd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000cc10 tx=0x7f1ce000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.365+0000 7f1ce4ff9700 1 -- 172.21.15.97:0/176503392 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000aa60 con 0x7f1ce815e160 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.365+0000 7f1ce4ff9700 1 -- 172.21.15.97:0/176503392 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce000abc0 con 0x7f1ce815e160 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.365+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce815e8c0 con 0x7f1ce815e160 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.365+0000 7f1ce4ff9700 1 -- 172.21.15.97:0/176503392 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0012420 con 0x7f1ce815e160 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.365+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce8161120 con 0x7f1ce815e160 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.366+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce8161310 con 0x7f1ce815e160 2023-11-24T06:11:14.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.366+0000 7f1ce4ff9700 1 -- 172.21.15.97:0/176503392 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f1ce0012580 con 0x7f1ce815e160 2023-11-24T06:11:14.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.367+0000 7f1ce4ff9700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cd406b110 0x7f1cd406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.367+0000 7f1ce4ff9700 1 -- 172.21.15.97:0/176503392 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ce0089c00 con 0x7f1ce815e160 2023-11-24T06:11:14.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.367+0000 7f1ce7fff700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cd406b110 0x7f1cd406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.367+0000 7f1ce7fff700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cd406b110 0x7f1cd406d5d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8006fd0 tx=0x7f1cd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.371+0000 7f1ce4ff9700 1 -- 172.21.15.97:0/176503392 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ce0090080 con 0x7f1ce815e160 2023-11-24T06:11:14.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.595+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol2389", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ce80b3eb0 con 0x7f1cd406b110 2023-11-24T06:11:14.603 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:14.600+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:14.603 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:14.600+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:14.604 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:14.600+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:14.604 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:14.600+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:14.604 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:14.600+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:14.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.631+0000 7f1ce4ff9700 1 -- 172.21.15.97:0/176503392 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1cd800c040 con 0x7f1cd406b110 2023-11-24T06:11:14.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cd406b110 msgr2=0x7f1cd406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cd406b110 0x7f1cd406d5d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8006fd0 tx=0x7f1cd8008040 comp rx=0 tx=0).stop 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce815e160 msgr2=0x7f1ce80cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce815e160 0x7f1ce80cfbd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000cc10 tx=0x7f1ce000cf20 comp rx=0 tx=0).stop 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 shutdown_connections 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ce815e160 0x7f1ce80cfbd0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1cd406b110 0x7f1cd406d5d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1ce80744d0 0x7f1ce81f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 --2- 172.21.15.97:0/176503392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ce80afc90 0x7f1ce8160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.633+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 >> 172.21.15.97:0/176503392 conn(0x7f1ce80d0bf0 msgr2=0x7f1ce806e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.634+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 shutdown_connections 2023-11-24T06:11:14.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.634+0000 7f1cee87f700 1 -- 172.21.15.97:0/176503392 wait complete. 2023-11-24T06:11:14.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol2389 subvol_snap4387 2023-11-24T06:11:14.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.770+0000 7ff4276ba700 1 Processor -- start 2023-11-24T06:11:14.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.771+0000 7ff4276ba700 1 -- start start 2023-11-24T06:11:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff4276ba700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff42006e830 0x7ff42006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff4276ba700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42006f1c0 con 0x7ff42006f480 2023-11-24T06:11:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff4276ba700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42006c9b0 con 0x7ff420144600 2023-11-24T06:11:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff4276ba700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42006aff0 con 0x7ff42006e830 2023-11-24T06:11:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff425c57700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff42006e830 0x7ff42006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff424c55700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff42006f480 0x7ff42006c2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41776/0 (socket says 172.21.15.97:41776) 2023-11-24T06:11:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff424c55700 1 -- 172.21.15.97:0/1258507419 learned_addr learned my addr 172.21.15.97:0/1258507419 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff425c57700 1 -- 172.21.15.97:0/1258507419 >> v1:172.21.15.183:6789/0 conn(0x7ff420144600 legacy=0x7ff4200b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff425c57700 1 -- 172.21.15.97:0/1258507419 >> v1:172.21.15.97:6789/0 conn(0x7ff42006f480 legacy=0x7ff42006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff425c57700 1 -- 172.21.15.97:0/1258507419 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42006b200 con 0x7ff42006e830 2023-11-24T06:11:14.785 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff417fff700 1 -- 172.21.15.97:0/1258507419 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923499583 0 0) 0x7ff41c001110 con 0x7ff42006f480 2023-11-24T06:11:14.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff425c57700 1 --2- 172.21.15.97:0/1258507419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff42006ec80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff418009d90 tx=0x7ff41800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1b2373b8fcf8fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff417fff700 1 -- 172.21.15.97:0/1258507419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41800e070 con 0x7ff42006e830 2023-11-24T06:11:14.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff417fff700 1 -- 172.21.15.97:0/1258507419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41800bb90 con 0x7ff42006e830 2023-11-24T06:11:14.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.772+0000 7ff417fff700 1 -- 172.21.15.97:0/1258507419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418003a20 con 0x7ff42006e830 2023-11-24T06:11:14.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.773+0000 7ff4276ba700 1 -- 172.21.15.97:0/1258507419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 msgr2=0x7ff42006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.773+0000 7ff4276ba700 1 --2- 172.21.15.97:0/1258507419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff42006ec80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff418009d90 tx=0x7ff41800b330 comp rx=0 tx=0).stop 2023-11-24T06:11:14.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.773+0000 7ff4276ba700 1 -- 172.21.15.97:0/1258507419 shutdown_connections 2023-11-24T06:11:14.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.773+0000 7ff4276ba700 1 --2- 172.21.15.97:0/1258507419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff42006ec80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.773+0000 7ff4276ba700 1 -- 172.21.15.97:0/1258507419 >> 172.21.15.97:0/1258507419 conn(0x7ff4200d0b40 msgr2=0x7ff4200d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:14.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.773+0000 7ff4276ba700 1 -- 172.21.15.97:0/1258507419 shutdown_connections 2023-11-24T06:11:14.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.773+0000 7ff4276ba700 1 -- 172.21.15.97:0/1258507419 wait complete. 2023-11-24T06:11:14.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.774+0000 7ff4276ba700 1 Processor -- start 2023-11-24T06:11:14.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.774+0000 7ff4276ba700 1 -- start start 2023-11-24T06:11:14.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff4276ba700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff4201f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff4276ba700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42006f480 0x7ff420159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff4276ba700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff420144600 0x7ff420073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff4276ba700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4201f4eb0 con 0x7ff42006e830 2023-11-24T06:11:14.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff4276ba700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42015b430 con 0x7ff42006f480 2023-11-24T06:11:14.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff4276ba700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42015b710 con 0x7ff420144600 2023-11-24T06:11:14.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff4201f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff4201f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49328/0 (socket says 172.21.15.97:49328) 2023-11-24T06:11:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 -- 172.21.15.97:0/2137745882 learned_addr learned my addr 172.21.15.97:0/2137745882 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff424c55700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42006f480 0x7ff420159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 -- 172.21.15.97:0/2137745882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff420144600 msgr2=0x7ff420073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff420144600 0x7ff420073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.790 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 -- 172.21.15.97:0/2137745882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42006f480 msgr2=0x7ff420159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42006f480 0x7ff420159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 -- 172.21.15.97:0/2137745882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff418009a70 con 0x7ff42006e830 2023-11-24T06:11:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff424c55700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42006f480 0x7ff420159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:14.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.775+0000 7ff425456700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff4201f4970 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff41000b4a0 tx=0x7ff41000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.776+0000 7ff415ffb700 1 -- 172.21.15.97:0/2137745882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410011830 con 0x7ff42006e830 2023-11-24T06:11:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.776+0000 7ff415ffb700 1 -- 172.21.15.97:0/2137745882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff410011990 con 0x7ff42006e830 2023-11-24T06:11:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.776+0000 7ff415ffb700 1 -- 172.21.15.97:0/2137745882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff410011d40 con 0x7ff42006e830 2023-11-24T06:11:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.776+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42015b9f0 con 0x7ff42006e830 2023-11-24T06:11:14.792 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.776+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42015bfb0 con 0x7ff42006e830 2023-11-24T06:11:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.777+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff420068a00 con 0x7ff42006e830 2023-11-24T06:11:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.777+0000 7ff415ffb700 1 -- 172.21.15.97:0/2137745882 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff41001a400 con 0x7ff42006e830 2023-11-24T06:11:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.777+0000 7ff415ffb700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff40406b110 0x7ff40406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.778+0000 7ff415ffb700 1 -- 172.21.15.97:0/2137745882 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4100896e0 con 0x7ff42006e830 2023-11-24T06:11:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.778+0000 7ff424c55700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff40406b110 0x7ff40406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:14.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.778+0000 7ff424c55700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff40406b110 0x7ff40406d5d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff41c007fd0 tx=0x7ff41c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:14.782+0000 7ff415ffb700 1 -- 172.21.15.97:0/2137745882 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4100544d0 con 0x7ff42006e830 2023-11-24T06:11:15.009 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.006+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol2389", "snap_name": "subvol_snap4387", "target": ["mon-mgr", ""]}) v1 -- 0x7ff42006baa0 con 0x7ff40406b110 2023-11-24T06:11:15.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.021+0000 7ff415ffb700 1 -- 172.21.15.97:0/2137745882 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff41c0014c0 con 0x7ff40406b110 2023-11-24T06:11:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff40406b110 msgr2=0x7ff40406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff40406b110 0x7ff40406d5d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff41c007fd0 tx=0x7ff41c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 msgr2=0x7ff4201f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff4201f4970 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff41000b4a0 tx=0x7ff41000b860 comp rx=0 tx=0).stop 2023-11-24T06:11:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 shutdown_connections 2023-11-24T06:11:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff40406b110 0x7ff40406d5d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff420144600 0x7ff420073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff42006e830 0x7ff4201f4970 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 --2- 172.21.15.97:0/2137745882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff42006f480 0x7ff420159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 >> 172.21.15.97:0/2137745882 conn(0x7ff4200d0b40 msgr2=0x7ff42006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 shutdown_connections 2023-11-24T06:11:15.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.024+0000 7ff4276ba700 1 -- 172.21.15.97:0/2137745882 wait complete. 2023-11-24T06:11:15.040 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol2389 subvol_snap4387 2023-11-24T06:11:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.161+0000 7facc5f0d700 1 Processor -- start 2023-11-24T06:11:15.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.161+0000 7facc5f0d700 1 -- start start 2023-11-24T06:11:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facc5f0d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facc00b18f0 0x7facc00cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facc5f0d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7facc00cf910 0x7facc00af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facc5f0d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc0161c80 con 0x7facc015e020 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facc5f0d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc0161f40 con 0x7facc00b18f0 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facc5f0d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc0162220 con 0x7facc00cf910 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facbeffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facc00b18f0 0x7facc00cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facbeffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facc00b18f0 0x7facc00cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49338/0 (socket says 172.21.15.97:49338) 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facbeffd700 1 -- 172.21.15.97:0/47041061 learned_addr learned my addr 172.21.15.97:0/47041061 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facbeffd700 1 -- 172.21.15.97:0/47041061 >> v2:172.21.15.183:3300/0 conn(0x7facc00cf910 msgr2=0x7facc00af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facbeffd700 1 --2- 172.21.15.97:0/47041061 >> v2:172.21.15.183:3300/0 conn(0x7facc00cf910 0x7facc00af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facbeffd700 1 -- 172.21.15.97:0/47041061 >> v1:172.21.15.183:6789/0 conn(0x7facc015e020 legacy=0x7facc00b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facbeffd700 1 -- 172.21.15.97:0/47041061 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0162500 con 0x7facc00b18f0 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.162+0000 7facbeffd700 1 --2- 172.21.15.97:0/47041061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc00b18f0 0x7facc00cf3d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7facc00cdd00 tx=0x7facb0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f9e0e2f850ebe0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facbe7fc700 1 -- 172.21.15.97:0/47041061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb0004230 con 0x7facc00b18f0 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facbe7fc700 1 -- 172.21.15.97:0/47041061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb0004390 con 0x7facc00b18f0 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facbe7fc700 1 -- 172.21.15.97:0/47041061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb00036a0 con 0x7facc00b18f0 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facc5f0d700 1 -- 172.21.15.97:0/47041061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc00b18f0 msgr2=0x7facc00cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facc5f0d700 1 --2- 172.21.15.97:0/47041061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc00b18f0 0x7facc00cf3d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7facc00cdd00 tx=0x7facb0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facc5f0d700 1 -- 172.21.15.97:0/47041061 shutdown_connections 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facc5f0d700 1 --2- 172.21.15.97:0/47041061 >> v2:172.21.15.183:3300/0 conn(0x7facc00cf910 0x7facc00af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facc5f0d700 1 --2- 172.21.15.97:0/47041061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc00b18f0 0x7facc00cf3d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facc5f0d700 1 -- 172.21.15.97:0/47041061 >> 172.21.15.97:0/47041061 conn(0x7facc0072710 msgr2=0x7facc0072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facc5f0d700 1 -- 172.21.15.97:0/47041061 shutdown_connections 2023-11-24T06:11:15.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.163+0000 7facc5f0d700 1 -- 172.21.15.97:0/47041061 wait complete. 2023-11-24T06:11:15.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.164+0000 7facc5f0d700 1 Processor -- start 2023-11-24T06:11:15.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.165+0000 7facc5f0d700 1 -- start start 2023-11-24T06:11:15.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.165+0000 7facc5f0d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc00b18f0 0x7facc015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.165+0000 7facc5f0d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7facc00cf910 0x7facc00c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.165+0000 7facc5f0d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc015e020 0x7facc00c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.165+0000 7facc5f0d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00aeeb0 con 0x7facc015e020 2023-11-24T06:11:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.165+0000 7facc5f0d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00af190 con 0x7facc00b18f0 2023-11-24T06:11:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.165+0000 7facc5f0d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc00af470 con 0x7facc00cf910 2023-11-24T06:11:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc015e020 0x7facc00c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc015e020 0x7facc00c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49344/0 (socket says 172.21.15.97:49344) 2023-11-24T06:11:15.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 -- 172.21.15.97:0/3311614288 learned_addr learned my addr 172.21.15.97:0/3311614288 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbeffd700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7facc00cf910 0x7facc00c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbf7fe700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc00b18f0 0x7facc015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 -- 172.21.15.97:0/3311614288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7facc00cf910 msgr2=0x7facc00c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7facc00cf910 0x7facc00c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 -- 172.21.15.97:0/3311614288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc00b18f0 msgr2=0x7facc015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc00b18f0 0x7facc015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 -- 172.21.15.97:0/3311614288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb0009550 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facbffff700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc015e020 0x7facc00c4280 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7facb800b970 tx=0x7facb800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facb7fff700 1 -- 172.21.15.97:0/3311614288 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800c9b0 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facb7fff700 1 -- 172.21.15.97:0/3311614288 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb800cb30 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc00af750 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facb7fff700 1 -- 172.21.15.97:0/3311614288 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8014440 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.166+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc015ca70 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.167+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc0067f60 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.168+0000 7facb7fff700 1 -- 172.21.15.97:0/3311614288 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7facb801d3b0 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.168+0000 7facb7fff700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faca006b110 0x7faca006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.168+0000 7facb7fff700 1 -- 172.21.15.97:0/3311614288 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facb8006e40 con 0x7facc015e020 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.168+0000 7facbf7fe700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faca006b110 0x7faca006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.168+0000 7facbf7fe700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faca006b110 0x7faca006d5d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faca8009870 tx=0x7faca8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.172+0000 7facb7fff700 1 -- 172.21.15.97:0/3311614288 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7facb80544d0 con 0x7facc015e020 2023-11-24T06:11:15.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.405+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol2389", "snap_name": "subvol_snap4387", "target": ["mon-mgr", ""]}) v1 -- 0x7facc00cfd60 con 0x7faca006b110 2023-11-24T06:11:15.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.425+0000 7facb7fff700 1 -- 172.21.15.97:0/3311614288 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faca80090c0 con 0x7faca006b110 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faca006b110 msgr2=0x7faca006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faca006b110 0x7faca006d5d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faca8009870 tx=0x7faca8008040 comp rx=0 tx=0).stop 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc015e020 msgr2=0x7facc00c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc015e020 0x7facc00c4280 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7facb800b970 tx=0x7facb800bd30 comp rx=0 tx=0).stop 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 shutdown_connections 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facc015e020 0x7facc00c4280 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facc00b18f0 0x7facc015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faca006b110 0x7faca006d5d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 --2- 172.21.15.97:0/3311614288 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7facc00cf910 0x7facc00c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 >> 172.21.15.97:0/3311614288 conn(0x7facc0072710 msgr2=0x7facc00ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 shutdown_connections 2023-11-24T06:11:15.433 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.428+0000 7facc5f0d700 1 -- 172.21.15.97:0/3311614288 wait complete. 2023-11-24T06:11:15.443 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol2389 subvol_snap4387 2023-11-24T06:11:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.567+0000 7f33a4d3e700 1 Processor -- start 2023-11-24T06:11:15.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.567+0000 7f33a4d3e700 1 -- start start 2023-11-24T06:11:15.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f33a4d3e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33a00cf040 0x7f33a006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f33a4d3e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f33a00702e0 0x7f33a00d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f33a4d3e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f33a00d1ff0 0x7f33a015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f33a4d3e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a00d01c0 con 0x7f33a00702e0 2023-11-24T06:11:15.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f33a4d3e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a0070910 con 0x7f33a00d1ff0 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f33a4d3e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0070bf0 con 0x7f33a00cf040 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f339dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f33a00702e0 0x7f33a00d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f339dd9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f33a00702e0 0x7f33a00d1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49352/0 (socket says 172.21.15.97:49352) 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f339dd9b700 1 -- 172.21.15.97:0/687945549 learned_addr learned my addr 172.21.15.97:0/687945549 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f339dd9b700 1 -- 172.21.15.97:0/687945549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33a00cf040 msgr2=0x7f33a006fda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f339dd9b700 1 --2- 172.21.15.97:0/687945549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33a00cf040 0x7f33a006fda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f339dd9b700 1 -- 172.21.15.97:0/687945549 >> v2:172.21.15.183:3300/0 conn(0x7f33a00d1ff0 msgr2=0x7f33a015f9e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f339dd9b700 1 --2- 172.21.15.97:0/687945549 >> v2:172.21.15.183:3300/0 conn(0x7f33a00d1ff0 0x7f33a015f9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.568+0000 7f339dd9b700 1 -- 172.21.15.97:0/687945549 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a00aebb0 con 0x7f33a00702e0 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f339dd9b700 1 --2- 172.21.15.97:0/687945549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a00702e0 0x7f33a00d1ab0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f33a0160a80 tx=0x7f3390009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8156e958ddff6e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f339d59a700 1 -- 172.21.15.97:0/687945549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339000b870 con 0x7f33a00702e0 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f339d59a700 1 -- 172.21.15.97:0/687945549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339000b9d0 con 0x7f33a00702e0 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f339d59a700 1 -- 172.21.15.97:0/687945549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339000bd60 con 0x7f33a00702e0 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 -- 172.21.15.97:0/687945549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a00702e0 msgr2=0x7f33a00d1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 --2- 172.21.15.97:0/687945549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a00702e0 0x7f33a00d1ab0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f33a0160a80 tx=0x7f3390009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 -- 172.21.15.97:0/687945549 shutdown_connections 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 --2- 172.21.15.97:0/687945549 >> v2:172.21.15.183:3300/0 conn(0x7f33a00d1ff0 0x7f33a015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 --2- 172.21.15.97:0/687945549 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33a00cf040 0x7f33a006fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 --2- 172.21.15.97:0/687945549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a00702e0 0x7f33a00d1ab0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 -- 172.21.15.97:0/687945549 >> 172.21.15.97:0/687945549 conn(0x7f33a009aa00 msgr2=0x7f33a009ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 -- 172.21.15.97:0/687945549 shutdown_connections 2023-11-24T06:11:15.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.569+0000 7f33a4d3e700 1 -- 172.21.15.97:0/687945549 wait complete. 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.571+0000 7f33a4d3e700 1 Processor -- start 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.571+0000 7f33a4d3e700 1 -- start start 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.571+0000 7f33a4d3e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a00702e0 0x7f33a00c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.571+0000 7f33a4d3e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a00cf040 0x7f33a00c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.571+0000 7f33a4d3e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33a00d1ff0 0x7f33a00c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.571+0000 7f33a4d3e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00ca550 con 0x7f33a00702e0 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.571+0000 7f33a4d3e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00ca830 con 0x7f33a00cf040 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.571+0000 7f33a4d3e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00cab10 con 0x7f33a00d1ff0 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a00702e0 0x7f33a00c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.578 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a00702e0 0x7f33a00c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49356/0 (socket says 172.21.15.97:49356) 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 -- 172.21.15.97:0/650807204 learned_addr learned my addr 172.21.15.97:0/650807204 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339dd9b700 1 --2- 172.21.15.97:0/650807204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a00cf040 0x7f33a00c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 -- 172.21.15.97:0/650807204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33a00d1ff0 msgr2=0x7f33a00c7930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 --2- 172.21.15.97:0/650807204 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f33a00d1ff0 0x7f33a00c7930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 -- 172.21.15.97:0/650807204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a00cf040 msgr2=0x7f33a00c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 --2- 172.21.15.97:0/650807204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33a00cf040 0x7f33a00c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 -- 172.21.15.97:0/650807204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009550 con 0x7f33a00702e0 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f339e59c700 1 --2- 172.21.15.97:0/650807204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33a00702e0 0x7f33a00c9d20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f339400b550 tx=0x7f339400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f338effd700 1 -- 172.21.15.97:0/650807204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394011830 con 0x7f33a00702e0 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f338effd700 1 -- 172.21.15.97:0/650807204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33940119b0 con 0x7f33a00702e0 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f338effd700 1 -- 172.21.15.97:0/650807204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394011d60 con 0x7f33a00702e0 2023-11-24T06:11:15.579 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f33a4d3e700 1 -- 172.21.15.97:0/650807204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a00cadf0 con 0x7f33a00702e0 2023-11-24T06:11:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.572+0000 7f33a4d3e700 1 -- 172.21.15.97:0/650807204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a00ca260 con 0x7f33a00702e0 2023-11-24T06:11:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.573+0000 7f33a4d3e700 1 -- 172.21.15.97:0/650807204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a0067f60 con 0x7f33a00702e0 2023-11-24T06:11:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.573+0000 7f338effd700 1 -- 172.21.15.97:0/650807204 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f339401a400 con 0x7f33a00702e0 2023-11-24T06:11:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.574+0000 7f338effd700 1 --2- 172.21.15.97:0/650807204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f338006b1a0 0x7f338006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.574+0000 7f338effd700 1 -- 172.21.15.97:0/650807204 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3394089b00 con 0x7f33a00702e0 2023-11-24T06:11:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.574+0000 7f339dd9b700 1 --2- 172.21.15.97:0/650807204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f338006b1a0 0x7f338006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.580 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.574+0000 7f339dd9b700 1 --2- 172.21.15.97:0/650807204 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f338006b1a0 0x7f338006d660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3390000c00 tx=0x7f3390008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.578+0000 7f338effd700 1 -- 172.21.15.97:0/650807204 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3394054860 con 0x7f33a00702e0 2023-11-24T06:11:15.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.808+0000 7f33a4d3e700 1 -- 172.21.15.97:0/650807204 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol2389", "snap_name": "subvol_snap4387", "target": ["mon-mgr", ""]}) v1 -- 0x7f33a00b06c0 con 0x7f338006b1a0 2023-11-24T06:11:15.824 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:15.820+0000 7fc9c0830700 -1 mgr.server reply reply (2) No such file or directory snapshot 'b'subvol_snap4387'' does not exist 2023-11-24T06:11:15.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.820+0000 7f338effd700 1 -- 172.21.15.97:0/650807204 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: -2 snapshot 'b'subvol_snap4387'' does not exist) v1 ==== 52+0+0 (secure 0 0 0) 0x7f33a00b06c0 con 0x7f338006b1a0 2023-11-24T06:11:15.825 INFO:teuthology.orchestra.run.smithi097.stderr:Error ENOENT: snapshot 'b'subvol_snap4387'' does not exist 2023-11-24T06:11:15.828 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-24T06:11:15.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol2389 2023-11-24T06:11:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.948+0000 7f442f970700 1 Processor -- start 2023-11-24T06:11:15.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.948+0000 7f442f970700 1 -- start start 2023-11-24T06:11:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.949+0000 7f442f970700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f442815bec0 0x7f442806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.949+0000 7f442f970700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f44280702e0 0x7f44280d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.949+0000 7f442f970700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44280cc4a0 con 0x7f44280d2000 2023-11-24T06:11:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.949+0000 7f442f970700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4428070770 con 0x7f442815bec0 2023-11-24T06:11:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.949+0000 7f442f970700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44280d03e0 con 0x7f44280702e0 2023-11-24T06:11:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.949+0000 7f442d70c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f442815bec0 0x7f442806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.949+0000 7f442df0d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f44280d2000 0x7f442815f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41820/0 (socket says 172.21.15.97:41820) 2023-11-24T06:11:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.949+0000 7f442df0d700 1 -- 172.21.15.97:0/3569860975 learned_addr learned my addr 172.21.15.97:0/3569860975 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442cf0b700 1 --2- 172.21.15.97:0/3569860975 >> v2:172.21.15.183:3300/0 conn(0x7f44280702e0 0x7f44280d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442d70c700 1 -- 172.21.15.97:0/3569860975 >> v2:172.21.15.183:3300/0 conn(0x7f44280702e0 msgr2=0x7f44280d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442d70c700 1 --2- 172.21.15.97:0/3569860975 >> v2:172.21.15.183:3300/0 conn(0x7f44280702e0 0x7f44280d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442d70c700 1 -- 172.21.15.97:0/3569860975 >> v1:172.21.15.97:6789/0 conn(0x7f44280d2000 legacy=0x7f442815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f441ffff700 1 -- 172.21.15.97:0/3569860975 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249607652 0 0) 0x7f44280cc4a0 con 0x7f44280d2000 2023-11-24T06:11:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442d70c700 1 -- 172.21.15.97:0/3569860975 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44280d2430 con 0x7f442815bec0 2023-11-24T06:11:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442cf0b700 1 --2- 172.21.15.97:0/3569860975 >> v2:172.21.15.183:3300/0 conn(0x7f44280702e0 0x7f44280d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442d70c700 1 --2- 172.21.15.97:0/3569860975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442815bec0 0x7f442806fda0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4428073030 tx=0x7f4418009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f920b21b3e9b6021 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f441ffff700 1 -- 172.21.15.97:0/3569860975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418011830 con 0x7f442815bec0 2023-11-24T06:11:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f441ffff700 1 -- 172.21.15.97:0/3569860975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4418011990 con 0x7f442815bec0 2023-11-24T06:11:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f441ffff700 1 -- 172.21.15.97:0/3569860975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418011d20 con 0x7f442815bec0 2023-11-24T06:11:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442f970700 1 -- 172.21.15.97:0/3569860975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442815bec0 msgr2=0x7f442806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.950+0000 7f442f970700 1 --2- 172.21.15.97:0/3569860975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442815bec0 0x7f442806fda0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4428073030 tx=0x7f4418009b20 comp rx=0 tx=0).stop 2023-11-24T06:11:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.951+0000 7f442f970700 1 -- 172.21.15.97:0/3569860975 shutdown_connections 2023-11-24T06:11:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.951+0000 7f442f970700 1 --2- 172.21.15.97:0/3569860975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f442815bec0 0x7f442806fda0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.951+0000 7f442f970700 1 --2- 172.21.15.97:0/3569860975 >> v2:172.21.15.183:3300/0 conn(0x7f44280702e0 0x7f44280d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.951+0000 7f442f970700 1 -- 172.21.15.97:0/3569860975 >> 172.21.15.97:0/3569860975 conn(0x7f44280730a0 msgr2=0x7f442809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.951+0000 7f442f970700 1 -- 172.21.15.97:0/3569860975 shutdown_connections 2023-11-24T06:11:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.951+0000 7f442f970700 1 -- 172.21.15.97:0/3569860975 wait complete. 2023-11-24T06:11:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.952+0000 7f442f970700 1 Processor -- start 2023-11-24T06:11:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.952+0000 7f442f970700 1 -- start start 2023-11-24T06:11:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.952+0000 7f442f970700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44280702e0 0x7f442815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.952+0000 7f442f970700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44280d2000 0x7f44280c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.952+0000 7f442f970700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442815bec0 0x7f442815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442f970700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442806c3b0 con 0x7f44280d2000 2023-11-24T06:11:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442f970700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442815d220 con 0x7f44280702e0 2023-11-24T06:11:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442f970700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442815c550 con 0x7f442815bec0 2023-11-24T06:11:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44280d2000 0x7f44280c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44280d2000 0x7f44280c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49374/0 (socket says 172.21.15.97:49374) 2023-11-24T06:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 -- 172.21.15.97:0/1873186915 learned_addr learned my addr 172.21.15.97:0/1873186915 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442d70c700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44280702e0 0x7f442815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442df0d700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442815bec0 0x7f442815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 -- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442815bec0 msgr2=0x7f442815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442815bec0 0x7f442815d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 -- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44280702e0 msgr2=0x7f442815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:15.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44280702e0 0x7f442815cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 -- 172.21.15.97:0/1873186915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180094a0 con 0x7f44280d2000 2023-11-24T06:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442df0d700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442815bec0 0x7f442815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442cf0b700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44280d2000 0x7f44280c7ec0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f442400c820 tx=0x7f442400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f441dffb700 1 -- 172.21.15.97:0/1873186915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424011800 con 0x7f44280d2000 2023-11-24T06:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f441dffb700 1 -- 172.21.15.97:0/1873186915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4424011960 con 0x7f44280d2000 2023-11-24T06:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44280c8400 con 0x7f44280d2000 2023-11-24T06:11:15.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f441dffb700 1 -- 172.21.15.97:0/1873186915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424011d10 con 0x7f44280d2000 2023-11-24T06:11:15.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.953+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442815c7c0 con 0x7f44280d2000 2023-11-24T06:11:15.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.954+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4428068fb0 con 0x7f44280d2000 2023-11-24T06:11:15.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.954+0000 7f441dffb700 1 -- 172.21.15.97:0/1873186915 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f4424018360 con 0x7f44280d2000 2023-11-24T06:11:15.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.955+0000 7f441dffb700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440c06b1a0 0x7f440c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:15.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.955+0000 7f441dffb700 1 -- 172.21.15.97:0/1873186915 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4424089ae0 con 0x7f44280d2000 2023-11-24T06:11:15.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.955+0000 7f442d70c700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440c06b1a0 0x7f440c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:15.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.955+0000 7f442d70c700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440c06b1a0 0x7f440c06d660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f44180090b0 tx=0x7f4418008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:15.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:15.959+0000 7f441dffb700 1 -- 172.21.15.97:0/1873186915 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4424054740 con 0x7f44280d2000 2023-11-24T06:11:16.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.182+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol2389", "target": ["mon-mgr", ""]}) v1 -- 0x7f4428075350 con 0x7f440c06b1a0 2023-11-24T06:11:16.199 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.196+0000 7f441dffb700 1 -- 172.21.15.97:0/1873186915 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4428075350 con 0x7f440c06b1a0 2023-11-24T06:11:16.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440c06b1a0 msgr2=0x7f440c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440c06b1a0 0x7f440c06d660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f44180090b0 tx=0x7f4418008030 comp rx=0 tx=0).stop 2023-11-24T06:11:16.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44280d2000 msgr2=0x7f44280c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.203 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44280d2000 0x7f44280c7ec0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f442400c820 tx=0x7f442400cbe0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 shutdown_connections 2023-11-24T06:11:16.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f440c06b1a0 0x7f440c06d660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f442815bec0 0x7f442815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44280702e0 0x7f442815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 --2- 172.21.15.97:0/1873186915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44280d2000 0x7f44280c7ec0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 >> 172.21.15.97:0/1873186915 conn(0x7f44280730a0 msgr2=0x7f44280b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.199+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 shutdown_connections 2023-11-24T06:11:16.204 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.200+0000 7f442f970700 1 -- 172.21.15.97:0/1873186915 wait complete. 2023-11-24T06:11:16.205 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.201+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.205 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.201+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.205 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.201+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.205 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.201+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.205 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.201+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:16.217 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T06:11:16.219 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.215+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.220 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.215+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.220 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.215+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.220 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.215+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.220 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:16.215+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:16.254 INFO:teuthology.orchestra.run.smithi097.stdout:2 2023-11-24T06:11:16.255 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:11:16.256 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:11:16.343 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:11:16.390 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:11:16.391 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:16.391 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:16.408 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:16.408 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:16.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:16.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.623+0000 7f174a800700 1 Processor -- start 2023-11-24T06:11:16.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f174a800700 1 -- start start 2023-11-24T06:11:16.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f174a800700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17440746a0 0x7f17440ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f174a800700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f17440ce9a0 0x7f174406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f174a800700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174415fb30 con 0x7f1744145510 2023-11-24T06:11:16.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f174a800700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174415fd80 con 0x7f17440746a0 2023-11-24T06:11:16.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f174a800700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1744160060 con 0x7f17440ce9a0 2023-11-24T06:11:16.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f17437fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17440746a0 0x7f17440ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f17437fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f17440746a0 0x7f17440ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49410/0 (socket says 172.21.15.97:49410) 2023-11-24T06:11:16.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.624+0000 7f17437fe700 1 -- 172.21.15.97:0/3207211685 learned_addr learned my addr 172.21.15.97:0/3207211685 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f1748d9d700 1 --2- 172.21.15.97:0/3207211685 >> v2:172.21.15.183:3300/0 conn(0x7f17440ce9a0 0x7f174406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f17437fe700 1 -- 172.21.15.97:0/3207211685 >> v2:172.21.15.183:3300/0 conn(0x7f17440ce9a0 msgr2=0x7f174406eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f17437fe700 1 --2- 172.21.15.97:0/3207211685 >> v2:172.21.15.183:3300/0 conn(0x7f17440ce9a0 0x7f174406eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f17437fe700 1 -- 172.21.15.97:0/3207211685 >> v1:172.21.15.183:6789/0 conn(0x7f1744145510 legacy=0x7f1744073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f17437fe700 1 -- 172.21.15.97:0/3207211685 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1744160340 con 0x7f17440746a0 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f1748d9d700 1 --2- 172.21.15.97:0/3207211685 >> v2:172.21.15.183:3300/0 conn(0x7f17440ce9a0 0x7f174406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f17437fe700 1 --2- 172.21.15.97:0/3207211685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440746a0 0x7f17440ce460 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1744156b80 tx=0x7f1738009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25ff20b889b942a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f1742ffd700 1 -- 172.21.15.97:0/3207211685 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738004230 con 0x7f17440746a0 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f1742ffd700 1 -- 172.21.15.97:0/3207211685 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738004390 con 0x7f17440746a0 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.625+0000 7f1742ffd700 1 -- 172.21.15.97:0/3207211685 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380036a0 con 0x7f17440746a0 2023-11-24T06:11:16.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.626+0000 7f174a800700 1 -- 172.21.15.97:0/3207211685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440746a0 msgr2=0x7f17440ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.626+0000 7f174a800700 1 --2- 172.21.15.97:0/3207211685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440746a0 0x7f17440ce460 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1744156b80 tx=0x7f1738009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.626+0000 7f174a800700 1 -- 172.21.15.97:0/3207211685 shutdown_connections 2023-11-24T06:11:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.626+0000 7f174a800700 1 --2- 172.21.15.97:0/3207211685 >> v2:172.21.15.183:3300/0 conn(0x7f17440ce9a0 0x7f174406eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.626+0000 7f174a800700 1 --2- 172.21.15.97:0/3207211685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440746a0 0x7f17440ce460 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.626+0000 7f174a800700 1 -- 172.21.15.97:0/3207211685 >> 172.21.15.97:0/3207211685 conn(0x7f1744072990 msgr2=0x7f174406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.626+0000 7f174a800700 1 -- 172.21.15.97:0/3207211685 shutdown_connections 2023-11-24T06:11:16.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.626+0000 7f174a800700 1 -- 172.21.15.97:0/3207211685 wait complete. 2023-11-24T06:11:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.627+0000 7f174a800700 1 Processor -- start 2023-11-24T06:11:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.627+0000 7f174a800700 1 -- start start 2023-11-24T06:11:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f174a800700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17440746a0 0x7f17440b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f174a800700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440ce9a0 0x7f1744155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f174a800700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1744145510 0x7f174415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f174a800700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441f5ec0 con 0x7f17440ce9a0 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f174a800700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441f5aa0 con 0x7f1744145510 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f174a800700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17440b2760 con 0x7f17440746a0 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440ce9a0 0x7f1744155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440ce9a0 0x7f1744155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49416/0 (socket says 172.21.15.97:49416) 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 -- 172.21.15.97:0/2701578638 learned_addr learned my addr 172.21.15.97:0/2701578638 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f1748d9d700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1744145510 0x7f174415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f1743fff700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17440746a0 0x7f17440b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 -- 172.21.15.97:0/2701578638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17440746a0 msgr2=0x7f17440b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17440746a0 0x7f17440b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 -- 172.21.15.97:0/2701578638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1744145510 msgr2=0x7f174415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1744145510 0x7f174415eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 -- 172.21.15.97:0/2701578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738009550 con 0x7f17440ce9a0 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.628+0000 7f17437fe700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440ce9a0 0x7f1744155670 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f17440b5fb0 tx=0x7f1738004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.629+0000 7f1740ff9700 1 -- 172.21.15.97:0/2701578638 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738003c10 con 0x7f17440ce9a0 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.629+0000 7f1740ff9700 1 -- 172.21.15.97:0/2701578638 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173801f070 con 0x7f17440ce9a0 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.629+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17440b2500 con 0x7f17440ce9a0 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.629+0000 7f1740ff9700 1 -- 172.21.15.97:0/2701578638 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738011630 con 0x7f17440ce9a0 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.629+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174413ec40 con 0x7f17440ce9a0 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.630+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17441f5be0 con 0x7f17440ce9a0 2023-11-24T06:11:16.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.630+0000 7f1740ff9700 1 -- 172.21.15.97:0/2701578638 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f1738021030 con 0x7f17440ce9a0 2023-11-24T06:11:16.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.630+0000 7f1740ff9700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f173006b1a0 0x7f173006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.631+0000 7f1740ff9700 1 -- 172.21.15.97:0/2701578638 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f173808af30 con 0x7f17440ce9a0 2023-11-24T06:11:16.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.631+0000 7f1743fff700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f173006b1a0 0x7f173006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.631+0000 7f1743fff700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f173006b1a0 0x7f173006d660 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17440ce6b0 tx=0x7f17340090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.635+0000 7f1740ff9700 1 -- 172.21.15.97:0/2701578638 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17441f5be0 con 0x7f17440ce9a0 2023-11-24T06:11:16.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.828+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17441f5be0 con 0x7f17440ce9a0 2023-11-24T06:11:16.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.829+0000 7f1740ff9700 1 -- 172.21.15.97:0/2701578638 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f17441f5be0 con 0x7f17440ce9a0 2023-11-24T06:11:16.833 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:16.833 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:05.166928+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:01.950100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:02.526161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:16.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f173006b1a0 msgr2=0x7f173006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f173006b1a0 0x7f173006d660 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17440ce6b0 tx=0x7f17340090e0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440ce9a0 msgr2=0x7f1744155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440ce9a0 0x7f1744155670 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f17440b5fb0 tx=0x7f1738004750 comp rx=0 tx=0).stop 2023-11-24T06:11:16.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 shutdown_connections 2023-11-24T06:11:16.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1744145510 0x7f174415eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f173006b1a0 0x7f173006d660 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f17440746a0 0x7f17440b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 --2- 172.21.15.97:0/2701578638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17440ce9a0 0x7f1744155670 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 >> 172.21.15.97:0/2701578638 conn(0x7f1744072990 msgr2=0x7f174406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 shutdown_connections 2023-11-24T06:11:16.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.831+0000 7f174a800700 1 -- 172.21.15.97:0/2701578638 wait complete. 2023-11-24T06:11:16.847 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:01.950100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:02.526161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:16.847 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:16.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.966+0000 7f26c557f700 1 Processor -- start 2023-11-24T06:11:16.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.966+0000 7f26c557f700 1 -- start start 2023-11-24T06:11:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26c557f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26c015e140 0x7f26c00d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26c557f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 0x7f26c00d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26c557f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c00b80e0 con 0x7f26c00b98c0 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26c557f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c015e520 con 0x7f26c015e140 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26c557f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00d2930 con 0x7f26c00d39d0 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26beffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26c015e140 0x7f26c00d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26beffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26c015e140 0x7f26c00d3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49422/0 (socket says 172.21.15.97:49422) 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26beffd700 1 -- 172.21.15.97:0/919955989 learned_addr learned my addr 172.21.15.97:0/919955989 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26be7fc700 1 --2- 172.21.15.97:0/919955989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 0x7f26c00d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26beffd700 1 -- 172.21.15.97:0/919955989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 msgr2=0x7f26c00d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26beffd700 1 --2- 172.21.15.97:0/919955989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 0x7f26c00d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26beffd700 1 -- 172.21.15.97:0/919955989 >> v1:172.21.15.183:6789/0 conn(0x7f26c00b98c0 legacy=0x7f26c00b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.967+0000 7f26beffd700 1 -- 172.21.15.97:0/919955989 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c00d2c10 con 0x7f26c015e140 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26beffd700 1 --2- 172.21.15.97:0/919955989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c00d3ec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f26b40098f0 tx=0x7f26b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9437f09ff84fd1d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26bdffb700 1 -- 172.21.15.97:0/919955989 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b400b8a0 con 0x7f26c015e140 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26bdffb700 1 -- 172.21.15.97:0/919955989 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b400ba00 con 0x7f26c015e140 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26bdffb700 1 -- 172.21.15.97:0/919955989 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b400bd90 con 0x7f26c015e140 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26c557f700 1 -- 172.21.15.97:0/919955989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 msgr2=0x7f26c00d3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26c557f700 1 --2- 172.21.15.97:0/919955989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c00d3ec0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f26b40098f0 tx=0x7f26b4009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26c557f700 1 -- 172.21.15.97:0/919955989 shutdown_connections 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26c557f700 1 --2- 172.21.15.97:0/919955989 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c00d3ec0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26c557f700 1 --2- 172.21.15.97:0/919955989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 0x7f26c00d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26c557f700 1 -- 172.21.15.97:0/919955989 >> 172.21.15.97:0/919955989 conn(0x7f26c006b410 msgr2=0x7f26c006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26c557f700 1 -- 172.21.15.97:0/919955989 shutdown_connections 2023-11-24T06:11:16.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.968+0000 7f26c557f700 1 -- 172.21.15.97:0/919955989 wait complete. 2023-11-24T06:11:16.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.970+0000 7f26c557f700 1 Processor -- start 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.970+0000 7f26c557f700 1 -- start start 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26c557f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26c00b98c0 0x7f26c00c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26c557f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 0x7f26c01f7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26c557f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c01f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26c557f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c01f7c60 con 0x7f26c015e140 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26c557f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00c7f70 con 0x7f26c00b98c0 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26c557f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c00c8250 con 0x7f26c00d39d0 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c01f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26beffd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26c00b98c0 0x7f26c00c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26bf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c01f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49432/0 (socket says 172.21.15.97:49432) 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26bf7fe700 1 -- 172.21.15.97:0/1922257579 learned_addr learned my addr 172.21.15.97:0/1922257579 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26be7fc700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 0x7f26c01f7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26bf7fe700 1 -- 172.21.15.97:0/1922257579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 msgr2=0x7f26c01f7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26bf7fe700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 0x7f26c01f7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26bf7fe700 1 -- 172.21.15.97:0/1922257579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26c00b98c0 msgr2=0x7f26c00c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26bf7fe700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26c00b98c0 0x7f26c00c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.971+0000 7f26bf7fe700 1 -- 172.21.15.97:0/1922257579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b4009550 con 0x7f26c015e140 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.972+0000 7f26bf7fe700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c01f3020 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f26b800cbc0 tx=0x7f26b800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.972+0000 7f26a77fe700 1 -- 172.21.15.97:0/1922257579 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8003c00 con 0x7f26c015e140 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.972+0000 7f26a77fe700 1 -- 172.21.15.97:0/1922257579 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b8003d60 con 0x7f26c015e140 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.972+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c00c8530 con 0x7f26c015e140 2023-11-24T06:11:16.978 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.972+0000 7f26a77fe700 1 -- 172.21.15.97:0/1922257579 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b800aae0 con 0x7f26c015e140 2023-11-24T06:11:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.972+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c00c8af0 con 0x7f26c015e140 2023-11-24T06:11:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.973+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c0068fb0 con 0x7f26c015e140 2023-11-24T06:11:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.973+0000 7f26a77fe700 1 -- 172.21.15.97:0/1922257579 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f26b800ac40 con 0x7f26c015e140 2023-11-24T06:11:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.973+0000 7f26a77fe700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f26ac06b1a0 0x7f26ac06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:16.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.974+0000 7f26a77fe700 1 -- 172.21.15.97:0/1922257579 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26b8089850 con 0x7f26c015e140 2023-11-24T06:11:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.974+0000 7f26beffd700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f26ac06b1a0 0x7f26ac06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:16.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.974+0000 7f26beffd700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f26ac06b1a0 0x7f26ac06d660 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f26b4014040 tx=0x7f26b4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:16.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:16.978+0000 7f26a77fe700 1 -- 172.21.15.97:0/1922257579 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26b80545b0 con 0x7f26c015e140 2023-11-24T06:11:17.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.165+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26c00b80e0 con 0x7f26c015e140 2023-11-24T06:11:17.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.165+0000 7f26a77fe700 1 -- 172.21.15.97:0/1922257579 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26c00b80e0 con 0x7f26c015e140 2023-11-24T06:11:17.169 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.167+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f26ac06b1a0 msgr2=0x7f26ac06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f26ac06b1a0 0x7f26ac06d660 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f26b4014040 tx=0x7f26b4008030 comp rx=0 tx=0).stop 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 msgr2=0x7f26c01f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c01f3020 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f26b800cbc0 tx=0x7f26b800ced0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 shutdown_connections 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26c015e140 0x7f26c01f3020 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26c00b98c0 0x7f26c00c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f26ac06b1a0 0x7f26ac06d660 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 --2- 172.21.15.97:0/1922257579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26c00d39d0 0x7f26c01f7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 >> 172.21.15.97:0/1922257579 conn(0x7f26c006b410 msgr2=0x7f26c00b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 shutdown_connections 2023-11-24T06:11:17.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.168+0000 7f26c557f700 1 -- 172.21.15.97:0/1922257579 wait complete. 2023-11-24T06:11:17.185 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:17.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.305+0000 7feb1ecc9700 1 Processor -- start 2023-11-24T06:11:17.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.305+0000 7feb1ecc9700 1 -- start start 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb1ecc9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7feb180cdf50 0x7feb180b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb1ecc9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb180b4970 con 0x7feb1815be20 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb1ecc9700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb180cfed0 con 0x7feb180b4e00 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb1ecc9700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb180b4360 con 0x7feb180cdf50 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb1ca65700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7feb1815be20 0x7feb180af790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41888/0 (socket says 172.21.15.97:41888) 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb1ca65700 1 -- 172.21.15.97:0/552109922 learned_addr learned my addr 172.21.15.97:0/552109922 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb1d266700 1 --2- 172.21.15.97:0/552109922 >> v2:172.21.15.183:3300/0 conn(0x7feb180cdf50 0x7feb180b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870690612 0 0) 0x7feb180b4970 con 0x7feb1815be20 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb080035b0 con 0x7feb1815be20 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4238488926 0 0) 0x7feb080035b0 con 0x7feb1815be20 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 >> v2:172.21.15.183:3300/0 conn(0x7feb180cdf50 msgr2=0x7feb180b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 --2- 172.21.15.97:0/552109922 >> v2:172.21.15.183:3300/0 conn(0x7feb180cdf50 0x7feb180b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 >> v1:172.21.15.183:6789/0 conn(0x7feb180b4e00 legacy=0x7feb180cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1806f940 con 0x7feb1815be20 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7feb0c001670 con 0x7feb1815be20 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 >> v1:172.21.15.97:6789/0 conn(0x7feb1815be20 legacy=0x7feb180af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb08003a00 0x7feb08003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08004850 0x7feb08004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.306+0000 7feb177fe700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08005210 0x7feb0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb180b4360 con 0x7feb08004850 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0800c0b0 con 0x7feb08005210 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0800c270 con 0x7feb08003a00 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb17fff700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08004850 0x7feb08004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb1ca65700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb08003a00 0x7feb08003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb1d266700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08005210 0x7feb0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb17fff700 1 -- 172.21.15.97:0/552109922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb08003a00 msgr2=0x7feb08003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb17fff700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb08003a00 0x7feb08003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb17fff700 1 -- 172.21.15.97:0/552109922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08005210 msgr2=0x7feb0800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb17fff700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08005210 0x7feb0800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb1d266700 1 -- 172.21.15.97:0/552109922 reap_dead start 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb17fff700 1 -- 172.21.15.97:0/552109922 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080035b0 con 0x7feb08004850 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb17fff700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08004850 0x7feb08004cd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7feb040017c0 tx=0x7feb08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeafcc2b744f0e0b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040052d0 con 0x7feb08004850 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb04005450 con 0x7feb08004850 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.307+0000 7feb177fe700 1 -- 172.21.15.97:0/552109922 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb040038c0 con 0x7feb08004850 2023-11-24T06:11:17.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.308+0000 7feb1ecc9700 1 -- 172.21.15.97:0/552109922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08004850 msgr2=0x7feb08004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.308+0000 7feb1ecc9700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08004850 0x7feb08004cd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7feb040017c0 tx=0x7feb08005880 comp rx=0 tx=0).stop 2023-11-24T06:11:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.308+0000 7feb1ecc9700 1 -- 172.21.15.97:0/552109922 shutdown_connections 2023-11-24T06:11:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.308+0000 7feb1ecc9700 1 --2- 172.21.15.97:0/552109922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb08004850 0x7feb08004cd0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.308+0000 7feb1ecc9700 1 -- 172.21.15.97:0/552109922 >> 172.21.15.97:0/552109922 conn(0x7feb180b2050 msgr2=0x7feb180b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.308+0000 7feb1ecc9700 1 -- 172.21.15.97:0/552109922 shutdown_connections 2023-11-24T06:11:17.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.308+0000 7feb1ecc9700 1 -- 172.21.15.97:0/552109922 wait complete. 2023-11-24T06:11:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.309+0000 7feb1ecc9700 1 Processor -- start 2023-11-24T06:11:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.309+0000 7feb1ecc9700 1 -- start start 2023-11-24T06:11:17.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1ecc9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08004850 0x7feb18148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1ecc9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb181e5730 0x7feb181e5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1ecc9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb1815d190 0x7feb1815d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1ecc9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb181e60f0 con 0x7feb1815d190 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1ecc9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1814f910 con 0x7feb08004850 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1ecc9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1815df00 con 0x7feb181e5730 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb1815d190 0x7feb1815d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb1815d190 0x7feb1815d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49452/0 (socket says 172.21.15.97:49452) 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 -- 172.21.15.97:0/500511221 learned_addr learned my addr 172.21.15.97:0/500511221 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1ca65700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08004850 0x7feb18148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb17fff700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb181e5730 0x7feb181e5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 -- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb181e5730 msgr2=0x7feb181e5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb181e5730 0x7feb181e5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 -- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08004850 msgr2=0x7feb18148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08004850 0x7feb18148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 -- 172.21.15.97:0/500511221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1814f2c0 con 0x7feb1815d190 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1ca65700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08004850 0x7feb18148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb1d266700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb1815d190 0x7feb1815d640 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7feb18072520 tx=0x7feb10011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb157fa700 1 -- 172.21.15.97:0/500511221 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000e040 con 0x7feb1815d190 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.310+0000 7feb157fa700 1 -- 172.21.15.97:0/500511221 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1000f040 con 0x7feb1815d190 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.311+0000 7feb157fa700 1 -- 172.21.15.97:0/500511221 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000fd30 con 0x7feb1815d190 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.311+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb1814f5a0 con 0x7feb1815d190 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.311+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb1815db80 con 0x7feb1815d190 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.312+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb18068d90 con 0x7feb1815d190 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.312+0000 7feb157fa700 1 -- 172.21.15.97:0/500511221 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7feb10022460 con 0x7feb1815d190 2023-11-24T06:11:17.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.312+0000 7feb157fa700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feafc06b1a0 0x7feafc06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.312+0000 7feb1ca65700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feafc06b1a0 0x7feafc06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.312+0000 7feb157fa700 1 -- 172.21.15.97:0/500511221 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb1008a170 con 0x7feb1815d190 2023-11-24T06:11:17.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.313+0000 7feb1ca65700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feafc06b1a0 0x7feafc06d660 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000fb0 tx=0x7feb0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.317+0000 7feb157fa700 1 -- 172.21.15.97:0/500511221 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb10054dc0 con 0x7feb1815d190 2023-11-24T06:11:17.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.507+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb1814f120 con 0x7feb1815d190 2023-11-24T06:11:17.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.508+0000 7feb157fa700 1 -- 172.21.15.97:0/500511221 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb1814f120 con 0x7feb1815d190 2023-11-24T06:11:17.512 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:11:17.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feafc06b1a0 msgr2=0x7feafc06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feafc06b1a0 0x7feafc06d660 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feb0c000fb0 tx=0x7feb0c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:17.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb1815d190 msgr2=0x7feb1815d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb1815d190 0x7feb1815d640 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7feb18072520 tx=0x7feb10011ed0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 shutdown_connections 2023-11-24T06:11:17.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb1815d190 0x7feb1815d640 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb08004850 0x7feb18148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feafc06b1a0 0x7feafc06d660 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 --2- 172.21.15.97:0/500511221 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feb181e5730 0x7feb181e5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 >> 172.21.15.97:0/500511221 conn(0x7feb180b2050 msgr2=0x7feb181e69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 shutdown_connections 2023-11-24T06:11:17.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.510+0000 7feb1ecc9700 1 -- 172.21.15.97:0/500511221 wait complete. 2023-11-24T06:11:17.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.647+0000 7f198e011700 1 Processor -- start 2023-11-24T06:11:17.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.647+0000 7f198e011700 1 -- start start 2023-11-24T06:11:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f198e011700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f198815be70 0x7f19880cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f198e011700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f19880b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f198e011700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19880b5ee0 con 0x7f19880add40 2023-11-24T06:11:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f198e011700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198806bfc0 con 0x7f198815be70 2023-11-24T06:11:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f198e011700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198806ba00 con 0x7f19880746a0 2023-11-24T06:11:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f1986ffd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f19880add40 0x7f1988073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51230/0 (socket says 172.21.15.97:51230) 2023-11-24T06:11:17.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f1986ffd700 1 -- 172.21.15.97:0/3196150604 learned_addr learned my addr 172.21.15.97:0/3196150604 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f1987fff700 1 --2- 172.21.15.97:0/3196150604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f19880b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.648+0000 7f19877fe700 1 --2- 172.21.15.97:0/3196150604 >> v2:172.21.15.183:3300/0 conn(0x7f198815be70 0x7f19880cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f19867fc700 1 -- 172.21.15.97:0/3196150604 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167401933 0 0) 0x7f19880b5ee0 con 0x7f19880add40 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f19867fc700 1 -- 172.21.15.97:0/3196150604 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19700035b0 con 0x7f19880add40 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f19867fc700 1 -- 172.21.15.97:0/3196150604 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 750092100 0 0) 0x7f19700035b0 con 0x7f19880add40 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f1987fff700 1 -- 172.21.15.97:0/3196150604 >> v2:172.21.15.183:3300/0 conn(0x7f198815be70 msgr2=0x7f19880cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f1987fff700 1 --2- 172.21.15.97:0/3196150604 >> v2:172.21.15.183:3300/0 conn(0x7f198815be70 0x7f19880cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f1987fff700 1 -- 172.21.15.97:0/3196150604 >> v1:172.21.15.183:6789/0 conn(0x7f19880add40 legacy=0x7f1988073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f1987fff700 1 -- 172.21.15.97:0/3196150604 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198806bce0 con 0x7f19880746a0 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f1987fff700 1 --2- 172.21.15.97:0/3196150604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f19880b3c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1980009d90 tx=0x7f198000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94525e8c06335c27 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f19867fc700 1 -- 172.21.15.97:0/3196150604 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198000e070 con 0x7f19880746a0 2023-11-24T06:11:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f19867fc700 1 -- 172.21.15.97:0/3196150604 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980003a20 con 0x7f19880746a0 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.649+0000 7f19867fc700 1 -- 172.21.15.97:0/3196150604 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1978001670 con 0x7f19880746a0 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.650+0000 7f198e011700 1 -- 172.21.15.97:0/3196150604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 msgr2=0x7f19880b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.650+0000 7f198e011700 1 --2- 172.21.15.97:0/3196150604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f19880b3c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1980009d90 tx=0x7f198000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.650+0000 7f198e011700 1 -- 172.21.15.97:0/3196150604 shutdown_connections 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.650+0000 7f198e011700 1 --2- 172.21.15.97:0/3196150604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f19880b3c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.650+0000 7f198e011700 1 --2- 172.21.15.97:0/3196150604 >> v2:172.21.15.183:3300/0 conn(0x7f198815be70 0x7f19880cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.650+0000 7f198e011700 1 -- 172.21.15.97:0/3196150604 >> 172.21.15.97:0/3196150604 conn(0x7f19880d0bf0 msgr2=0x7f19880d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.650+0000 7f198e011700 1 -- 172.21.15.97:0/3196150604 shutdown_connections 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.650+0000 7f198e011700 1 -- 172.21.15.97:0/3196150604 wait complete. 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.651+0000 7f198e011700 1 Processor -- start 2023-11-24T06:11:17.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.651+0000 7f198e011700 1 -- start start 2023-11-24T06:11:17.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f198e011700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f198815e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f198e011700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19880add40 0x7f198815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f198e011700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198815be70 0x7f19881f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f198e011700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198815e9b0 con 0x7f198815be70 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f198e011700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198815ec90 con 0x7f19880add40 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f198e011700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198815ef70 con 0x7f19880746a0 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f1987fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198815be70 0x7f19881f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f1987fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198815be70 0x7f19881f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49466/0 (socket says 172.21.15.97:49466) 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f1987fff700 1 -- 172.21.15.97:0/428564104 learned_addr learned my addr 172.21.15.97:0/428564104 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.652+0000 7f1986ffd700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19880add40 0x7f198815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f19877fe700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f198815e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f1987fff700 1 -- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 msgr2=0x7f198815e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f1987fff700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f198815e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f1987fff700 1 -- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19880add40 msgr2=0x7f198815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f1987fff700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19880add40 0x7f198815d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f1987fff700 1 -- 172.21.15.97:0/428564104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980009a70 con 0x7f198815be70 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f1986ffd700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19880add40 0x7f198815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f19877fe700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f198815e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f1987fff700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198815be70 0x7f19881f7410 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1980004000 tx=0x7f1980004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f196ffff700 1 -- 172.21.15.97:0/428564104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19800046b0 con 0x7f198815be70 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f196ffff700 1 -- 172.21.15.97:0/428564104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1980004850 con 0x7f198815be70 2023-11-24T06:11:17.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198815f250 con 0x7f198815be70 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f196ffff700 1 -- 172.21.15.97:0/428564104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198001c680 con 0x7f198815be70 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.653+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198815e760 con 0x7f198815be70 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.654+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198806db60 con 0x7f198815be70 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.654+0000 7f196ffff700 1 -- 172.21.15.97:0/428564104 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f198001c7e0 con 0x7f198815be70 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.655+0000 7f196ffff700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f197406b2c0 0x7f197406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.655+0000 7f196ffff700 1 -- 172.21.15.97:0/428564104 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f198008b8c0 con 0x7f198815be70 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.655+0000 7f19877fe700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f197406b2c0 0x7f197406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.655+0000 7f19877fe700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f197406b2c0 0x7f197406d780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f198815b880 tx=0x7f197c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.659+0000 7f196ffff700 1 -- 172.21.15.97:0/428564104 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1980056310 con 0x7f198815be70 2023-11-24T06:11:17.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.848+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19880ae1b0 con 0x7f198815be70 2023-11-24T06:11:17.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.848+0000 7f196ffff700 1 -- 172.21.15.97:0/428564104 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f198005ee60 con 0x7f198815be70 2023-11-24T06:11:17.852 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:11:17.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f197406b2c0 msgr2=0x7f197406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f197406b2c0 0x7f197406d780 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f198815b880 tx=0x7f197c008040 comp rx=0 tx=0).stop 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198815be70 msgr2=0x7f19881f7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198815be70 0x7f19881f7410 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1980004000 tx=0x7f1980004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 shutdown_connections 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198815be70 0x7f19881f7410 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f197406b2c0 0x7f197406d780 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19880746a0 0x7f198815e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 --2- 172.21.15.97:0/428564104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19880add40 0x7f198815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.850+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 >> 172.21.15.97:0/428564104 conn(0x7f19880d0bf0 msgr2=0x7f1988158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.851+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 shutdown_connections 2023-11-24T06:11:17.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.851+0000 7f198e011700 1 -- 172.21.15.97:0/428564104 wait complete. 2023-11-24T06:11:17.867 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.984+0000 7fd387ce7700 1 Processor -- start 2023-11-24T06:11:17.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.984+0000 7fd387ce7700 1 -- start start 2023-11-24T06:11:17.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd387ce7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd38006c710 0x7fd380165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd387ce7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3800af950 con 0x7fd3800adf00 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd387ce7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3800afc30 con 0x7fd3801446a0 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd387ce7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd380144a80 con 0x7fd38006c710 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd386284700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd38006c710 0x7fd380165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd386284700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd38006c710 0x7fd380165b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49474/0 (socket says 172.21.15.97:49474) 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd385282700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd3800adf00 0x7fd38006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41922/0 (socket says 172.21.15.97:41922) 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd386284700 1 -- 172.21.15.97:0/765601077 learned_addr learned my addr 172.21.15.97:0/765601077 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd386284700 1 -- 172.21.15.97:0/765601077 >> v1:172.21.15.183:6789/0 conn(0x7fd3801446a0 legacy=0x7fd3800b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd386284700 1 -- 172.21.15.97:0/765601077 >> v1:172.21.15.97:6789/0 conn(0x7fd3800adf00 legacy=0x7fd38006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd386284700 1 -- 172.21.15.97:0/765601077 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800aff10 con 0x7fd38006c710 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.985+0000 7fd384a81700 1 -- 172.21.15.97:0/765601077 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489574521 0 0) 0x7fd3800af950 con 0x7fd3800adf00 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd386284700 1 --2- 172.21.15.97:0/765601077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd380165b70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd378009e50 tx=0x7fd37800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f59dd3d4c425f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd384a81700 1 -- 172.21.15.97:0/765601077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37800e070 con 0x7fd38006c710 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd384a81700 1 -- 172.21.15.97:0/765601077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd378003a20 con 0x7fd38006c710 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd384a81700 1 -- 172.21.15.97:0/765601077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378003dd0 con 0x7fd38006c710 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd387ce7700 1 -- 172.21.15.97:0/765601077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 msgr2=0x7fd380165b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd387ce7700 1 --2- 172.21.15.97:0/765601077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd380165b70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd378009e50 tx=0x7fd37800b330 comp rx=0 tx=0).stop 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd387ce7700 1 -- 172.21.15.97:0/765601077 shutdown_connections 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd387ce7700 1 --2- 172.21.15.97:0/765601077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd380165b70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd387ce7700 1 -- 172.21.15.97:0/765601077 >> 172.21.15.97:0/765601077 conn(0x7fd380071b90 msgr2=0x7fd380071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:17.992 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd387ce7700 1 -- 172.21.15.97:0/765601077 shutdown_connections 2023-11-24T06:11:17.993 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.986+0000 7fd387ce7700 1 -- 172.21.15.97:0/765601077 wait complete. 2023-11-24T06:11:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd387ce7700 1 Processor -- start 2023-11-24T06:11:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd387ce7700 1 -- start start 2023-11-24T06:11:17.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd387ce7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd38015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd387ce7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3800adf00 0x7fd38014f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd387ce7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3801446a0 0x7fd38013d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd387ce7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3800ce140 con 0x7fd38006c710 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd387ce7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38014fa20 con 0x7fd3800adf00 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd387ce7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38013e020 con 0x7fd3801446a0 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd385a83700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd38015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd385a83700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd38015b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49476/0 (socket says 172.21.15.97:49476) 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.988+0000 7fd385a83700 1 -- 172.21.15.97:0/2036327327 learned_addr learned my addr 172.21.15.97:0/2036327327 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd385282700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3800adf00 0x7fd38014f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd386284700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3801446a0 0x7fd38013d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd385a83700 1 -- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3801446a0 msgr2=0x7fd38013d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd385a83700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3801446a0 0x7fd38013d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd385a83700 1 -- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3800adf00 msgr2=0x7fd38014f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd385a83700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3800adf00 0x7fd38014f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd385a83700 1 -- 172.21.15.97:0/2036327327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd378009b30 con 0x7fd38006c710 2023-11-24T06:11:17.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd386284700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3801446a0 0x7fd38013d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd385a83700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd38015b5b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b4a0 tx=0x7fd37c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd3767fc700 1 -- 172.21.15.97:0/2036327327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c011830 con 0x7fd38006c710 2023-11-24T06:11:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd3767fc700 1 -- 172.21.15.97:0/2036327327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c011990 con 0x7fd38006c710 2023-11-24T06:11:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd380159960 con 0x7fd38006c710 2023-11-24T06:11:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd3767fc700 1 -- 172.21.15.97:0/2036327327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c011d40 con 0x7fd38006c710 2023-11-24T06:11:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.989+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3800cda80 con 0x7fd38006c710 2023-11-24T06:11:17.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.990+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3800753f0 con 0x7fd38006c710 2023-11-24T06:11:17.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.991+0000 7fd3767fc700 1 -- 172.21.15.97:0/2036327327 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd37c01a400 con 0x7fd38006c710 2023-11-24T06:11:17.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.991+0000 7fd3767fc700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd36406b2c0 0x7fd36406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:17.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.991+0000 7fd385282700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd36406b2c0 0x7fd36406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:17.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.991+0000 7fd3767fc700 1 -- 172.21.15.97:0/2036327327 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd37c089990 con 0x7fd38006c710 2023-11-24T06:11:17.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.991+0000 7fd385282700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd36406b2c0 0x7fd36406d780 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd38013ca30 tx=0x7fd370007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:17.996+0000 7fd3767fc700 1 -- 172.21.15.97:0/2036327327 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd37c0545a0 con 0x7fd38006c710 2023-11-24T06:11:18.205 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.202+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd380067f60 con 0x7fd38006c710 2023-11-24T06:11:18.206 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.202+0000 7fd3767fc700 1 -- 172.21.15.97:0/2036327327 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd37c015080 con 0x7fd38006c710 2023-11-24T06:11:18.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd36406b2c0 msgr2=0x7fd36406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd36406b2c0 0x7fd36406d780 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd38013ca30 tx=0x7fd370007be0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 msgr2=0x7fd38015b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.208 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd38015b5b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b4a0 tx=0x7fd37c00b860 comp rx=0 tx=0).stop 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 shutdown_connections 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd36406b2c0 0x7fd36406d780 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd3801446a0 0x7fd38013d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd38006c710 0x7fd38015b5b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 --2- 172.21.15.97:0/2036327327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd3800adf00 0x7fd38014f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 >> 172.21.15.97:0/2036327327 conn(0x7fd380071b90 msgr2=0x7fd380140a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 shutdown_connections 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.204+0000 7fd387ce7700 1 -- 172.21.15.97:0/2036327327 wait complete. 2023-11-24T06:11:18.209 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:11:18.219 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:11:18.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:18.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:18.267 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:11:18.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:18.267 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:11:18.284 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:18.284 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:18.307 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:18.307 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:18.369 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:18.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.485+0000 7f3bd5d62700 1 Processor -- start 2023-11-24T06:11:18.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.485+0000 7f3bd5d62700 1 -- start start 2023-11-24T06:11:18.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bd5d62700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f3bd015bdd0 0x7f3bd01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bd5d62700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3bd00b3370 0x7f3bd00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bd5d62700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd00b2030 con 0x7f3bd00d1c70 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bd5d62700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00b2f50 con 0x7f3bd00b3370 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bd5d62700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd00b22d0 con 0x7f3bd015bdd0 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bceffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3bd00b3370 0x7f3bd00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bceffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3bd00b3370 0x7f3bd00b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49486/0 (socket says 172.21.15.97:49486) 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bcffff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3bd00d1c70 0x7f3bd00ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41942/0 (socket says 172.21.15.97:41942) 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.486+0000 7f3bceffd700 1 -- 172.21.15.97:0/3245523836 learned_addr learned my addr 172.21.15.97:0/3245523836 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bcf7fe700 1 --2- 172.21.15.97:0/3245523836 >> v2:172.21.15.183:3300/0 conn(0x7f3bd015bdd0 0x7f3bd01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bceffd700 1 -- 172.21.15.97:0/3245523836 >> v2:172.21.15.183:3300/0 conn(0x7f3bd015bdd0 msgr2=0x7f3bd01604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bceffd700 1 --2- 172.21.15.97:0/3245523836 >> v2:172.21.15.183:3300/0 conn(0x7f3bd015bdd0 0x7f3bd01604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bceffd700 1 -- 172.21.15.97:0/3245523836 >> v1:172.21.15.97:6789/0 conn(0x7f3bd00d1c70 legacy=0x7f3bd00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bceffd700 1 -- 172.21.15.97:0/3245523836 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00b25b0 con 0x7f3bd00b3370 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bce7fc700 1 -- 172.21.15.97:0/3245523836 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288952348 0 0) 0x7f3bd00b2030 con 0x7f3bd00d1c70 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bceffd700 1 --2- 172.21.15.97:0/3245523836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00b3370 0x7f3bd00b1af0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006ec10 tx=0x7f3bb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde85e03ed4a9cdb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bce7fc700 1 -- 172.21.15.97:0/3245523836 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800b870 con 0x7f3bd00b3370 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bce7fc700 1 -- 172.21.15.97:0/3245523836 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb800b9d0 con 0x7f3bd00b3370 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bce7fc700 1 -- 172.21.15.97:0/3245523836 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800bd60 con 0x7f3bd00b3370 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3245523836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00b3370 msgr2=0x7f3bd00b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.487+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3245523836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00b3370 0x7f3bd00b1af0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3bd006ec10 tx=0x7f3bb8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.488+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3245523836 shutdown_connections 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.488+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3245523836 >> v2:172.21.15.183:3300/0 conn(0x7f3bd015bdd0 0x7f3bd01604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.488+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3245523836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00b3370 0x7f3bd00b1af0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.488+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3245523836 >> 172.21.15.97:0/3245523836 conn(0x7f3bd00731a0 msgr2=0x7f3bd009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.488+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3245523836 shutdown_connections 2023-11-24T06:11:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.488+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3245523836 wait complete. 2023-11-24T06:11:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.489+0000 7f3bd5d62700 1 Processor -- start 2023-11-24T06:11:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.489+0000 7f3bd5d62700 1 -- start start 2023-11-24T06:11:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bd5d62700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3bd00b3370 0x7f3bd00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bd5d62700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00d1c70 0x7f3bd01e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bd5d62700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bd015bdd0 0x7f3bd01e6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bd5d62700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00cb8b0 con 0x7f3bd00d1c70 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bd5d62700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd0150aa0 con 0x7f3bd015bdd0 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bd5d62700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00caca0 con 0x7f3bd00b3370 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00d1c70 0x7f3bd01e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00d1c70 0x7f3bd01e5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49494/0 (socket says 172.21.15.97:49494) 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 -- 172.21.15.97:0/3276253052 learned_addr learned my addr 172.21.15.97:0/3276253052 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 -- 172.21.15.97:0/3276253052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3bd00b3370 msgr2=0x7f3bd00ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bcffff700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bd015bdd0 0x7f3bd01e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3bd00b3370 0x7f3bd00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 -- 172.21.15.97:0/3276253052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bd015bdd0 msgr2=0x7f3bd01e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bd015bdd0 0x7f3bd01e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 -- 172.21.15.97:0/3276253052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8009550 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bcffff700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bd015bdd0 0x7f3bd01e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bceffd700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00d1c70 0x7f3bd01e5c40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009c10 tx=0x7f3bb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bc7fff700 1 -- 172.21.15.97:0/3276253052 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8004310 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bc7fff700 1 -- 172.21.15.97:0/3276253052 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb801f070 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd00caf80 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bc7fff700 1 -- 172.21.15.97:0/3276253052 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80036a0 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.490+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd00cb540 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.491+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd0067f60 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.492+0000 7f3bc7fff700 1 -- 172.21.15.97:0/3276253052 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3bb8021030 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.492+0000 7f3bc7fff700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3bb006b2c0 0x7f3bb006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.492+0000 7f3bc7fff700 1 -- 172.21.15.97:0/3276253052 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bb808b080 con 0x7f3bd00d1c70 2023-11-24T06:11:18.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.492+0000 7f3bcf7fe700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3bb006b2c0 0x7f3bb006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.492+0000 7f3bcf7fe700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3bb006b2c0 0x7f3bb006d780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000a890 tx=0x7f3bc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.496+0000 7f3bc7fff700 1 -- 172.21.15.97:0/3276253052 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3bb80bde90 con 0x7f3bd00d1c70 2023-11-24T06:11:18.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.691+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bd00b7c60 con 0x7f3bd00d1c70 2023-11-24T06:11:18.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.692+0000 7f3bc7fff700 1 -- 172.21.15.97:0/3276253052 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f3bb8015090 con 0x7f3bd00d1c70 2023-11-24T06:11:18.696 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:18.696 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":60,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:05.166928+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:01.950100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:02.526161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3bb006b2c0 msgr2=0x7f3bb006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3bb006b2c0 0x7f3bb006d780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000a890 tx=0x7f3bc0008040 comp rx=0 tx=0).stop 2023-11-24T06:11:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00d1c70 msgr2=0x7f3bd01e5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00d1c70 0x7f3bd01e5c40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009c10 tx=0x7f3bb8004750 comp rx=0 tx=0).stop 2023-11-24T06:11:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 shutdown_connections 2023-11-24T06:11:18.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bd015bdd0 0x7f3bd01e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3bb006b2c0 0x7f3bb006d780 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3bd00b3370 0x7f3bd00ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 --2- 172.21.15.97:0/3276253052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bd00d1c70 0x7f3bd01e5c40 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 >> 172.21.15.97:0/3276253052 conn(0x7f3bd00731a0 msgr2=0x7f3bd00b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:18.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 shutdown_connections 2023-11-24T06:11:18.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.694+0000 7f3bd5d62700 1 -- 172.21.15.97:0/3276253052 wait complete. 2023-11-24T06:11:18.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:01.950100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:02.526161+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:18.711 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:18.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.830+0000 7f9f7b20c700 1 Processor -- start 2023-11-24T06:11:18.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.830+0000 7f9f7b20c700 1 -- start start 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.830+0000 7f9f7b20c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f7415e160 0x7f9f740b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.830+0000 7f9f7b20c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f740746a0 0x7f9f7406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.830+0000 7f9f7b20c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f740b3c60 con 0x7f9f740afc90 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.830+0000 7f9f7b20c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740b4500 con 0x7f9f7415e160 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.830+0000 7f9f7b20c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740b58c0 con 0x7f9f740746a0 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.830+0000 7f9f78fa8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f7415e160 0x7f9f740b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f78fa8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f7415e160 0x7f9f740b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49502/0 (socket says 172.21.15.97:49502) 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f78fa8700 1 -- 172.21.15.97:0/712055935 learned_addr learned my addr 172.21.15.97:0/712055935 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f78fa8700 1 -- 172.21.15.97:0/712055935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f740746a0 msgr2=0x7f9f7406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f78fa8700 1 --2- 172.21.15.97:0/712055935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f740746a0 0x7f9f7406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f78fa8700 1 -- 172.21.15.97:0/712055935 >> v1:172.21.15.183:6789/0 conn(0x7f9f740afc90 legacy=0x7f9f74073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f78fa8700 1 -- 172.21.15.97:0/712055935 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f740b0140 con 0x7f9f7415e160 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f78fa8700 1 --2- 172.21.15.97:0/712055935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7415e160 0x7f9f740b5da0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9f680098f0 tx=0x7f9f68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdae2750f29c03b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f737fe700 1 -- 172.21.15.97:0/712055935 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68004230 con 0x7f9f7415e160 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f737fe700 1 -- 172.21.15.97:0/712055935 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f68004390 con 0x7f9f7415e160 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.831+0000 7f9f737fe700 1 -- 172.21.15.97:0/712055935 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680036a0 con 0x7f9f7415e160 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.832+0000 7f9f7b20c700 1 -- 172.21.15.97:0/712055935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7415e160 msgr2=0x7f9f740b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.832+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/712055935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7415e160 0x7f9f740b5da0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9f680098f0 tx=0x7f9f68009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.832+0000 7f9f7b20c700 1 -- 172.21.15.97:0/712055935 shutdown_connections 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.832+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/712055935 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f740746a0 0x7f9f7406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.832+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/712055935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f7415e160 0x7f9f740b5da0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.832+0000 7f9f7b20c700 1 -- 172.21.15.97:0/712055935 >> 172.21.15.97:0/712055935 conn(0x7f9f740d0bf0 msgr2=0x7f9f740d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:18.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.832+0000 7f9f7b20c700 1 -- 172.21.15.97:0/712055935 shutdown_connections 2023-11-24T06:11:18.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.832+0000 7f9f7b20c700 1 -- 172.21.15.97:0/712055935 wait complete. 2023-11-24T06:11:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.833+0000 7f9f7b20c700 1 Processor -- start 2023-11-24T06:11:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.833+0000 7f9f7b20c700 1 -- start start 2023-11-24T06:11:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f7b20c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f740746a0 0x7f9f741f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f7b20c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740afc90 0x7f9f74157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f7b20c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f7415e160 0x7f9f741581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f7b20c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7414f1c0 con 0x7f9f740afc90 2023-11-24T06:11:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f7b20c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7415e850 con 0x7f9f740746a0 2023-11-24T06:11:18.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f7b20c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f741f3a90 con 0x7f9f7415e160 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740afc90 0x7f9f74157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740afc90 0x7f9f74157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49508/0 (socket says 172.21.15.97:49508) 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 -- 172.21.15.97:0/3453338674 learned_addr learned my addr 172.21.15.97:0/3453338674 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f78fa8700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f740746a0 0x7f9f741f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f797a9700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f7415e160 0x7f9f741581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 -- 172.21.15.97:0/3453338674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f7415e160 msgr2=0x7f9f741581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f7415e160 0x7f9f741581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 -- 172.21.15.97:0/3453338674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f740746a0 msgr2=0x7f9f741f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f740746a0 0x7f9f741f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 -- 172.21.15.97:0/3453338674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f68009550 con 0x7f9f740afc90 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f73fff700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740afc90 0x7f9f74157c60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c4d0 tx=0x7f9f6000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f717fa700 1 -- 172.21.15.97:0/3453338674 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f600054e0 con 0x7f9f740afc90 2023-11-24T06:11:18.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f717fa700 1 -- 172.21.15.97:0/3453338674 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f60003680 con 0x7f9f740afc90 2023-11-24T06:11:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.834+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f74157010 con 0x7f9f740afc90 2023-11-24T06:11:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.835+0000 7f9f717fa700 1 -- 172.21.15.97:0/3453338674 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f60003a30 con 0x7f9f740afc90 2023-11-24T06:11:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.835+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f741575d0 con 0x7f9f740afc90 2023-11-24T06:11:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.835+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f74068fb0 con 0x7f9f740afc90 2023-11-24T06:11:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.836+0000 7f9f717fa700 1 -- 172.21.15.97:0/3453338674 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9f60003b90 con 0x7f9f740afc90 2023-11-24T06:11:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.836+0000 7f9f717fa700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f5806b2c0 0x7f9f5806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.836+0000 7f9f717fa700 1 -- 172.21.15.97:0/3453338674 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f60089240 con 0x7f9f740afc90 2023-11-24T06:11:18.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.836+0000 7f9f78fa8700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f5806b2c0 0x7f9f5806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:18.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.837+0000 7f9f78fa8700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f5806b2c0 0x7f9f5806d780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009130 tx=0x7f9f68008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:18.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:18.841+0000 7f9f717fa700 1 -- 172.21.15.97:0/3453338674 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f60053d10 con 0x7f9f740afc90 2023-11-24T06:11:19.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.029+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f7406c140 con 0x7f9f740afc90 2023-11-24T06:11:19.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.029+0000 7f9f717fa700 1 -- 172.21.15.97:0/3453338674 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f60017020 con 0x7f9f740afc90 2023-11-24T06:11:19.033 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f5806b2c0 msgr2=0x7f9f5806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f5806b2c0 0x7f9f5806d780 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009130 tx=0x7f9f68008030 comp rx=0 tx=0).stop 2023-11-24T06:11:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740afc90 msgr2=0x7f9f74157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740afc90 0x7f9f74157c60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000c4d0 tx=0x7f9f6000c890 comp rx=0 tx=0).stop 2023-11-24T06:11:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 shutdown_connections 2023-11-24T06:11:19.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9f5806b2c0 0x7f9f5806d780 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9f7415e160 0x7f9f741581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f740746a0 0x7f9f741f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 --2- 172.21.15.97:0/3453338674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f740afc90 0x7f9f74157c60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 >> 172.21.15.97:0/3453338674 conn(0x7f9f740d0bf0 msgr2=0x7f9f7406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 shutdown_connections 2023-11-24T06:11:19.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.031+0000 7f9f7b20c700 1 -- 172.21.15.97:0/3453338674 wait complete. 2023-11-24T06:11:19.047 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:19.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c7a60700 1 Processor -- start 2023-11-24T06:11:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c7a60700 1 -- start start 2023-11-24T06:11:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c7a60700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9c00b93c0 0x7fe9c006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c7a60700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 0x7fe9c006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c7a60700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c00b6fb0 con 0x7fe9c015e320 2023-11-24T06:11:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c7a60700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c00d4da0 con 0x7fe9c00b93c0 2023-11-24T06:11:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c7a60700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00d2c10 con 0x7fe9c00b3250 2023-11-24T06:11:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c4ffb700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9c00b93c0 0x7fe9c006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c4ffb700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9c00b93c0 0x7fe9c006aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49516/0 (socket says 172.21.15.97:49516) 2023-11-24T06:11:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c4ffb700 1 -- 172.21.15.97:0/3257137911 learned_addr learned my addr 172.21.15.97:0/3257137911 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c4ffb700 1 -- 172.21.15.97:0/3257137911 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 msgr2=0x7fe9c006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c4ffb700 1 --2- 172.21.15.97:0/3257137911 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 0x7fe9c006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c4ffb700 1 -- 172.21.15.97:0/3257137911 >> v1:172.21.15.97:6789/0 conn(0x7fe9c015e320 legacy=0x7fe9c00d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9b7fff700 1 -- 172.21.15.97:0/3257137911 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192061969 0 0) 0x7fe9c00b6fb0 con 0x7fe9c015e320 2023-11-24T06:11:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c4ffb700 1 -- 172.21.15.97:0/3257137911 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00d2ef0 con 0x7fe9c00b93c0 2023-11-24T06:11:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9c4ffb700 1 --2- 172.21.15.97:0/3257137911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c006aff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00098f0 tx=0x7fe9b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9791e5c178ea608 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9b7fff700 1 -- 172.21.15.97:0/3257137911 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000b8a0 con 0x7fe9c00b93c0 2023-11-24T06:11:19.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9b7fff700 1 -- 172.21.15.97:0/3257137911 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b000ba00 con 0x7fe9c00b93c0 2023-11-24T06:11:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.167+0000 7fe9b7fff700 1 -- 172.21.15.97:0/3257137911 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000bd90 con 0x7fe9c00b93c0 2023-11-24T06:11:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.168+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3257137911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 msgr2=0x7fe9c006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.168+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3257137911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c006aff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00098f0 tx=0x7fe9b0009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.168+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3257137911 shutdown_connections 2023-11-24T06:11:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.168+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3257137911 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 0x7fe9c006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.168+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3257137911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c006aff0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.168+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3257137911 >> 172.21.15.97:0/3257137911 conn(0x7fe9c00d3e50 msgr2=0x7fe9c00d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.168+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3257137911 shutdown_connections 2023-11-24T06:11:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.168+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3257137911 wait complete. 2023-11-24T06:11:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.169+0000 7fe9c7a60700 1 Processor -- start 2023-11-24T06:11:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.169+0000 7fe9c7a60700 1 -- start start 2023-11-24T06:11:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c7a60700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 0x7fe9c015f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c7a60700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c015ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c7a60700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c015e320 0x7fe9c00c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c7a60700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0160730 con 0x7fe9c00b93c0 2023-11-24T06:11:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c7a60700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0160a10 con 0x7fe9c015e320 2023-11-24T06:11:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c7a60700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c015ed50 con 0x7fe9c00b3250 2023-11-24T06:11:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c4ffb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c015ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c4ffb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c015ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49522/0 (socket says 172.21.15.97:49522) 2023-11-24T06:11:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c4ffb700 1 -- 172.21.15.97:0/3023649460 learned_addr learned my addr 172.21.15.97:0/3023649460 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c57fc700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 0x7fe9c015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c5ffd700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c015e320 0x7fe9c00c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c4ffb700 1 -- 172.21.15.97:0/3023649460 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 msgr2=0x7fe9c015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c4ffb700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 0x7fe9c015f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c4ffb700 1 -- 172.21.15.97:0/3023649460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c015e320 msgr2=0x7fe9c00c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c4ffb700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c015e320 0x7fe9c00c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.170+0000 7fe9c4ffb700 1 -- 172.21.15.97:0/3023649460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b0009550 con 0x7fe9c00b93c0 2023-11-24T06:11:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.171+0000 7fe9c4ffb700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c015ce70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0000c00 tx=0x7fe9b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.171+0000 7fe9b5ffb700 1 -- 172.21.15.97:0/3023649460 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00045d0 con 0x7fe9c00b93c0 2023-11-24T06:11:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.171+0000 7fe9b5ffb700 1 -- 172.21.15.97:0/3023649460 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b001f070 con 0x7fe9c00b93c0 2023-11-24T06:11:19.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.171+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c015f030 con 0x7fe9c00b93c0 2023-11-24T06:11:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.171+0000 7fe9b5ffb700 1 -- 172.21.15.97:0/3023649460 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00038a0 con 0x7fe9c00b93c0 2023-11-24T06:11:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.171+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c015d3b0 con 0x7fe9c00b93c0 2023-11-24T06:11:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.172+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c0068fb0 con 0x7fe9c00b93c0 2023-11-24T06:11:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.172+0000 7fe9b5ffb700 1 -- 172.21.15.97:0/3023649460 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe9b0003a00 con 0x7fe9c00b93c0 2023-11-24T06:11:19.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.173+0000 7fe9b5ffb700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9a406b2c0 0x7fe9a406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.173+0000 7fe9b5ffb700 1 -- 172.21.15.97:0/3023649460 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9b0021070 con 0x7fe9c00b93c0 2023-11-24T06:11:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.173+0000 7fe9c57fc700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9a406b2c0 0x7fe9a406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.173+0000 7fe9c57fc700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9a406b2c0 0x7fe9a406d780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc000f60 tx=0x7fe9bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.178+0000 7fe9b5ffb700 1 -- 172.21.15.97:0/3023649460 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9b0056200 con 0x7fe9c00b93c0 2023-11-24T06:11:19.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.375+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9c00b6c80 con 0x7fe9c00b93c0 2023-11-24T06:11:19.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.375+0000 7fe9b5ffb700 1 -- 172.21.15.97:0/3023649460 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9b0059820 con 0x7fe9c00b93c0 2023-11-24T06:11:19.379 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:11:19.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9a406b2c0 msgr2=0x7fe9a406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9a406b2c0 0x7fe9a406d780 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc000f60 tx=0x7fe9bc009040 comp rx=0 tx=0).stop 2023-11-24T06:11:19.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 msgr2=0x7fe9c015ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c015ce70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0000c00 tx=0x7fe9b0004750 comp rx=0 tx=0).stop 2023-11-24T06:11:19.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 shutdown_connections 2023-11-24T06:11:19.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c015e320 0x7fe9c00c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9a406b2c0 0x7fe9a406d780 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9c00b3250 0x7fe9c015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 --2- 172.21.15.97:0/3023649460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9c00b93c0 0x7fe9c015ce70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 >> 172.21.15.97:0/3023649460 conn(0x7fe9c00d3e50 msgr2=0x7fe9c00d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 shutdown_connections 2023-11-24T06:11:19.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.377+0000 7fe9c7a60700 1 -- 172.21.15.97:0/3023649460 wait complete. 2023-11-24T06:11:19.393 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.514+0000 7fa3591d9700 1 Processor -- start 2023-11-24T06:11:19.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.514+0000 7fa3591d9700 1 -- start start 2023-11-24T06:11:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.514+0000 7fa3591d9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa3540b5da0 0x7fa3540afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.514+0000 7fa3591d9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3540b3c60 con 0x7fa35415e160 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.514+0000 7fa3591d9700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3540b58c0 con 0x7fa3540746a0 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.514+0000 7fa3591d9700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3540b4500 con 0x7fa3540b5da0 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.514+0000 7fa352d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa35415e160 0x7fa354073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:41980/0 (socket says 172.21.15.97:41980) 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.514+0000 7fa352d9d700 1 -- 172.21.15.97:0/568407151 learned_addr learned my addr 172.21.15.97:0/568407151 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050432936 0 0) 0x7fa3540b3c60 con 0x7fa35415e160 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa33c0035b0 con 0x7fa35415e160 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 130672784 0 0) 0x7fa33c0035b0 con 0x7fa35415e160 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 >> v2:172.21.15.183:3300/0 conn(0x7fa3540b5da0 msgr2=0x7fa3540afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 --2- 172.21.15.97:0/568407151 >> v2:172.21.15.183:3300/0 conn(0x7fa3540b5da0 0x7fa3540afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 >> v1:172.21.15.183:6789/0 conn(0x7fa3540746a0 legacy=0x7fa35406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35406bdc0 con 0x7fa35415e160 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fa348001670 con 0x7fa35415e160 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 >> v1:172.21.15.97:6789/0 conn(0x7fa35415e160 legacy=0x7fa354073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa33c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa33c004850 0x7fa33c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa33c005330 0x7fa33c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540b4500 con 0x7fa33c003a00 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c00c230 con 0x7fa33c004850 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c00c3f0 con 0x7fa33c005330 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa352d9d700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa33c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa35359e700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa33c005330 0x7fa33c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa352d9d700 1 -- 172.21.15.97:0/568407151 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa33c005330 msgr2=0x7fa33c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa352d9d700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa33c005330 0x7fa33c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa352d9d700 1 -- 172.21.15.97:0/568407151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa33c004850 msgr2=0x7fa33c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa352d9d700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa33c004850 0x7fa33c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa352d9d700 1 -- 172.21.15.97:0/568407151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c0035b0 con 0x7fa33c003a00 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa35359e700 1 -- 172.21.15.97:0/568407151 reap_dead start 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.515+0000 7fa352d9d700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa33c003e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c210 tx=0x7fa33c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f77bf88dfa1e120 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34800cd50 con 0x7fa33c003a00 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa348005960 con 0x7fa33c003a00 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa351d9b700 1 -- 172.21.15.97:0/568407151 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348005cf0 con 0x7fa33c003a00 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa3591d9700 1 -- 172.21.15.97:0/568407151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 msgr2=0x7fa33c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa3591d9700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa33c003e00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c210 tx=0x7fa33c005880 comp rx=0 tx=0).stop 2023-11-24T06:11:19.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa3591d9700 1 -- 172.21.15.97:0/568407151 shutdown_connections 2023-11-24T06:11:19.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa3591d9700 1 --2- 172.21.15.97:0/568407151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa33c003e00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa3591d9700 1 -- 172.21.15.97:0/568407151 >> 172.21.15.97:0/568407151 conn(0x7fa3540d0bf0 msgr2=0x7fa3540d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa3591d9700 1 -- 172.21.15.97:0/568407151 shutdown_connections 2023-11-24T06:11:19.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.516+0000 7fa3591d9700 1 -- 172.21.15.97:0/568407151 wait complete. 2023-11-24T06:11:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa3591d9700 1 Processor -- start 2023-11-24T06:11:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa3591d9700 1 -- start start 2023-11-24T06:11:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa3591d9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa354158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa3591d9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa354155b20 0x7fa35415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa3591d9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa354157220 0x7fa35415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa3591d9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa354155fb0 con 0x7fa33c003a00 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa3591d9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3541606a0 con 0x7fa354155b20 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa3591d9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa354160980 con 0x7fa354157220 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa352d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa354158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.518+0000 7fa352d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa354158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49540/0 (socket says 172.21.15.97:49540) 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa352d9d700 1 -- 172.21.15.97:0/4108700323 learned_addr learned my addr 172.21.15.97:0/4108700323 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa35259c700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa354155b20 0x7fa35415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa352d9d700 1 -- 172.21.15.97:0/4108700323 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa354157220 msgr2=0x7fa35415a630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa352d9d700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa354157220 0x7fa35415a630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa352d9d700 1 -- 172.21.15.97:0/4108700323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa354155b20 msgr2=0x7fa35415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa352d9d700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa354155b20 0x7fa35415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa352d9d700 1 -- 172.21.15.97:0/4108700323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354160c60 con 0x7fa33c003a00 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa352d9d700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa354158950 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c200 tx=0x7fa35406c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa33b7fe700 1 -- 172.21.15.97:0/4108700323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348004c80 con 0x7fa33c003a00 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa33b7fe700 1 -- 172.21.15.97:0/4108700323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa348004de0 con 0x7fa33c003a00 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa354160f40 con 0x7fa33c003a00 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa33b7fe700 1 -- 172.21.15.97:0/4108700323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3480072f0 con 0x7fa33c003a00 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.519+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa354147800 con 0x7fa33c003a00 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.520+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa354068fb0 con 0x7fa33c003a00 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.521+0000 7fa33b7fe700 1 -- 172.21.15.97:0/4108700323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa348007450 con 0x7fa33c003a00 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.521+0000 7fa33b7fe700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa34006b2f0 0x7fa34006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.521+0000 7fa33b7fe700 1 -- 172.21.15.97:0/4108700323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa34808aa10 con 0x7fa33c003a00 2023-11-24T06:11:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.521+0000 7fa35259c700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa34006b2f0 0x7fa34006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.522+0000 7fa35259c700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa34006b2f0 0x7fa34006d7b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa3540b7100 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.526+0000 7fa33b7fe700 1 -- 172.21.15.97:0/4108700323 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa348055670 con 0x7fa33c003a00 2023-11-24T06:11:19.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.718+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa354161180 con 0x7fa33c003a00 2023-11-24T06:11:19.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.718+0000 7fa33b7fe700 1 -- 172.21.15.97:0/4108700323 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa354161180 con 0x7fa33c003a00 2023-11-24T06:11:19.722 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa34006b2f0 msgr2=0x7fa34006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa34006b2f0 0x7fa34006d7b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa3540b7100 tx=0x7fa344009040 comp rx=0 tx=0).stop 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 msgr2=0x7fa354158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa354158950 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c200 tx=0x7fa35406c0a0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 shutdown_connections 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa34006b2f0 0x7fa34006d7b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa354157220 0x7fa35415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa33c003a00 0x7fa354158950 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 --2- 172.21.15.97:0/4108700323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa354155b20 0x7fa35415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 >> 172.21.15.97:0/4108700323 conn(0x7fa3540d0bf0 msgr2=0x7fa3540d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.720+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 shutdown_connections 2023-11-24T06:11:19.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.721+0000 7fa3591d9700 1 -- 172.21.15.97:0/4108700323 wait complete. 2023-11-24T06:11:19.738 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:19.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.856+0000 7f9509356700 1 Processor -- start 2023-11-24T06:11:19.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.856+0000 7f9509356700 1 -- start start 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.856+0000 7f9509356700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f950415e080 0x7f95040b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.856+0000 7f9509356700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95040af4d0 0x7f95040afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.856+0000 7f9509356700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f950406c000 0x7f950406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.856+0000 7f9509356700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95040af0b0 con 0x7f950406c000 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.856+0000 7f9509356700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950406c9f0 con 0x7f950415e080 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.856+0000 7f9509356700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95040b05a0 con 0x7f95040af4d0 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f950406c000 0x7f950406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f950406c000 0x7f950406c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49550/0 (socket says 172.21.15.97:49550) 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 -- 172.21.15.97:0/4112422202 learned_addr learned my addr 172.21.15.97:0/4112422202 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:19.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f9502ffd700 1 --2- 172.21.15.97:0/4112422202 >> v2:172.21.15.183:3300/0 conn(0x7f950415e080 0x7f95040b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95027fc700 1 --2- 172.21.15.97:0/4112422202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95040af4d0 0x7f95040afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 -- 172.21.15.97:0/4112422202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95040af4d0 msgr2=0x7f95040afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 --2- 172.21.15.97:0/4112422202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95040af4d0 0x7f95040afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 -- 172.21.15.97:0/4112422202 >> v2:172.21.15.183:3300/0 conn(0x7f950415e080 msgr2=0x7f95040b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 --2- 172.21.15.97:0/4112422202 >> v2:172.21.15.183:3300/0 conn(0x7f950415e080 0x7f95040b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 -- 172.21.15.97:0/4112422202 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95040ae430 con 0x7f950406c000 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f9502ffd700 1 --2- 172.21.15.97:0/4112422202 >> v2:172.21.15.183:3300/0 conn(0x7f950415e080 0x7f95040b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f95037fe700 1 --2- 172.21.15.97:0/4112422202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f950406c000 0x7f950406c4b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9504072a90 tx=0x7f94fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5eab44bb03669 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f9501ffb700 1 -- 172.21.15.97:0/4112422202 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc00e070 con 0x7f950406c000 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f9501ffb700 1 -- 172.21.15.97:0/4112422202 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc00b980 con 0x7f950406c000 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.857+0000 7f9501ffb700 1 -- 172.21.15.97:0/4112422202 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc00bd30 con 0x7f950406c000 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 -- 172.21.15.97:0/4112422202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f950406c000 msgr2=0x7f950406c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 --2- 172.21.15.97:0/4112422202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f950406c000 0x7f950406c4b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9504072a90 tx=0x7f94fc00b330 comp rx=0 tx=0).stop 2023-11-24T06:11:19.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 -- 172.21.15.97:0/4112422202 shutdown_connections 2023-11-24T06:11:19.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 --2- 172.21.15.97:0/4112422202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f950406c000 0x7f950406c4b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 --2- 172.21.15.97:0/4112422202 >> v2:172.21.15.183:3300/0 conn(0x7f950415e080 0x7f95040b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 --2- 172.21.15.97:0/4112422202 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95040af4d0 0x7f95040afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 -- 172.21.15.97:0/4112422202 >> 172.21.15.97:0/4112422202 conn(0x7f9504071b90 msgr2=0x7f9504071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:19.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 -- 172.21.15.97:0/4112422202 shutdown_connections 2023-11-24T06:11:19.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.858+0000 7f9509356700 1 -- 172.21.15.97:0/4112422202 wait complete. 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.859+0000 7f9509356700 1 Processor -- start 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.859+0000 7f9509356700 1 -- start start 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f9509356700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950406c000 0x7f9504159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f9509356700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95040af4d0 0x7f950414a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f9509356700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f950415e080 0x7f950414a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f9509356700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95040cdb10 con 0x7f95040af4d0 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f9509356700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950414aa90 con 0x7f950406c000 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f9509356700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9504154590 con 0x7f950415e080 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95027fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95040af4d0 0x7f950414a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95027fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95040af4d0 0x7f950414a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49556/0 (socket says 172.21.15.97:49556) 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95027fc700 1 -- 172.21.15.97:0/3554044130 learned_addr learned my addr 172.21.15.97:0/3554044130 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f9502ffd700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950406c000 0x7f9504159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95037fe700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f950415e080 0x7f950414a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95027fc700 1 -- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f950415e080 msgr2=0x7f950414a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95027fc700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f950415e080 0x7f950414a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95027fc700 1 -- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950406c000 msgr2=0x7f9504159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95027fc700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950406c000 0x7f9504159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95027fc700 1 -- 172.21.15.97:0/3554044130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc009b30 con 0x7f95040af4d0 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.860+0000 7f95037fe700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f950415e080 0x7f950414a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.861+0000 7f95027fc700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95040af4d0 0x7f950414a010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f94f800c7f0 tx=0x7f94f800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.861+0000 7f94eb7fe700 1 -- 172.21.15.97:0/3554044130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8004230 con 0x7f95040af4d0 2023-11-24T06:11:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.861+0000 7f94eb7fe700 1 -- 172.21.15.97:0/3554044130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f8004390 con 0x7f95040af4d0 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.861+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9504158ca0 con 0x7f95040af4d0 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.861+0000 7f94eb7fe700 1 -- 172.21.15.97:0/3554044130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f80036a0 con 0x7f95040af4d0 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.861+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9504158790 con 0x7f95040af4d0 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.862+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95040753f0 con 0x7f95040af4d0 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.862+0000 7f94eb7fe700 1 -- 172.21.15.97:0/3554044130 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f94f8003820 con 0x7f95040af4d0 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.863+0000 7f94eb7fe700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94f006b2c0 0x7f94f006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.863+0000 7f9502ffd700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94f006b2c0 0x7f94f006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.863+0000 7f94eb7fe700 1 -- 172.21.15.97:0/3554044130 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f94f8089b70 con 0x7f95040af4d0 2023-11-24T06:11:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.863+0000 7f9502ffd700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94f006b2c0 0x7f94f006d780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f95040b0530 tx=0x7f94f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:19.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:19.868+0000 7f94eb7fe700 1 -- 172.21.15.97:0/3554044130 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94f8054500 con 0x7f95040af4d0 2023-11-24T06:11:20.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.075+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f95040cdc50 con 0x7f95040af4d0 2023-11-24T06:11:20.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.075+0000 7f94eb7fe700 1 -- 172.21.15.97:0/3554044130 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7f95040cdc50 con 0x7f95040af4d0 2023-11-24T06:11:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94f006b2c0 msgr2=0x7f94f006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94f006b2c0 0x7f94f006d780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f95040b0530 tx=0x7f94f4008040 comp rx=0 tx=0).stop 2023-11-24T06:11:20.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95040af4d0 msgr2=0x7f950414a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95040af4d0 0x7f950414a010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f94f800c7f0 tx=0x7f94f800cb00 comp rx=0 tx=0).stop 2023-11-24T06:11:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 shutdown_connections 2023-11-24T06:11:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f94f006b2c0 0x7f94f006d780 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f950415e080 0x7f950414a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f950406c000 0x7f9504159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 --2- 172.21.15.97:0/3554044130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95040af4d0 0x7f950414a010 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 >> 172.21.15.97:0/3554044130 conn(0x7f9504071b90 msgr2=0x7f950415d270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 shutdown_connections 2023-11-24T06:11:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.076+0000 7f9509356700 1 -- 172.21.15.97:0/3554044130 wait complete. 2023-11-24T06:11:20.082 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:11:20.092 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:11:20.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:20.092 DEBUG:teuthology.orchestra.run.smithi183:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:20.140 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:11:20.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:20.141 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:11:20.160 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:20.160 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:20.178 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:11:20.178 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:20.178 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:20.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:11:20.220 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.338+0000 7f6d85db6700 1 Processor -- start 2023-11-24T06:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.338+0000 7f6d85db6700 1 -- start start 2023-11-24T06:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d85db6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015ddd0 0x7f6d800b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d85db6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d800d2240 0x7f6d800cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d85db6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d800b1d40 con 0x7f6d8015e1b0 2023-11-24T06:11:20.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d85db6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d800b28c0 con 0x7f6d800d2240 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d85db6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8006c000 con 0x7f6d8015ddd0 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d7ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d800d2240 0x7f6d800cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d7effd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6d8015e1b0 0x7f6d800d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42014/0 (socket says 172.21.15.97:42014) 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d7ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d800d2240 0x7f6d800cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49566/0 (socket says 172.21.15.97:49566) 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.339+0000 7f6d7effd700 1 -- 172.21.15.97:0/536570046 learned_addr learned my addr 172.21.15.97:0/536570046 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7ffff700 1 -- 172.21.15.97:0/536570046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015ddd0 msgr2=0x7f6d800b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7ffff700 1 --2- 172.21.15.97:0/536570046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015ddd0 0x7f6d800b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7ffff700 1 -- 172.21.15.97:0/536570046 >> v1:172.21.15.97:6789/0 conn(0x7f6d8015e1b0 legacy=0x7f6d800d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7ffff700 1 -- 172.21.15.97:0/536570046 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800b1f90 con 0x7f6d800d2240 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7e7fc700 1 -- 172.21.15.97:0/536570046 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093704788 0 0) 0x7f6d800b1d40 con 0x7f6d8015e1b0 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7ffff700 1 --2- 172.21.15.97:0/536570046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d800d2240 0x7f6d800cf000 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009d90 tx=0x7f6d7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a46e548bfd4b9d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7e7fc700 1 -- 172.21.15.97:0/536570046 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7800e070 con 0x7f6d800d2240 2023-11-24T06:11:20.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7e7fc700 1 -- 172.21.15.97:0/536570046 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7800bbd0 con 0x7f6d800d2240 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.340+0000 7f6d7e7fc700 1 -- 172.21.15.97:0/536570046 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78012430 con 0x7f6d800d2240 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.341+0000 7f6d85db6700 1 -- 172.21.15.97:0/536570046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d800d2240 msgr2=0x7f6d800cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.341+0000 7f6d85db6700 1 --2- 172.21.15.97:0/536570046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d800d2240 0x7f6d800cf000 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009d90 tx=0x7f6d7800b330 comp rx=0 tx=0).stop 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.341+0000 7f6d85db6700 1 -- 172.21.15.97:0/536570046 shutdown_connections 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.341+0000 7f6d85db6700 1 --2- 172.21.15.97:0/536570046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d800d2240 0x7f6d800cf000 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.341+0000 7f6d85db6700 1 --2- 172.21.15.97:0/536570046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015ddd0 0x7f6d800b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.341+0000 7f6d85db6700 1 -- 172.21.15.97:0/536570046 >> 172.21.15.97:0/536570046 conn(0x7f6d800731a0 msgr2=0x7f6d8009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.341+0000 7f6d85db6700 1 -- 172.21.15.97:0/536570046 shutdown_connections 2023-11-24T06:11:20.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.341+0000 7f6d85db6700 1 -- 172.21.15.97:0/536570046 wait complete. 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.342+0000 7f6d85db6700 1 Processor -- start 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.342+0000 7f6d85db6700 1 -- start start 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d85db6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d800d2240 0x7f6d8015f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d85db6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8015ddd0 0x7f6d8014b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d85db6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015e1b0 0x7f6d8014b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d85db6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8014bd70 con 0x7f6d8015ddd0 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d85db6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8009d580 con 0x7f6d800d2240 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d85db6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8009d860 con 0x7f6d8015e1b0 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8015ddd0 0x7f6d8014b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8015ddd0 0x7f6d8014b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49570/0 (socket says 172.21.15.97:49570) 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 -- 172.21.15.97:0/3575913780 learned_addr learned my addr 172.21.15.97:0/3575913780 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7ffff700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015e1b0 0x7f6d8014b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 -- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015e1b0 msgr2=0x7f6d8014b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015e1b0 0x7f6d8014b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 -- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d800d2240 msgr2=0x7f6d8015f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7f7fe700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d800d2240 0x7f6d8015f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d800d2240 0x7f6d8015f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 -- 172.21.15.97:0/3575913780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009a70 con 0x7f6d8015ddd0 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7ffff700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015e1b0 0x7f6d8014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7f7fe700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d800d2240 0x7f6d8015f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d7effd700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8015ddd0 0x7f6d8014b000 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6d700011e0 tx=0x7f6d7000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.350 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d67fff700 1 -- 172.21.15.97:0/3575913780 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70004c80 con 0x7f6d8015ddd0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d67fff700 1 -- 172.21.15.97:0/3575913780 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d70004e00 con 0x7f6d8015ddd0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d800b2ed0 con 0x7f6d8015ddd0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.343+0000 7f6d67fff700 1 -- 172.21.15.97:0/3575913780 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d70007310 con 0x7f6d8015ddd0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.344+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8014ba80 con 0x7f6d8015ddd0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.345+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8009dc60 con 0x7f6d8015ddd0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.345+0000 7f6d67fff700 1 -- 172.21.15.97:0/3575913780 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6d70007470 con 0x7f6d8015ddd0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.345+0000 7f6d67fff700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6d6c06b2c0 0x7f6d6c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.345+0000 7f6d67fff700 1 -- 172.21.15.97:0/3575913780 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6d70089a70 con 0x7f6d8015ddd0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.345+0000 7f6d7f7fe700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6d6c06b2c0 0x7f6d6c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.346+0000 7f6d7f7fe700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6d6c06b2c0 0x7f6d6c06d780 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d74007900 tx=0x7f6d74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.350+0000 7f6d67fff700 1 -- 172.21.15.97:0/3575913780 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d70054610 con 0x7f6d8015ddd0 2023-11-24T06:11:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.581+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d800b2d30 con 0x7f6d8015ddd0 2023-11-24T06:11:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.582+0000 7f6d67fff700 1 -- 172.21.15.97:0/3575913780 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f6d800b2d30 con 0x7f6d8015ddd0 2023-11-24T06:11:20.585 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:20.586 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3508641674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3508641674},{"type":"v1","addr":"172.21.15.183:6833","nonce":3508641674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5557,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/116508824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":116508824},{"type":"v1","addr":"172.21.15.183:6835","nonce":116508824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2785455707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2785455707},{"type":"v1","addr":"172.21.15.97:6837","nonce":2785455707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:03.572274+0000","modified":"2023-11-24T06:11:06.595718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3662495999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3662495999},{"type":"v1","addr":"172.21.15.97:6836","nonce":3662495999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:11:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6d6c06b2c0 msgr2=0x7f6d6c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6d6c06b2c0 0x7f6d6c06d780 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6d74007900 tx=0x7f6d74008040 comp rx=0 tx=0).stop 2023-11-24T06:11:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8015ddd0 msgr2=0x7f6d8014b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8015ddd0 0x7f6d8014b000 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f6d700011e0 tx=0x7f6d7000cae0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 shutdown_connections 2023-11-24T06:11:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6d6c06b2c0 0x7f6d6c06d780 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6d8015e1b0 0x7f6d8014b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d800d2240 0x7f6d8015f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 --2- 172.21.15.97:0/3575913780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d8015ddd0 0x7f6d8014b000 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 >> 172.21.15.97:0/3575913780 conn(0x7f6d800731a0 msgr2=0x7f6d800cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 shutdown_connections 2023-11-24T06:11:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.584+0000 7f6d85db6700 1 -- 172.21.15.97:0/3575913780 wait complete. 2023-11-24T06:11:20.589 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 37 2023-11-24T06:11:20.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:20.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.716+0000 7f4eec199700 1 Processor -- start 2023-11-24T06:11:20.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.716+0000 7f4eec199700 1 -- start start 2023-11-24T06:11:20.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4eec199700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee415e080 0x7f4ee40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4eec199700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee40ae430 con 0x7f4ee40af4d0 2023-11-24T06:11:20.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4eec199700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee40ae710 con 0x7f4ee406c710 2023-11-24T06:11:20.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4eec199700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40af0b0 con 0x7f4ee415e080 2023-11-24T06:11:20.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4ee9734700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4ee40af4d0 0x7f4ee406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42028/0 (socket says 172.21.15.97:42028) 2023-11-24T06:11:20.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4ee9734700 1 -- 172.21.15.97:0/4219394643 learned_addr learned my addr 172.21.15.97:0/4219394643 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874360955 0 0) 0x7f4ee40ae430 con 0x7f4ee40af4d0 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4ee9f35700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee415e080 0x7f4ee40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.717+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed00035b0 con 0x7f4ee40af4d0 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119460437 0 0) 0x7f4ed00035b0 con 0x7f4ee40af4d0 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee415e080 msgr2=0x7f4ee40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee415e080 0x7f4ee40b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 >> v1:172.21.15.183:6789/0 conn(0x7f4ee406c710 legacy=0x7f4ee406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee40ae9f0 con 0x7f4ee40af4d0 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f4ee0001670 con 0x7f4ee40af4d0 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 >> v1:172.21.15.97:6789/0 conn(0x7f4ee40af4d0 legacy=0x7f4ee406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee415e080 0x7f4ed0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ed0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ed0004d90 0x7f4ed0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee40af0b0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed000baf0 con 0x7f4ed0004d90 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed000bcb0 con 0x7f4ee415e080 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee9734700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ed0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee9f35700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee415e080 0x7f4ed0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4eea736700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ed0004d90 0x7f4ed0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee9734700 1 -- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee415e080 msgr2=0x7f4ed0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee9734700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee415e080 0x7f4ed0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee9734700 1 -- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ed0004d90 msgr2=0x7f4ed0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee9734700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ed0004d90 0x7f4ed0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee9734700 1 -- 172.21.15.97:0/4219394643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00035b0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4eea736700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ed0004d90 0x7f4ed0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee9734700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ed0004850 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000c210 tx=0x7f4ed0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cc459376a8f942 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0007c10 con 0x7f4ed0003fb0 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee000cdd0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.718+0000 7f4ee8f33700 1 -- 172.21.15.97:0/4219394643 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0005ba0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.719+0000 7f4eec199700 1 -- 172.21.15.97:0/4219394643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 msgr2=0x7f4ed0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.719+0000 7f4eec199700 1 --2- 172.21.15.97:0/4219394643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ed0004850 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000c210 tx=0x7f4ed0005880 comp rx=0 tx=0).stop 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.719+0000 7f4eea736700 1 -- 172.21.15.97:0/4219394643 reap_dead start 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.719+0000 7f4eec199700 1 -- 172.21.15.97:0/4219394643 shutdown_connections 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.719+0000 7f4eec199700 1 -- 172.21.15.97:0/4219394643 >> 172.21.15.97:0/4219394643 conn(0x7f4ee4071b90 msgr2=0x7f4ee4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.719+0000 7f4eec199700 1 -- 172.21.15.97:0/4219394643 shutdown_connections 2023-11-24T06:11:20.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.719+0000 7f4eec199700 1 -- 172.21.15.97:0/4219394643 wait complete. 2023-11-24T06:11:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.720+0000 7f4eec199700 1 Processor -- start 2023-11-24T06:11:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.720+0000 7f4eec199700 1 -- start start 2023-11-24T06:11:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4eec199700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ee41566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4eec199700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee41f4ab0 0x7f4ee415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4eec199700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee41f4f30 0x7f4ee415f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4eec199700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee41f48e0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4eec199700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee41588f0 con 0x7f4ee41f4f30 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4eec199700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee4158bd0 con 0x7f4ee41f4ab0 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9f35700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ee41566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9f35700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ee41566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49588/0 (socket says 172.21.15.97:49588) 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9f35700 1 -- 172.21.15.97:0/2916137566 learned_addr learned my addr 172.21.15.97:0/2916137566 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9734700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee41f4ab0 0x7f4ee415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4eea736700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee41f4f30 0x7f4ee415f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9f35700 1 -- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee41f4ab0 msgr2=0x7f4ee415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9f35700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee41f4ab0 0x7f4ee415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9f35700 1 -- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee41f4f30 msgr2=0x7f4ee415f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9f35700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee41f4f30 0x7f4ee415f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4ee9f35700 1 -- 172.21.15.97:0/2916137566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4158eb0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.721+0000 7f4eea736700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee41f4f30 0x7f4ee415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.722+0000 7f4ee9f35700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ee41566e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4011aa0 tx=0x7f4ed4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.722+0000 7f4eda7fc700 1 -- 172.21.15.97:0/2916137566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400f940 con 0x7f4ed0003fb0 2023-11-24T06:11:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.722+0000 7f4eda7fc700 1 -- 172.21.15.97:0/2916137566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed400faa0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.722+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee4159190 con 0x7f4ed0003fb0 2023-11-24T06:11:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.722+0000 7f4eda7fc700 1 -- 172.21.15.97:0/2916137566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400fe50 con 0x7f4ed0003fb0 2023-11-24T06:11:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.722+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee41596b0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.723+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee40753f0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.723+0000 7f4eda7fc700 1 -- 172.21.15.97:0/2916137566 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f4ed40273f0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.724+0000 7f4eda7fc700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ec806b2c0 0x7f4ec806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.724+0000 7f4eda7fc700 1 -- 172.21.15.97:0/2916137566 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4ed408a6b0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.724+0000 7f4ee9734700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ec806b2c0 0x7f4ec806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.724+0000 7f4ee9734700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ec806b2c0 0x7f4ec806d780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000c00 tx=0x7f4ee0007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:20.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.728+0000 7f4eda7fc700 1 -- 172.21.15.97:0/2916137566 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ed4055390 con 0x7f4ed0003fb0 2023-11-24T06:11:20.923 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.920+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ee41593d0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.921+0000 7f4eda7fc700 1 -- 172.21.15.97:0/2916137566 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16685 (secure 0 0 0) 0x7f4ed40589b0 con 0x7f4ed0003fb0 2023-11-24T06:11:20.924 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:20.924 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":61,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:19.547545+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:01.950100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:02.526161+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:20.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ec806b2c0 msgr2=0x7f4ec806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ec806b2c0 0x7f4ec806d780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000c00 tx=0x7f4ee0007000 comp rx=0 tx=0).stop 2023-11-24T06:11:20.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 msgr2=0x7f4ee41566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ee41566e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4011aa0 tx=0x7f4ed4011db0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 shutdown_connections 2023-11-24T06:11:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ee41f4f30 0x7f4ee415f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed0003fb0 0x7f4ee41566e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4ec806b2c0 0x7f4ec806d780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 --2- 172.21.15.97:0/2916137566 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4ee41f4ab0 0x7f4ee415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 >> 172.21.15.97:0/2916137566 conn(0x7f4ee4071b90 msgr2=0x7f4ee40af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:20.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 shutdown_connections 2023-11-24T06:11:20.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:20.923+0000 7f4eec199700 1 -- 172.21.15.97:0/2916137566 wait complete. 2023-11-24T06:11:20.938 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:01.950100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:02.526161+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:20.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.059+0000 7fbff02c9700 1 Processor -- start 2023-11-24T06:11:21.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.059+0000 7fbff02c9700 1 -- start start 2023-11-24T06:11:21.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.060+0000 7fbff02c9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfe80cc540 0x7fbfe806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.060+0000 7fbff02c9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe806eb20 0x7fbfe80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.060+0000 7fbff02c9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe80b2f20 con 0x7fbfe815a620 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.060+0000 7fbff02c9700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe806ef90 con 0x7fbfe80cc540 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.060+0000 7fbff02c9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe815aa00 con 0x7fbfe806eb20 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.060+0000 7fbfed864700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfe80cc540 0x7fbfe806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.060+0000 7fbfed864700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbfe80cc540 0x7fbfe806e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49596/0 (socket says 172.21.15.97:49596) 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.060+0000 7fbfed864700 1 -- 172.21.15.97:0/1028456601 learned_addr learned my addr 172.21.15.97:0/1028456601 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfee866700 1 --2- 172.21.15.97:0/1028456601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe806eb20 0x7fbfe80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed864700 1 -- 172.21.15.97:0/1028456601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe806eb20 msgr2=0x7fbfe80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed864700 1 --2- 172.21.15.97:0/1028456601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe806eb20 0x7fbfe80b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed864700 1 -- 172.21.15.97:0/1028456601 >> v1:172.21.15.97:6789/0 conn(0x7fbfe815a620 legacy=0x7fbfe80ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed063700 1 -- 172.21.15.97:0/1028456601 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166250319 0 0) 0x7fbfe80b2f20 con 0x7fbfe815a620 2023-11-24T06:11:21.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed864700 1 -- 172.21.15.97:0/1028456601 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80ce960 con 0x7fbfe80cc540 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed864700 1 --2- 172.21.15.97:0/1028456601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe80cc540 0x7fbfe806e5e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400bb90 tx=0x7fbfe400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3c1d63366d597 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed063700 1 -- 172.21.15.97:0/1028456601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400dd50 con 0x7fbfe80cc540 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed063700 1 -- 172.21.15.97:0/1028456601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4010470 con 0x7fbfe80cc540 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.061+0000 7fbfed063700 1 -- 172.21.15.97:0/1028456601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4010800 con 0x7fbfe80cc540 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.062+0000 7fbff02c9700 1 -- 172.21.15.97:0/1028456601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe80cc540 msgr2=0x7fbfe806e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.062+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1028456601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe80cc540 0x7fbfe806e5e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400bb90 tx=0x7fbfe400bea0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.062+0000 7fbff02c9700 1 -- 172.21.15.97:0/1028456601 shutdown_connections 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.062+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1028456601 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe806eb20 0x7fbfe80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.062+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1028456601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe80cc540 0x7fbfe806e5e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.062+0000 7fbff02c9700 1 -- 172.21.15.97:0/1028456601 >> 172.21.15.97:0/1028456601 conn(0x7fbfe80b2050 msgr2=0x7fbfe80b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.062+0000 7fbff02c9700 1 -- 172.21.15.97:0/1028456601 shutdown_connections 2023-11-24T06:11:21.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.062+0000 7fbff02c9700 1 -- 172.21.15.97:0/1028456601 wait complete. 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.063+0000 7fbff02c9700 1 Processor -- start 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.063+0000 7fbff02c9700 1 -- start start 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbff02c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe806eb20 0x7fbfe81487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbff02c9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe80cc540 0x7fbfe81573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbff02c9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe815a620 0x7fbfe8153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbff02c9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8148d20 con 0x7fbfe806eb20 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbff02c9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8147d90 con 0x7fbfe815a620 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbff02c9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe81578f0 con 0x7fbfe80cc540 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe806eb20 0x7fbfe81487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe806eb20 0x7fbfe81487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49602/0 (socket says 172.21.15.97:49602) 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 -- 172.21.15.97:0/1827515224 learned_addr learned my addr 172.21.15.97:0/1827515224 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:21.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 -- 172.21.15.97:0/1827515224 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe80cc540 msgr2=0x7fbfe81573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe80cc540 0x7fbfe81573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 -- 172.21.15.97:0/1827515224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe815a620 msgr2=0x7fbfe8153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe815a620 0x7fbfe8153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 -- 172.21.15.97:0/1827515224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe400b7f0 con 0x7fbfe806eb20 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfee065700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe806eb20 0x7fbfe81487e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80011e0 tx=0x7fbfd800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfdeffd700 1 -- 172.21.15.97:0/1827515224 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80054e0 con 0x7fbfe806eb20 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfdeffd700 1 -- 172.21.15.97:0/1827515224 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd8003680 con 0x7fbfe806eb20 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe8153a90 con 0x7fbfe806eb20 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbfdeffd700 1 -- 172.21.15.97:0/1827515224 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8003a30 con 0x7fbfe806eb20 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.064+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe8147fe0 con 0x7fbfe806eb20 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.065+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe8068d90 con 0x7fbfe806eb20 2023-11-24T06:11:21.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.066+0000 7fbfdeffd700 1 -- 172.21.15.97:0/1827515224 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbfd8003b90 con 0x7fbfe806eb20 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.066+0000 7fbfdeffd700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbfcc06b2f0 0x7fbfcc06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.066+0000 7fbfdeffd700 1 -- 172.21.15.97:0/1827515224 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbfd8089290 con 0x7fbfe806eb20 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.066+0000 7fbfed864700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbfcc06b2f0 0x7fbfcc06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.066+0000 7fbfed864700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbfcc06b2f0 0x7fbfcc06d7b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbfe806d4a0 tx=0x7fbfe4012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.071+0000 7fbfdeffd700 1 -- 172.21.15.97:0/1827515224 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbfd8053d20 con 0x7fbfe806eb20 2023-11-24T06:11:21.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.261+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbfe8068fb0 con 0x7fbfe806eb20 2023-11-24T06:11:21.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.261+0000 7fbfdeffd700 1 -- 172.21.15.97:0/1827515224 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfd80170a0 con 0x7fbfe806eb20 2023-11-24T06:11:21.264 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbfcc06b2f0 msgr2=0x7fbfcc06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbfcc06b2f0 0x7fbfcc06d7b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbfe806d4a0 tx=0x7fbfe4012000 comp rx=0 tx=0).stop 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe806eb20 msgr2=0x7fbfe81487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe806eb20 0x7fbfe81487e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80011e0 tx=0x7fbfd800cae0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 shutdown_connections 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfe815a620 0x7fbfe8153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfe806eb20 0x7fbfe81487e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbfcc06b2f0 0x7fbfcc06d7b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 --2- 172.21.15.97:0/1827515224 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbfe80cc540 0x7fbfe81573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 >> 172.21.15.97:0/1827515224 conn(0x7fbfe80b2050 msgr2=0x7fbfe8154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 shutdown_connections 2023-11-24T06:11:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.263+0000 7fbff02c9700 1 -- 172.21.15.97:0/1827515224 wait complete. 2023-11-24T06:11:21.278 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:21.400 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.398+0000 7f1888dfa700 1 Processor -- start 2023-11-24T06:11:21.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.398+0000 7f1888dfa700 1 -- start start 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1888dfa700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18840b4310 0x7f188406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1888dfa700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f18840ce430 0x7f18840ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1888dfa700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18840cedc0 con 0x7f188415e080 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1888dfa700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18840d1200 con 0x7f18840b4310 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1888dfa700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188415e460 con 0x7f18840ce430 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1881d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18840b4310 0x7f188406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1881d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18840b4310 0x7f188406bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49612/0 (socket says 172.21.15.97:49612) 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1881d9b700 1 -- 172.21.15.97:0/624770624 learned_addr learned my addr 172.21.15.97:0/624770624 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1882d9d700 1 --2- 172.21.15.97:0/624770624 >> v2:172.21.15.183:3300/0 conn(0x7f18840ce430 0x7f18840ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1881d9b700 1 -- 172.21.15.97:0/624770624 >> v2:172.21.15.183:3300/0 conn(0x7f18840ce430 msgr2=0x7f18840ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1881d9b700 1 --2- 172.21.15.97:0/624770624 >> v2:172.21.15.183:3300/0 conn(0x7f18840ce430 0x7f18840ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1881d9b700 1 -- 172.21.15.97:0/624770624 >> v1:172.21.15.183:6789/0 conn(0x7f188415e080 legacy=0x7f18840d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1881d9b700 1 -- 172.21.15.97:0/624770624 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188406ba50 con 0x7f18840b4310 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.399+0000 7f1881d9b700 1 --2- 172.21.15.97:0/624770624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840b4310 0x7f188406bee0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f188406e1c0 tx=0x7f1874009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e89fe885124c41e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f188159a700 1 -- 172.21.15.97:0/624770624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874011830 con 0x7f18840b4310 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f188159a700 1 -- 172.21.15.97:0/624770624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1874011990 con 0x7f18840b4310 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f188159a700 1 -- 172.21.15.97:0/624770624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874011d20 con 0x7f18840b4310 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f1888dfa700 1 -- 172.21.15.97:0/624770624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840b4310 msgr2=0x7f188406bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f1888dfa700 1 --2- 172.21.15.97:0/624770624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840b4310 0x7f188406bee0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f188406e1c0 tx=0x7f1874009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f1888dfa700 1 -- 172.21.15.97:0/624770624 shutdown_connections 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f1888dfa700 1 --2- 172.21.15.97:0/624770624 >> v2:172.21.15.183:3300/0 conn(0x7f18840ce430 0x7f18840ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f1888dfa700 1 --2- 172.21.15.97:0/624770624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840b4310 0x7f188406bee0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f1888dfa700 1 -- 172.21.15.97:0/624770624 >> 172.21.15.97:0/624770624 conn(0x7f18840b3780 msgr2=0x7f18840b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f1888dfa700 1 -- 172.21.15.97:0/624770624 shutdown_connections 2023-11-24T06:11:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.400+0000 7f1888dfa700 1 -- 172.21.15.97:0/624770624 wait complete. 2023-11-24T06:11:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1888dfa700 1 Processor -- start 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1888dfa700 1 -- start start 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1888dfa700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18840b4310 0x7f1884147f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1888dfa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840ce430 0x7f18840c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1888dfa700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f188415e080 0x7f18840c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1888dfa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18840b6250 con 0x7f18840ce430 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1888dfa700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18840b5680 con 0x7f18840b4310 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1888dfa700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18840b5920 con 0x7f188415e080 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1881d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840ce430 0x7f18840c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1881d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840ce430 0x7f18840c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49618/0 (socket says 172.21.15.97:49618) 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.402+0000 7f1881d9b700 1 -- 172.21.15.97:0/2829542105 learned_addr learned my addr 172.21.15.97:0/2829542105 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1882d9d700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f188415e080 0x7f18840c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f188259c700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18840b4310 0x7f1884147f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1881d9b700 1 -- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f188415e080 msgr2=0x7f18840c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1881d9b700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f188415e080 0x7f18840c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1881d9b700 1 -- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18840b4310 msgr2=0x7f1884147f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1881d9b700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18840b4310 0x7f1884147f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1881d9b700 1 -- 172.21.15.97:0/2829542105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1874009550 con 0x7f18840ce430 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f188259c700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18840b4310 0x7f1884147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1881d9b700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840ce430 0x7f18840c7d70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1874000c00 tx=0x7f1874004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1872ffd700 1 -- 172.21.15.97:0/2829542105 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874004570 con 0x7f18840ce430 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1872ffd700 1 -- 172.21.15.97:0/2829542105 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187401f070 con 0x7f18840ce430 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18840b5c00 con 0x7f18840ce430 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1872ffd700 1 -- 172.21.15.97:0/2829542105 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18740038e0 con 0x7f18840ce430 2023-11-24T06:11:21.409 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.403+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1884148460 con 0x7f18840ce430 2023-11-24T06:11:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.404+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1884068fb0 con 0x7f18840ce430 2023-11-24T06:11:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.404+0000 7f1872ffd700 1 -- 172.21.15.97:0/2829542105 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1874021030 con 0x7f18840ce430 2023-11-24T06:11:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.405+0000 7f1872ffd700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f186406b2c0 0x7f186406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.405+0000 7f1872ffd700 1 -- 172.21.15.97:0/2829542105 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f187408afd0 con 0x7f18840ce430 2023-11-24T06:11:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.405+0000 7f188259c700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f186406b2c0 0x7f186406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.410 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.405+0000 7f188259c700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f186406b2c0 0x7f186406d780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1878000f60 tx=0x7f1878009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.409+0000 7f1872ffd700 1 -- 172.21.15.97:0/2829542105 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1874055c00 con 0x7f18840ce430 2023-11-24T06:11:21.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.597+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18840b6390 con 0x7f18840ce430 2023-11-24T06:11:21.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.598+0000 7f1872ffd700 1 -- 172.21.15.97:0/2829542105 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18840b6390 con 0x7f18840ce430 2023-11-24T06:11:21.601 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f186406b2c0 msgr2=0x7f186406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f186406b2c0 0x7f186406d780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1878000f60 tx=0x7f1878009040 comp rx=0 tx=0).stop 2023-11-24T06:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840ce430 msgr2=0x7f18840c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840ce430 0x7f18840c7d70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1874000c00 tx=0x7f1874004750 comp rx=0 tx=0).stop 2023-11-24T06:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 shutdown_connections 2023-11-24T06:11:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f186406b2c0 0x7f186406d780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f188415e080 0x7f18840c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18840b4310 0x7f1884147f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 --2- 172.21.15.97:0/2829542105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18840ce430 0x7f18840c7d70 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 >> 172.21.15.97:0/2829542105 conn(0x7f18840b3780 msgr2=0x7f18840b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 shutdown_connections 2023-11-24T06:11:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.600+0000 7f1888dfa700 1 -- 172.21.15.97:0/2829542105 wait complete. 2023-11-24T06:11:21.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.736+0000 7f5b09187700 1 Processor -- start 2023-11-24T06:11:21.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.736+0000 7f5b09187700 1 -- start start 2023-11-24T06:11:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.736+0000 7f5b09187700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b040b5da0 0x7f5b040afc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.736+0000 7f5b09187700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040b3c60 con 0x7f5b0415e160 2023-11-24T06:11:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.736+0000 7f5b09187700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040b58c0 con 0x7f5b040746a0 2023-11-24T06:11:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.736+0000 7f5b09187700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040b4500 con 0x7f5b040b5da0 2023-11-24T06:11:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b02d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5b0415e160 0x7f5b04073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42076/0 (socket says 172.21.15.97:42076) 2023-11-24T06:11:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b02d9d700 1 -- 172.21.15.97:0/1216974736 learned_addr learned my addr 172.21.15.97:0/1216974736 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706994490 0 0) 0x7f5b040b3c60 con 0x7f5b0415e160 2023-11-24T06:11:21.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af00035b0 con 0x7f5b0415e160 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b0359e700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b040b5da0 0x7f5b040afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213794575 0 0) 0x7f5af00035b0 con 0x7f5b0415e160 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b040b5da0 msgr2=0x7f5b040afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b040b5da0 0x7f5b040afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 >> v1:172.21.15.183:6789/0 conn(0x7f5b040746a0 legacy=0x7f5b0406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0406bdc0 con 0x7f5b0415e160 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f5aec001670 con 0x7f5b0415e160 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 >> v1:172.21.15.97:6789/0 conn(0x7f5b0415e160 legacy=0x7f5b04073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0003a30 0x7f5af0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b040b5da0 0x7f5af0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5af0004d90 0x7f5af0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040b4500 con 0x7f5af0003a30 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af000bac0 con 0x7f5af0004d90 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af000bc80 con 0x7f5b040b5da0 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b02d9d700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0003a30 0x7f5af0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b0359e700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5af0004d90 0x7f5af0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.737+0000 7f5b0259c700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b040b5da0 0x7f5af0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b02d9d700 1 -- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b040b5da0 msgr2=0x7f5af0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b02d9d700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b040b5da0 0x7f5af0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b02d9d700 1 -- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5af0004d90 msgr2=0x7f5af0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b02d9d700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5af0004d90 0x7f5af0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b02d9d700 1 -- 172.21.15.97:0/1216974736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af00035b0 con 0x7f5af0003a30 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b0359e700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5af0004d90 0x7f5af0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b02d9d700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0003a30 0x7f5af0003e50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c210 tx=0x7f5af0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6d9f92fa5bbcb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec007c10 con 0x7f5af0003a30 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec00cdb0 con 0x7f5af0003a30 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b01d9b700 1 -- 172.21.15.97:0/1216974736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec005ba0 con 0x7f5af0003a30 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b09187700 1 -- 172.21.15.97:0/1216974736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0003a30 msgr2=0x7f5af0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b09187700 1 --2- 172.21.15.97:0/1216974736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0003a30 0x7f5af0003e50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c210 tx=0x7f5af0005880 comp rx=0 tx=0).stop 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b0359e700 1 -- 172.21.15.97:0/1216974736 reap_dead start 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b09187700 1 -- 172.21.15.97:0/1216974736 shutdown_connections 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b09187700 1 -- 172.21.15.97:0/1216974736 >> 172.21.15.97:0/1216974736 conn(0x7f5b040d0bf0 msgr2=0x7f5b040d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b09187700 1 -- 172.21.15.97:0/1216974736 shutdown_connections 2023-11-24T06:11:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.738+0000 7f5b09187700 1 -- 172.21.15.97:0/1216974736 wait complete. 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.740+0000 7f5b09187700 1 Processor -- start 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.740+0000 7f5b09187700 1 -- start start 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b09187700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5af0003a30 0x7f5b040cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b09187700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b0414b280 0x7f5b0414b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b09187700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b040c6ec0 0x7f5b040c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b09187700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b041f4ed0 con 0x7f5b040c6ec0 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b09187700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0414bc40 con 0x7f5b0414b280 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b09187700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b040c7c10 con 0x7f5af0003a30 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b0359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b040c6ec0 0x7f5b040c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b0359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b040c6ec0 0x7f5b040c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49638/0 (socket says 172.21.15.97:49638) 2023-11-24T06:11:21.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b0359e700 1 -- 172.21.15.97:0/1975730048 learned_addr learned my addr 172.21.15.97:0/1975730048 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b02d9d700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5af0003a30 0x7f5b040cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.741+0000 7f5b0259c700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b0414b280 0x7f5b0414b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5b0359e700 1 -- 172.21.15.97:0/1975730048 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5af0003a30 msgr2=0x7f5b040cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5b0359e700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5af0003a30 0x7f5b040cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5b0359e700 1 -- 172.21.15.97:0/1975730048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b0414b280 msgr2=0x7f5b0414b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5b0359e700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b0414b280 0x7f5b0414b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5b0359e700 1 -- 172.21.15.97:0/1975730048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b040c7ef0 con 0x7f5b040c6ec0 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5b0359e700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b040c6ec0 0x7f5b040c7340 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5b040d16e0 tx=0x7f5afc0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5afb7fe700 1 -- 172.21.15.97:0/1975730048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc01b070 con 0x7f5b040c6ec0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5afb7fe700 1 -- 172.21.15.97:0/1975730048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc00f040 con 0x7f5b040c6ec0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b040c81d0 con 0x7f5b040c6ec0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5afb7fe700 1 -- 172.21.15.97:0/1975730048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00e050 con 0x7f5b040c6ec0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.742+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b040c7880 con 0x7f5b040c6ec0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.743+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b040c86f0 con 0x7f5b040c6ec0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.743+0000 7f5afb7fe700 1 -- 172.21.15.97:0/1975730048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5afc003a20 con 0x7f5b040c6ec0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.744+0000 7f5afb7fe700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5ae406b2f0 0x7f5ae406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.744+0000 7f5afb7fe700 1 -- 172.21.15.97:0/1975730048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5afc08a0f0 con 0x7f5b040c6ec0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.744+0000 7f5b02d9d700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5ae406b2f0 0x7f5ae406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.744+0000 7f5b02d9d700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5ae406b2f0 0x7f5ae406d7b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5aec000c00 tx=0x7f5aec007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.748+0000 7f5afb7fe700 1 -- 172.21.15.97:0/1975730048 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5afc054b00 con 0x7f5b040c6ec0 2023-11-24T06:11:21.939 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.937+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b041f5010 con 0x7f5b040c6ec0 2023-11-24T06:11:21.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.937+0000 7f5afb7fe700 1 -- 172.21.15.97:0/1975730048 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b041f5010 con 0x7f5b040c6ec0 2023-11-24T06:11:21.940 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:11:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5ae406b2f0 msgr2=0x7f5ae406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5ae406b2f0 0x7f5ae406d7b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5aec000c00 tx=0x7f5aec007000 comp rx=0 tx=0).stop 2023-11-24T06:11:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b040c6ec0 msgr2=0x7f5b040c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b040c6ec0 0x7f5b040c7340 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5b040d16e0 tx=0x7f5afc0104a0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 shutdown_connections 2023-11-24T06:11:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b040c6ec0 0x7f5b040c7340 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5ae406b2f0 0x7f5ae406d7b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5af0003a30 0x7f5b040cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 --2- 172.21.15.97:0/1975730048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b0414b280 0x7f5b0414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 >> 172.21.15.97:0/1975730048 conn(0x7f5b040d0bf0 msgr2=0x7f5b0406db40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 shutdown_connections 2023-11-24T06:11:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:21.939+0000 7f5b09187700 1 -- 172.21.15.97:0/1975730048 wait complete. 2023-11-24T06:11:21.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.071+0000 7fcdb2be5700 1 Processor -- start 2023-11-24T06:11:22.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.071+0000 7fcdb2be5700 1 -- start start 2023-11-24T06:11:22.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.072+0000 7fcdb2be5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcdac1446a0 0x7fcdac0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.072+0000 7fcdb2be5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac0af950 con 0x7fcdac0adf00 2023-11-24T06:11:22.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.072+0000 7fcdb2be5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac0afc30 con 0x7fcdac06c710 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.072+0000 7fcdb2be5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac144a80 con 0x7fcdac1446a0 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.072+0000 7fcdb0981700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcdac1446a0 0x7fcdac0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.072+0000 7fcdabfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcdac0adf00 0x7fcdac06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42094/0 (socket says 172.21.15.97:42094) 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.072+0000 7fcdabfff700 1 -- 172.21.15.97:0/1032751843 learned_addr learned my addr 172.21.15.97:0/1032751843 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb0981700 1 -- 172.21.15.97:0/1032751843 >> v1:172.21.15.183:6789/0 conn(0x7fcdac06c710 legacy=0x7fcdac0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb0981700 1 -- 172.21.15.97:0/1032751843 >> v1:172.21.15.97:6789/0 conn(0x7fcdac0adf00 legacy=0x7fcdac06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdab7fe700 1 -- 172.21.15.97:0/1032751843 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589910741 0 0) 0x7fcdac0af950 con 0x7fcdac0adf00 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb0981700 1 -- 172.21.15.97:0/1032751843 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0aff10 con 0x7fcdac1446a0 2023-11-24T06:11:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb0981700 1 --2- 172.21.15.97:0/1032751843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac1446a0 0x7fcdac0b5ae0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009840 tx=0x7fcda0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14725a2eed347536 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdab7fe700 1 -- 172.21.15.97:0/1032751843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0004230 con 0x7fcdac1446a0 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdab7fe700 1 -- 172.21.15.97:0/1032751843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda0004390 con 0x7fcdac1446a0 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdab7fe700 1 -- 172.21.15.97:0/1032751843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00036a0 con 0x7fcdac1446a0 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1032751843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac1446a0 msgr2=0x7fcdac0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb2be5700 1 --2- 172.21.15.97:0/1032751843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac1446a0 0x7fcdac0b5ae0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009840 tx=0x7fcda0009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1032751843 shutdown_connections 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb2be5700 1 --2- 172.21.15.97:0/1032751843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac1446a0 0x7fcdac0b5ae0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.073+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1032751843 >> 172.21.15.97:0/1032751843 conn(0x7fcdac071b90 msgr2=0x7fcdac071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.074+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1032751843 shutdown_connections 2023-11-24T06:11:22.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.074+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1032751843 wait complete. 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.075+0000 7fcdb2be5700 1 Processor -- start 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.075+0000 7fcdb2be5700 1 -- start start 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb2be5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdac06c710 0x7fcdac09d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb2be5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac0adf00 0x7fcdac09ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb2be5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcdac1446a0 0x7fcdac13a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb2be5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0ca110 con 0x7fcdac0adf00 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb2be5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac09e000 con 0x7fcdac06c710 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb2be5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac09d760 con 0x7fcdac1446a0 2023-11-24T06:11:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac0adf00 0x7fcdac09ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac0adf00 0x7fcdac09ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49650/0 (socket says 172.21.15.97:49650) 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 -- 172.21.15.97:0/1740993789 learned_addr learned my addr 172.21.15.97:0/1740993789 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb1182700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcdac1446a0 0x7fcdac13a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb0981700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdac06c710 0x7fcdac09d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 -- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcdac1446a0 msgr2=0x7fcdac13a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcdac1446a0 0x7fcdac13a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 -- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdac06c710 msgr2=0x7fcdac09d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdac06c710 0x7fcdac09d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 -- 172.21.15.97:0/1740993789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00094a0 con 0x7fcdac0adf00 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb0981700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdac06c710 0x7fcdac09d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:22.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdabfff700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac0adf00 0x7fcdac09ac30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c4d0 tx=0x7fcd9800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcda97fa700 1 -- 172.21.15.97:0/1740993789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98004c80 con 0x7fcdac0adf00 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcda97fa700 1 -- 172.21.15.97:0/1740993789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98004e00 con 0x7fcdac0adf00 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac09da40 con 0x7fcdac0adf00 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcda97fa700 1 -- 172.21.15.97:0/1740993789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98007310 con 0x7fcdac0adf00 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.076+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac09b170 con 0x7fcdac0adf00 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.077+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac0753f0 con 0x7fcdac0adf00 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.078+0000 7fcda97fa700 1 -- 172.21.15.97:0/1740993789 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcd98007470 con 0x7fcdac0adf00 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.078+0000 7fcda97fa700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd9006b2c0 0x7fcd9006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.078+0000 7fcdb0981700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd9006b2c0 0x7fcd9006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.078+0000 7fcda97fa700 1 -- 172.21.15.97:0/1740993789 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcd980899a0 con 0x7fcdac0adf00 2023-11-24T06:11:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.078+0000 7fcdb0981700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd9006b2c0 0x7fcd9006d780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcda0006ec0 tx=0x7fcda0013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.082+0000 7fcda97fa700 1 -- 172.21.15.97:0/1740993789 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd98054540 con 0x7fcdac0adf00 2023-11-24T06:11:22.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.315+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdac0cd3e0 con 0x7fcdac0adf00 2023-11-24T06:11:22.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.315+0000 7fcda97fa700 1 -- 172.21.15.97:0/1740993789 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fcd98057b60 con 0x7fcdac0adf00 2023-11-24T06:11:22.318 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:22.318 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3508641674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3508641674},{"type":"v1","addr":"172.21.15.183:6833","nonce":3508641674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5557,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/116508824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":116508824},{"type":"v1","addr":"172.21.15.183:6835","nonce":116508824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2785455707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2785455707},{"type":"v1","addr":"172.21.15.97:6837","nonce":2785455707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:03.572274+0000","modified":"2023-11-24T06:11:06.595718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3662495999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3662495999},{"type":"v1","addr":"172.21.15.97:6836","nonce":3662495999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:11:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd9006b2c0 msgr2=0x7fcd9006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd9006b2c0 0x7fcd9006d780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcda0006ec0 tx=0x7fcda0013040 comp rx=0 tx=0).stop 2023-11-24T06:11:22.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac0adf00 msgr2=0x7fcdac09ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac0adf00 0x7fcdac09ac30 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c4d0 tx=0x7fcd9800c890 comp rx=0 tx=0).stop 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 shutdown_connections 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd9006b2c0 0x7fcd9006d780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcdac1446a0 0x7fcdac13a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdac06c710 0x7fcdac09d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.317+0000 7fcdb2be5700 1 --2- 172.21.15.97:0/1740993789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdac0adf00 0x7fcdac09ac30 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.318+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 >> 172.21.15.97:0/1740993789 conn(0x7fcdac071b90 msgr2=0x7fcdac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.318+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 shutdown_connections 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.318+0000 7fcdb2be5700 1 -- 172.21.15.97:0/1740993789 wait complete. 2023-11-24T06:11:22.321 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 37 2023-11-24T06:11:22.332 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:22.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.449+0000 7f0128edb700 1 Processor -- start 2023-11-24T06:11:22.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.449+0000 7f0128edb700 1 -- start start 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.450+0000 7f0128edb700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f012415e2a0 0x7f012406ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.450+0000 7f0128edb700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0124073f90 0x7f0124074420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.450+0000 7f0128edb700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012406e7f0 con 0x7f01240ce090 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.450+0000 7f0128edb700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0124074960 con 0x7f0124073f90 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.450+0000 7f0128edb700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012406dec0 con 0x7f012415e2a0 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f0122d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0124073f90 0x7f0124074420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f0121d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f01240ce090 0x7f012406ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42110/0 (socket says 172.21.15.97:42110) 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f0121d9b700 1 -- 172.21.15.97:0/129018465 learned_addr learned my addr 172.21.15.97:0/129018465 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f012259c700 1 --2- 172.21.15.97:0/129018465 >> v2:172.21.15.183:3300/0 conn(0x7f012415e2a0 0x7f012406ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f0122d9d700 1 -- 172.21.15.97:0/129018465 >> v2:172.21.15.183:3300/0 conn(0x7f012415e2a0 msgr2=0x7f012406ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f0122d9d700 1 --2- 172.21.15.97:0/129018465 >> v2:172.21.15.183:3300/0 conn(0x7f012415e2a0 0x7f012406ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f0122d9d700 1 -- 172.21.15.97:0/129018465 >> v1:172.21.15.97:6789/0 conn(0x7f01240ce090 legacy=0x7f012406ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f0122d9d700 1 -- 172.21.15.97:0/129018465 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012406e1a0 con 0x7f0124073f90 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f012159a700 1 -- 172.21.15.97:0/129018465 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031115224 0 0) 0x7f012406e7f0 con 0x7f01240ce090 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f0122d9d700 1 --2- 172.21.15.97:0/129018465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124073f90 0x7f0124074420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f012406d0c0 tx=0x7f011c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3a2e176e69c516 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f012159a700 1 -- 172.21.15.97:0/129018465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c00e070 con 0x7f0124073f90 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f012159a700 1 -- 172.21.15.97:0/129018465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011c00bb20 con 0x7f0124073f90 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.451+0000 7f012159a700 1 -- 172.21.15.97:0/129018465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c012430 con 0x7f0124073f90 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.452+0000 7f0128edb700 1 -- 172.21.15.97:0/129018465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124073f90 msgr2=0x7f0124074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.452+0000 7f0128edb700 1 --2- 172.21.15.97:0/129018465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124073f90 0x7f0124074420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f012406d0c0 tx=0x7f011c00b330 comp rx=0 tx=0).stop 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.452+0000 7f0128edb700 1 -- 172.21.15.97:0/129018465 shutdown_connections 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.452+0000 7f0128edb700 1 --2- 172.21.15.97:0/129018465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0124073f90 0x7f0124074420 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.452+0000 7f0128edb700 1 --2- 172.21.15.97:0/129018465 >> v2:172.21.15.183:3300/0 conn(0x7f012415e2a0 0x7f012406ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.452+0000 7f0128edb700 1 -- 172.21.15.97:0/129018465 >> 172.21.15.97:0/129018465 conn(0x7f01240cd570 msgr2=0x7f01240cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:22.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.452+0000 7f0128edb700 1 -- 172.21.15.97:0/129018465 shutdown_connections 2023-11-24T06:11:22.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.452+0000 7f0128edb700 1 -- 172.21.15.97:0/129018465 wait complete. 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.453+0000 7f0128edb700 1 Processor -- start 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.453+0000 7f0128edb700 1 -- start start 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0128edb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0124073f90 0x7f01241576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0128edb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01240ce090 0x7f01240ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0128edb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415e2a0 0x7f01240aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0128edb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01241f38a0 con 0x7f012415e2a0 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0128edb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124157c30 con 0x7f01240ce090 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0128edb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01241f3b10 con 0x7f0124073f90 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415e2a0 0x7f01240aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415e2a0 0x7f01240aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49666/0 (socket says 172.21.15.97:49666) 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 -- 172.21.15.97:0/2800797518 learned_addr learned my addr 172.21.15.97:0/2800797518 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0121d9b700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01240ce090 0x7f01240ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f012259c700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0124073f90 0x7f01241576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 -- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0124073f90 msgr2=0x7f01241576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0124073f90 0x7f01241576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 -- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01240ce090 msgr2=0x7f01240ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01240ce090 0x7f01240ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 -- 172.21.15.97:0/2800797518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c009a70 con 0x7f012415e2a0 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f012259c700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0124073f90 0x7f01241576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0121d9b700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01240ce090 0x7f01240ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0122d9d700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415e2a0 0x7f01240aea10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0124073510 tx=0x7f011c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.454+0000 7f0112ffd700 1 -- 172.21.15.97:0/2800797518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c012430 con 0x7f012415e2a0 2023-11-24T06:11:22.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.455+0000 7f0112ffd700 1 -- 172.21.15.97:0/2800797518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011c020070 con 0x7f012415e2a0 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.455+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01241f3df0 con 0x7f012415e2a0 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.455+0000 7f0112ffd700 1 -- 172.21.15.97:0/2800797518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c004280 con 0x7f012415e2a0 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.455+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01241f43b0 con 0x7f012415e2a0 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.455+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0124067f60 con 0x7f012415e2a0 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.456+0000 7f0112ffd700 1 -- 172.21.15.97:0/2800797518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f011c0043e0 con 0x7f012415e2a0 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.456+0000 7f0112ffd700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f010406b2c0 0x7f010406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.456+0000 7f0112ffd700 1 -- 172.21.15.97:0/2800797518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f011c022070 con 0x7f012415e2a0 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.456+0000 7f012259c700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f010406b2c0 0x7f010406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.461 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.457+0000 7f012259c700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f010406b2c0 0x7f010406d780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0118009b60 tx=0x7f0118008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.461+0000 7f0112ffd700 1 -- 172.21.15.97:0/2800797518 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f011c056560 con 0x7f012415e2a0 2023-11-24T06:11:22.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.652+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0124072e00 con 0x7f012415e2a0 2023-11-24T06:11:22.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.653+0000 7f0112ffd700 1 -- 172.21.15.97:0/2800797518 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16685 (secure 0 0 0) 0x7f011c059b80 con 0x7f012415e2a0 2023-11-24T06:11:22.656 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:22.656 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":61,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:19.547545+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:01.950100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:02.526161+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f010406b2c0 msgr2=0x7f010406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f010406b2c0 0x7f010406d780 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0118009b60 tx=0x7f0118008040 comp rx=0 tx=0).stop 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415e2a0 msgr2=0x7f01240aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415e2a0 0x7f01240aea10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0124073510 tx=0x7f011c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 shutdown_connections 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415e2a0 0x7f01240aea10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f010406b2c0 0x7f010406d780 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0124073f90 0x7f01241576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 --2- 172.21.15.97:0/2800797518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01240ce090 0x7f01240ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 >> 172.21.15.97:0/2800797518 conn(0x7f01240cd570 msgr2=0x7f012415d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 shutdown_connections 2023-11-24T06:11:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.655+0000 7f0128edb700 1 -- 172.21.15.97:0/2800797518 wait complete. 2023-11-24T06:11:22.670 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:11:22.670 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:22.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.788+0000 7efe3c2ee700 1 Processor -- start 2023-11-24T06:11:22.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.788+0000 7efe3c2ee700 1 -- start start 2023-11-24T06:11:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.789+0000 7efe3c2ee700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efe34144660 0x7efe340b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.789+0000 7efe3c2ee700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3415f520 con 0x7efe3406c710 2023-11-24T06:11:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.789+0000 7efe3c2ee700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3415f800 con 0x7efe3415ea60 2023-11-24T06:11:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.789+0000 7efe3c2ee700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe340b4230 con 0x7efe34144660 2023-11-24T06:11:22.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.789+0000 7efe3a88b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7efe3406c710 0x7efe3406ed70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42126/0 (socket says 172.21.15.97:42126) 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.789+0000 7efe3a88b700 1 -- 172.21.15.97:0/2084440310 learned_addr learned my addr 172.21.15.97:0/2084440310 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.789+0000 7efe3a08a700 1 --2- 172.21.15.97:0/2084440310 >> v2:172.21.15.183:3300/0 conn(0x7efe34144660 0x7efe340b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe39088700 1 -- 172.21.15.97:0/2084440310 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069538638 0 0) 0x7efe3415f520 con 0x7efe3406c710 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe39088700 1 -- 172.21.15.97:0/2084440310 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe200035b0 con 0x7efe3406c710 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe39088700 1 -- 172.21.15.97:0/2084440310 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216882589 0 0) 0x7efe3415f800 con 0x7efe3415ea60 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe39088700 1 -- 172.21.15.97:0/2084440310 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe3415f520 con 0x7efe3415ea60 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe3a08a700 1 -- 172.21.15.97:0/2084440310 >> v1:172.21.15.183:6789/0 conn(0x7efe3415ea60 legacy=0x7efe3406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe3a08a700 1 -- 172.21.15.97:0/2084440310 >> v1:172.21.15.97:6789/0 conn(0x7efe3406c710 legacy=0x7efe3406ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe39088700 1 -- 172.21.15.97:0/2084440310 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4266748290 0 0) 0x7efe200035b0 con 0x7efe3406c710 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe3a08a700 1 -- 172.21.15.97:0/2084440310 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340b3e50 con 0x7efe34144660 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe3a08a700 1 --2- 172.21.15.97:0/2084440310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34144660 0x7efe340b4650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe30009840 tx=0x7efe30009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94821f1f038c3fd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe39088700 1 -- 172.21.15.97:0/2084440310 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0019f0 con 0x7efe34144660 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe39088700 1 -- 172.21.15.97:0/2084440310 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe30004030 con 0x7efe34144660 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.790+0000 7efe39088700 1 -- 172.21.15.97:0/2084440310 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe300043c0 con 0x7efe34144660 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.791+0000 7efe3c2ee700 1 -- 172.21.15.97:0/2084440310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34144660 msgr2=0x7efe340b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.791+0000 7efe3c2ee700 1 --2- 172.21.15.97:0/2084440310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34144660 0x7efe340b4650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efe30009840 tx=0x7efe30009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.791+0000 7efe3c2ee700 1 -- 172.21.15.97:0/2084440310 shutdown_connections 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.791+0000 7efe3c2ee700 1 --2- 172.21.15.97:0/2084440310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe34144660 0x7efe340b4650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.791+0000 7efe3c2ee700 1 -- 172.21.15.97:0/2084440310 >> 172.21.15.97:0/2084440310 conn(0x7efe34071ac0 msgr2=0x7efe34071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.791+0000 7efe3c2ee700 1 -- 172.21.15.97:0/2084440310 shutdown_connections 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.791+0000 7efe3c2ee700 1 -- 172.21.15.97:0/2084440310 wait complete. 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.792+0000 7efe3c2ee700 1 Processor -- start 2023-11-24T06:11:22.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.792+0000 7efe3c2ee700 1 -- start start 2023-11-24T06:11:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3c2ee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3406c710 0x7efe3415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3c2ee700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe34144660 0x7efe3415c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3c2ee700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe3415ea60 0x7efe34136e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3c2ee700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe341f17e0 con 0x7efe3406c710 2023-11-24T06:11:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3c2ee700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3415ce20 con 0x7efe3415ea60 2023-11-24T06:11:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3c2ee700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3415d100 con 0x7efe34144660 2023-11-24T06:11:22.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3a08a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3406c710 0x7efe3415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3a08a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3406c710 0x7efe3415dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49676/0 (socket says 172.21.15.97:49676) 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.793+0000 7efe3a08a700 1 -- 172.21.15.97:0/1599075779 learned_addr learned my addr 172.21.15.97:0/1599075779 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe3a08a700 1 -- 172.21.15.97:0/1599075779 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe34144660 msgr2=0x7efe3415c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe3a08a700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe34144660 0x7efe3415c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe3a08a700 1 -- 172.21.15.97:0/1599075779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe3415ea60 msgr2=0x7efe34136e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe3a08a700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe3415ea60 0x7efe34136e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe3a08a700 1 -- 172.21.15.97:0/1599075779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300094a0 con 0x7efe3406c710 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe3a08a700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3406c710 0x7efe3415dec0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe2affd700 1 -- 172.21.15.97:0/1599075779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30004590 con 0x7efe3406c710 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe2affd700 1 -- 172.21.15.97:0/1599075779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3001f070 con 0x7efe3406c710 2023-11-24T06:11:22.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3415d3e0 con 0x7efe3406c710 2023-11-24T06:11:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe2affd700 1 -- 172.21.15.97:0/1599075779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30011870 con 0x7efe3406c710 2023-11-24T06:11:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.794+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3415d9a0 con 0x7efe3406c710 2023-11-24T06:11:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.795+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe34067f60 con 0x7efe3406c710 2023-11-24T06:11:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.795+0000 7efe2affd700 1 -- 172.21.15.97:0/1599075779 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7efe300119d0 con 0x7efe3406c710 2023-11-24T06:11:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.796+0000 7efe2affd700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efe1806b2f0 0x7efe1806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.796+0000 7efe2affd700 1 -- 172.21.15.97:0/1599075779 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7efe30021070 con 0x7efe3406c710 2023-11-24T06:11:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.796+0000 7efe39889700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efe1806b2f0 0x7efe1806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:22.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.796+0000 7efe39889700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efe1806b2f0 0x7efe1806d7b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efe24007fd0 tx=0x7efe24007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:22.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:22.800+0000 7efe2affd700 1 -- 172.21.15.97:0/1599075779 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe30056150 con 0x7efe3406c710 2023-11-24T06:11:23.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.032+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe340ce400 con 0x7efe3406c710 2023-11-24T06:11:23.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.033+0000 7efe2affd700 1 -- 172.21.15.97:0/1599075779 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4916 (secure 0 0 0) 0x7efe30059770 con 0x7efe3406c710 2023-11-24T06:11:23.036 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:23.036 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3508641674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3508641674},{"type":"v1","addr":"172.21.15.183:6833","nonce":3508641674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5557,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/116508824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":116508824},{"type":"v1","addr":"172.21.15.183:6835","nonce":116508824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2785455707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2785455707},{"type":"v1","addr":"172.21.15.97:6837","nonce":2785455707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:03.572274+0000","modified":"2023-11-24T06:11:06.595718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3662495999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3662495999},{"type":"v1","addr":"172.21.15.97:6836","nonce":3662495999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:11:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efe1806b2f0 msgr2=0x7efe1806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efe1806b2f0 0x7efe1806d7b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7efe24007fd0 tx=0x7efe24007980 comp rx=0 tx=0).stop 2023-11-24T06:11:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3406c710 msgr2=0x7efe3415dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3406c710 0x7efe3415dec0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe30004750 comp rx=0 tx=0).stop 2023-11-24T06:11:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 shutdown_connections 2023-11-24T06:11:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe3415ea60 0x7efe34136e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe3406c710 0x7efe3415dec0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efe1806b2f0 0x7efe1806d7b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 --2- 172.21.15.97:0/1599075779 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efe34144660 0x7efe3415c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 >> 172.21.15.97:0/1599075779 conn(0x7efe34071ac0 msgr2=0x7efe34071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 shutdown_connections 2023-11-24T06:11:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.035+0000 7efe3c2ee700 1 -- 172.21.15.97:0/1599075779 wait complete. 2023-11-24T06:11:23.039 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 37 2023-11-24T06:11:23.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:23.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.166+0000 7f11212f6700 1 Processor -- start 2023-11-24T06:11:23.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.166+0000 7f11212f6700 1 -- start start 2023-11-24T06:11:23.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f11212f6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f111c15e1e0 0x7f111c0d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f11212f6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f11212f6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c0d20b0 con 0x7f111c0d0db0 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f11212f6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c0d3150 con 0x7f111c15e1e0 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f11212f6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0d1240 con 0x7f111c0d3a00 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f111c15e1e0 0x7f111c0d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111affd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f111c15e1e0 0x7f111c0d34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49684/0 (socket says 172.21.15.97:49684) 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111affd700 1 -- 172.21.15.97:0/2741301376 learned_addr learned my addr 172.21.15.97:0/2741301376 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111a7fc700 1 --2- 172.21.15.97:0/2741301376 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111affd700 1 -- 172.21.15.97:0/2741301376 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 msgr2=0x7f111c0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111affd700 1 --2- 172.21.15.97:0/2741301376 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111affd700 1 -- 172.21.15.97:0/2741301376 >> v1:172.21.15.183:6789/0 conn(0x7f111c0d0db0 legacy=0x7f111c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111affd700 1 -- 172.21.15.97:0/2741301376 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0d2390 con 0x7f111c15e1e0 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.167+0000 7f111a7fc700 1 --2- 172.21.15.97:0/2741301376 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f111affd700 1 --2- 172.21.15.97:0/2741301376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c0d34c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f11100098f0 tx=0x7f1110009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218f1e8f7002a80e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f1119ffb700 1 -- 172.21.15.97:0/2741301376 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000b8a0 con 0x7f111c15e1e0 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f1119ffb700 1 -- 172.21.15.97:0/2741301376 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111000ba00 con 0x7f111c15e1e0 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f1119ffb700 1 -- 172.21.15.97:0/2741301376 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000bd90 con 0x7f111c15e1e0 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f11212f6700 1 -- 172.21.15.97:0/2741301376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 msgr2=0x7f111c0d34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f11212f6700 1 --2- 172.21.15.97:0/2741301376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c0d34c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f11100098f0 tx=0x7f1110009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f11212f6700 1 -- 172.21.15.97:0/2741301376 shutdown_connections 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f11212f6700 1 --2- 172.21.15.97:0/2741301376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c0d34c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f11212f6700 1 --2- 172.21.15.97:0/2741301376 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f11212f6700 1 -- 172.21.15.97:0/2741301376 >> 172.21.15.97:0/2741301376 conn(0x7f111c0b5b50 msgr2=0x7f111c0b5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f11212f6700 1 -- 172.21.15.97:0/2741301376 shutdown_connections 2023-11-24T06:11:23.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.168+0000 7f11212f6700 1 -- 172.21.15.97:0/2741301376 wait complete. 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.170+0000 7f11212f6700 1 Processor -- start 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.170+0000 7f11212f6700 1 -- start start 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f11212f6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c0d0db0 0x7f111c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f11212f6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f11212f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c1f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f11212f6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0cdb40 con 0x7f111c15e1e0 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f11212f6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0c8a60 con 0x7f111c0d0db0 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f11212f6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1f3ec0 con 0x7f111c0d3a00 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c1f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c1f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49694/0 (socket says 172.21.15.97:49694) 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f111b7fe700 1 -- 172.21.15.97:0/2388191009 learned_addr learned my addr 172.21.15.97:0/2388191009 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:23.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f111a7fc700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f111affd700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c0d0db0 0x7f111c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f111b7fe700 1 -- 172.21.15.97:0/2388191009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 msgr2=0x7f111c0c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.171+0000 7f111b7fe700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f111b7fe700 1 -- 172.21.15.97:0/2388191009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c0d0db0 msgr2=0x7f111c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f111b7fe700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c0d0db0 0x7f111c0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f111b7fe700 1 -- 172.21.15.97:0/2388191009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110009550 con 0x7f111c15e1e0 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f111b7fe700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c1f3980 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f111400cc80 tx=0x7f111400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f11037fe700 1 -- 172.21.15.97:0/2388191009 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114003c00 con 0x7f111c15e1e0 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f11037fe700 1 -- 172.21.15.97:0/2388191009 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1114003d60 con 0x7f111c15e1e0 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c0cdd50 con 0x7f111c15e1e0 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f11037fe700 1 -- 172.21.15.97:0/2388191009 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400aaa0 con 0x7f111c15e1e0 2023-11-24T06:11:23.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.172+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c0cb210 con 0x7f111c15e1e0 2023-11-24T06:11:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.173+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c0751e0 con 0x7f111c15e1e0 2023-11-24T06:11:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.173+0000 7f11037fe700 1 -- 172.21.15.97:0/2388191009 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f111400ac00 con 0x7f111c15e1e0 2023-11-24T06:11:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.173+0000 7f11037fe700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f110806b2c0 0x7f110806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.174+0000 7f11037fe700 1 -- 172.21.15.97:0/2388191009 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1114089cb0 con 0x7f111c15e1e0 2023-11-24T06:11:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.174+0000 7f111affd700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f110806b2c0 0x7f110806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.174+0000 7f111affd700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f110806b2c0 0x7f110806d780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1110000c00 tx=0x7f1110008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.178+0000 7f11037fe700 1 -- 172.21.15.97:0/2388191009 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11140547d0 con 0x7f111c15e1e0 2023-11-24T06:11:23.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.410+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f111c0b5640 con 0x7f111c15e1e0 2023-11-24T06:11:23.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.410+0000 7f11037fe700 1 -- 172.21.15.97:0/2388191009 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f1114057df0 con 0x7f111c15e1e0 2023-11-24T06:11:23.413 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:23.413 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3508641674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3508641674},{"type":"v1","addr":"172.21.15.183:6833","nonce":3508641674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5557,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/116508824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":116508824},{"type":"v1","addr":"172.21.15.183:6835","nonce":116508824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2785455707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2785455707},{"type":"v1","addr":"172.21.15.97:6837","nonce":2785455707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:03.572274+0000","modified":"2023-11-24T06:11:06.595718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6003},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6003":{"gid":6003,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6836/3662495999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3662495999},{"type":"v1","addr":"172.21.15.97:6836","nonce":3662495999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T06:11:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f110806b2c0 msgr2=0x7f110806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f110806b2c0 0x7f110806d780 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1110000c00 tx=0x7f1110008030 comp rx=0 tx=0).stop 2023-11-24T06:11:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 msgr2=0x7f111c1f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c1f3980 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f111400cc80 tx=0x7f111400cf90 comp rx=0 tx=0).stop 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 shutdown_connections 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c15e1e0 0x7f111c1f3980 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c0d0db0 0x7f111c0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f110806b2c0 0x7f110806d780 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 --2- 172.21.15.97:0/2388191009 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f111c0d3a00 0x7f111c0c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 >> 172.21.15.97:0/2388191009 conn(0x7f111c0b5b50 msgr2=0x7f111c0b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 shutdown_connections 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.412+0000 7f11212f6700 1 -- 172.21.15.97:0/2388191009 wait complete. 2023-11-24T06:11:23.416 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 37 2023-11-24T06:11:23.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:23.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.544+0000 7efdad1e5700 1 Processor -- start 2023-11-24T06:11:23.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.544+0000 7efdad1e5700 1 -- start start 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efdad1e5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efda815bdd0 0x7efda81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efdad1e5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda80b3370 0x7efda80b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efdad1e5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda80b2030 con 0x7efda80d1c70 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efdad1e5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80b2f50 con 0x7efda815bdd0 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efdad1e5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda80b22d0 con 0x7efda80b3370 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efda659c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda80b3370 0x7efda80b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efda6d9d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efda815bdd0 0x7efda81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efda6d9d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7efda815bdd0 0x7efda81604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:43858/0 (socket says 172.21.15.97:43858) 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efda759e700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7efda80d1c70 0x7efda80ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51476/0 (socket says 172.21.15.97:51476) 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efda659c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda80b3370 0x7efda80b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:36734/0 (socket says 172.21.15.97:36734) 2023-11-24T06:11:23.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.545+0000 7efda6d9d700 1 -- 172.21.15.97:0/1479981124 learned_addr learned my addr 172.21.15.97:0/1479981124 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda659c700 1 -- 172.21.15.97:0/1479981124 >> v2:172.21.15.183:3300/0 conn(0x7efda815bdd0 msgr2=0x7efda81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda5d9b700 1 -- 172.21.15.97:0/1479981124 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877737252 0 0) 0x7efda80b2030 con 0x7efda80d1c70 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda659c700 1 --2- 172.21.15.97:0/1479981124 >> v2:172.21.15.183:3300/0 conn(0x7efda815bdd0 0x7efda81604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda659c700 1 -- 172.21.15.97:0/1479981124 >> v1:172.21.15.183:6789/0 conn(0x7efda80d1c70 legacy=0x7efda80ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda659c700 1 -- 172.21.15.97:0/1479981124 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80b25b0 con 0x7efda80b3370 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda659c700 1 --2- 172.21.15.97:0/1479981124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda80b3370 0x7efda80b1af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009840 tx=0x7efd9c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c79bf336809f46e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda5d9b700 1 -- 172.21.15.97:0/1479981124 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c011830 con 0x7efda80b3370 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda5d9b700 1 -- 172.21.15.97:0/1479981124 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c011990 con 0x7efda80b3370 2023-11-24T06:11:23.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efda5d9b700 1 -- 172.21.15.97:0/1479981124 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c011d20 con 0x7efda80b3370 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efdad1e5700 1 -- 172.21.15.97:0/1479981124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda80b3370 msgr2=0x7efda80b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.546+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1479981124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda80b3370 0x7efda80b1af0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009840 tx=0x7efd9c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.547+0000 7efdad1e5700 1 -- 172.21.15.97:0/1479981124 shutdown_connections 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.547+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1479981124 >> v2:172.21.15.183:3300/0 conn(0x7efda815bdd0 0x7efda81604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.547+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1479981124 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda80b3370 0x7efda80b1af0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.547+0000 7efdad1e5700 1 -- 172.21.15.97:0/1479981124 >> 172.21.15.97:0/1479981124 conn(0x7efda80731a0 msgr2=0x7efda809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.547+0000 7efdad1e5700 1 -- 172.21.15.97:0/1479981124 shutdown_connections 2023-11-24T06:11:23.552 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.547+0000 7efdad1e5700 1 -- 172.21.15.97:0/1479981124 wait complete. 2023-11-24T06:11:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.548+0000 7efdad1e5700 1 Processor -- start 2023-11-24T06:11:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.548+0000 7efdad1e5700 1 -- start start 2023-11-24T06:11:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efdad1e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda80b3370 0x7efda815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.555 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efdad1e5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda80d1c70 0x7efda80c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efdad1e5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda815bdd0 0x7efda80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efdad1e5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda815cfd0 con 0x7efda80b3370 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efdad1e5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80ce7f0 con 0x7efda80d1c70 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efdad1e5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda80cead0 con 0x7efda815bdd0 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda80b3370 0x7efda815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda80b3370 0x7efda815d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49704/0 (socket says 172.21.15.97:49704) 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 -- 172.21.15.97:0/1141122956 learned_addr learned my addr 172.21.15.97:0/1141122956 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda759e700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda815bdd0 0x7efda80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda659c700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda80d1c70 0x7efda80c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.556 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 -- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda815bdd0 msgr2=0x7efda80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda815bdd0 0x7efda80c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 -- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda80d1c70 msgr2=0x7efda80c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda80d1c70 0x7efda80c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 -- 172.21.15.97:0/1141122956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0094a0 con 0x7efda80b3370 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda659c700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda80d1c70 0x7efda80c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.549+0000 7efda6d9d700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda80b3370 0x7efda815d900 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efd9800e8b0 tx=0x7efd9800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.550+0000 7efd8f7fe700 1 -- 172.21.15.97:0/1141122956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98009600 con 0x7efda80b3370 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.550+0000 7efd8f7fe700 1 -- 172.21.15.97:0/1141122956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98013680 con 0x7efda80b3370 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.550+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda80cedb0 con 0x7efda80b3370 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.550+0000 7efd8f7fe700 1 -- 172.21.15.97:0/1141122956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98013a30 con 0x7efda80b3370 2023-11-24T06:11:23.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.550+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda80cf370 con 0x7efda80b3370 2023-11-24T06:11:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.551+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda8067f60 con 0x7efda80b3370 2023-11-24T06:11:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.551+0000 7efd8f7fe700 1 -- 172.21.15.97:0/1141122956 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7efd98013b90 con 0x7efda80b3370 2023-11-24T06:11:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.551+0000 7efd8f7fe700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd9406b2c0 0x7efd9406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.551+0000 7efd8f7fe700 1 -- 172.21.15.97:0/1141122956 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7efd9808a090 con 0x7efda80b3370 2023-11-24T06:11:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.551+0000 7efda659c700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd9406b2c0 0x7efd9406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.552+0000 7efda659c700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd9406b2c0 0x7efd9406d780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0090b0 tx=0x7efd9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.556+0000 7efd8f7fe700 1 -- 172.21.15.97:0/1141122956 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd98054a90 con 0x7efda80b3370 2023-11-24T06:11:23.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.744+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efda80d0e30 con 0x7efda80b3370 2023-11-24T06:11:23.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.745+0000 7efd8f7fe700 1 -- 172.21.15.97:0/1141122956 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16685 (secure 0 0 0) 0x7efda80d0e30 con 0x7efda80b3370 2023-11-24T06:11:23.748 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:23.748 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":61,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:19.547545+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:01.950100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:02.526161+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:23.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd9406b2c0 msgr2=0x7efd9406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd9406b2c0 0x7efd9406d780 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0090b0 tx=0x7efd9c008030 comp rx=0 tx=0).stop 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda80b3370 msgr2=0x7efda815d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda80b3370 0x7efda815d900 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efd9800e8b0 tx=0x7efd9800ebc0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 shutdown_connections 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efd9406b2c0 0x7efd9406d780 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7efda815bdd0 0x7efda80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda80b3370 0x7efda815d900 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 --2- 172.21.15.97:0/1141122956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efda80d1c70 0x7efda80c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 >> 172.21.15.97:0/1141122956 conn(0x7efda80731a0 msgr2=0x7efda806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 shutdown_connections 2023-11-24T06:11:23.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.747+0000 7efdad1e5700 1 -- 172.21.15.97:0/1141122956 wait complete. 2023-11-24T06:11:23.762 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:11:23.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.881+0000 7f356ace7700 1 Processor -- start 2023-11-24T06:11:23.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.881+0000 7f356ace7700 1 -- start start 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f356ace7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f356415e1b0 0x7f35640b1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f356ace7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35640b1d40 con 0x7f356415ddd0 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f356ace7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35640b28c0 con 0x7f35640d2240 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f356ace7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35640b1f90 con 0x7f356415e1b0 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f3568a83700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f356415ddd0 0x7f35640d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42164/0 (socket says 172.21.15.97:42164) 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f3568a83700 1 -- 172.21.15.97:0/870795710 learned_addr learned my addr 172.21.15.97:0/870795710 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f3569284700 1 --2- 172.21.15.97:0/870795710 >> v2:172.21.15.183:3300/0 conn(0x7f356415e1b0 0x7f35640b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706375315 0 0) 0x7f35640b1d40 con 0x7f356415ddd0 2023-11-24T06:11:23.887 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35540035b0 con 0x7f356415ddd0 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1853843961 0 0) 0x7f35540035b0 con 0x7f356415ddd0 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 >> v2:172.21.15.183:3300/0 conn(0x7f356415e1b0 msgr2=0x7f35640b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 --2- 172.21.15.97:0/870795710 >> v2:172.21.15.183:3300/0 conn(0x7f356415e1b0 0x7f35640b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 >> v1:172.21.15.183:6789/0 conn(0x7f35640d2240 legacy=0x7f356406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640b2270 con 0x7f356415ddd0 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f3558001670 con 0x7f356415ddd0 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 >> v1:172.21.15.97:6789/0 conn(0x7f356415ddd0 legacy=0x7f35640d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3554003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554004850 0x7f3554004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3554005330 0x7f355400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35640b1f90 con 0x7f3554003a00 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355400c200 con 0x7f3554004850 2023-11-24T06:11:23.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355400c3c0 con 0x7f3554005330 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.882+0000 7f3568a83700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3554003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3563fff700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554004850 0x7f3554004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3569284700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3554005330 0x7f355400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3568a83700 1 -- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3554005330 msgr2=0x7f355400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3568a83700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3554005330 0x7f355400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3568a83700 1 -- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554004850 msgr2=0x7f3554004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3568a83700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554004850 0x7f3554004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3568a83700 1 -- 172.21.15.97:0/870795710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35540035b0 con 0x7f3554003a00 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3569284700 1 -- 172.21.15.97:0/870795710 reap_dead start 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3563fff700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3554004850 0x7f3554004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f3568a83700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3554003e20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f355800c210 tx=0x7f3554005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde6c39ebad25d11 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558007c10 con 0x7f3554003a00 2023-11-24T06:11:23.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355800cd50 con 0x7f3554003a00 2023-11-24T06:11:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f35637fe700 1 -- 172.21.15.97:0/870795710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558005ba0 con 0x7f3554003a00 2023-11-24T06:11:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f356ace7700 1 -- 172.21.15.97:0/870795710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 msgr2=0x7f3554003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f356ace7700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3554003e20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f355800c210 tx=0x7f3554005880 comp rx=0 tx=0).stop 2023-11-24T06:11:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f356ace7700 1 -- 172.21.15.97:0/870795710 shutdown_connections 2023-11-24T06:11:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f356ace7700 1 --2- 172.21.15.97:0/870795710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3554003e20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.883+0000 7f356ace7700 1 -- 172.21.15.97:0/870795710 >> 172.21.15.97:0/870795710 conn(0x7f35640731a0 msgr2=0x7f356409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.884+0000 7f356ace7700 1 -- 172.21.15.97:0/870795710 shutdown_connections 2023-11-24T06:11:23.890 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.884+0000 7f356ace7700 1 -- 172.21.15.97:0/870795710 wait complete. 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.885+0000 7f356ace7700 1 Processor -- start 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.885+0000 7f356ace7700 1 -- start start 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f356ace7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3564159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f356ace7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3564161020 0x7f35641f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f356ace7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35641f7590 0x7f35641f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f356ace7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356415a010 con 0x7f3554003a00 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f356ace7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356415a2d0 con 0x7f3564161020 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f356ace7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356415a5b0 con 0x7f35641f7590 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3564159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3564159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49724/0 (socket says 172.21.15.97:49724) 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 -- 172.21.15.97:0/1618744736 learned_addr learned my addr 172.21.15.97:0/1618744736 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:23.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3569284700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35641f7590 0x7f35641f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3563fff700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3564161020 0x7f35641f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 -- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35641f7590 msgr2=0x7f35641f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35641f7590 0x7f35641f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 -- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3564161020 msgr2=0x7f35641f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3564161020 0x7f35641f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 -- 172.21.15.97:0/1618744736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35640b2ed0 con 0x7f3554003a00 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3569284700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35641f7590 0x7f35641f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f3568a83700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3564159ad0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f355400c1d0 tx=0x7f3558001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f35617fa700 1 -- 172.21.15.97:0/1618744736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558004c80 con 0x7f3554003a00 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f35617fa700 1 -- 172.21.15.97:0/1618744736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3558004de0 con 0x7f3554003a00 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356415a890 con 0x7f3554003a00 2023-11-24T06:11:23.894 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f35617fa700 1 -- 172.21.15.97:0/1618744736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558007310 con 0x7f3554003a00 2023-11-24T06:11:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.886+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356415ae50 con 0x7f3554003a00 2023-11-24T06:11:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.887+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356415b010 con 0x7f3554003a00 2023-11-24T06:11:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.888+0000 7f35617fa700 1 -- 172.21.15.97:0/1618744736 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3558007470 con 0x7f3554003a00 2023-11-24T06:11:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.888+0000 7f35617fa700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f354806b2c0 0x7f354806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.888+0000 7f35617fa700 1 -- 172.21.15.97:0/1618744736 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f355808a5f0 con 0x7f3554003a00 2023-11-24T06:11:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.888+0000 7f3563fff700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f354806b2c0 0x7f354806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.889+0000 7f3563fff700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f354806b2c0 0x7f354806d780 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3550000c00 tx=0x7f3550009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:23.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:23.893+0000 7f35617fa700 1 -- 172.21.15.97:0/1618744736 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3558055110 con 0x7f3554003a00 2023-11-24T06:11:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.127+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3564067f60 con 0x7f3554003a00 2023-11-24T06:11:24.130 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-11-24T06:11:24.127+0000 7fbb14a83700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:11:24.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.129+0000 7f35617fa700 1 -- 172.21.15.97:0/1618744736 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3558058730 con 0x7f3554003a00 2023-11-24T06:11:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.131+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f354806b2c0 msgr2=0x7f354806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.131+0000 7f356ace7700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f354806b2c0 0x7f354806d780 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3550000c00 tx=0x7f3550009040 comp rx=0 tx=0).stop 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.131+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 msgr2=0x7f3564159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.131+0000 7f356ace7700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3564159ad0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f355400c1d0 tx=0x7f3558001a00 comp rx=0 tx=0).stop 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.132+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 shutdown_connections 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.132+0000 7f356ace7700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f354806b2c0 0x7f354806d780 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.132+0000 7f356ace7700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f35641f7590 0x7f35641f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.132+0000 7f356ace7700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3554003a00 0x7f3564159ad0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.132+0000 7f356ace7700 1 --2- 172.21.15.97:0/1618744736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3564161020 0x7f35641f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.132+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 >> 172.21.15.97:0/1618744736 conn(0x7f35640731a0 msgr2=0x7f35640cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.132+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 shutdown_connections 2023-11-24T06:11:24.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.132+0000 7f356ace7700 1 -- 172.21.15.97:0/1618744736 wait complete. 2023-11-24T06:11:24.136 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:11:24.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:11:24.173 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-11-24T06:11:24.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.269+0000 7ff4ceee4700 1 Processor -- start 2023-11-24T06:11:24.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.269+0000 7ff4ceee4700 1 -- start start 2023-11-24T06:11:24.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.269+0000 7ff4ceee4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4c80af520 0x7ff4c80afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.269+0000 7ff4ceee4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4c80b0070 0x7ff4c8073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.269+0000 7ff4ceee4700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c815bf70 con 0x7ff4c80b4020 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.269+0000 7ff4ceee4700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c815ee00 con 0x7ff4c80af520 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.269+0000 7ff4ceee4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c815e4f0 con 0x7ff4c80b0070 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4ccc80700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4c80af520 0x7ff4c80afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4ccc80700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4c80af520 0x7ff4c80afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49744/0 (socket says 172.21.15.97:49744) 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4cd481700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff4c80b4020 0x7ff4c8161ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42200/0 (socket says 172.21.15.97:42200) 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4ccc80700 1 -- 172.21.15.97:0/2019639817 learned_addr learned my addr 172.21.15.97:0/2019639817 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4ccc80700 1 -- 172.21.15.97:0/2019639817 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4c80b0070 msgr2=0x7ff4c8073f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4ccc80700 1 --2- 172.21.15.97:0/2019639817 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4c80b0070 0x7ff4c8073f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4ccc80700 1 -- 172.21.15.97:0/2019639817 >> v1:172.21.15.97:6789/0 conn(0x7ff4c80b4020 legacy=0x7ff4c8161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4ccc80700 1 -- 172.21.15.97:0/2019639817 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80b04f0 con 0x7ff4c80af520 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4c77fe700 1 -- 172.21.15.97:0/2019639817 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3072725760 0 0) 0x7ff4c0001490 con 0x7ff4c80b4020 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4ccc80700 1 --2- 172.21.15.97:0/2019639817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c80afb30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0098f0 tx=0x7ff4bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17759485f32abaf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4c77fe700 1 -- 172.21.15.97:0/2019639817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00b8a0 con 0x7ff4c80af520 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4c77fe700 1 -- 172.21.15.97:0/2019639817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc00ba00 con 0x7ff4c80af520 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.270+0000 7ff4c77fe700 1 -- 172.21.15.97:0/2019639817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc00bd90 con 0x7ff4c80af520 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.271+0000 7ff4ceee4700 1 -- 172.21.15.97:0/2019639817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 msgr2=0x7ff4c80afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.271+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/2019639817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c80afb30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc0098f0 tx=0x7ff4bc009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.271+0000 7ff4ceee4700 1 -- 172.21.15.97:0/2019639817 shutdown_connections 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.271+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/2019639817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c80afb30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.271+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/2019639817 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4c80b0070 0x7ff4c8073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.271+0000 7ff4ceee4700 1 -- 172.21.15.97:0/2019639817 >> 172.21.15.97:0/2019639817 conn(0x7ff4c80b28f0 msgr2=0x7ff4c80b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:24.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.271+0000 7ff4ceee4700 1 -- 172.21.15.97:0/2019639817 shutdown_connections 2023-11-24T06:11:24.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.271+0000 7ff4ceee4700 1 -- 172.21.15.97:0/2019639817 wait complete. 2023-11-24T06:11:24.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.272+0000 7ff4ceee4700 1 Processor -- start 2023-11-24T06:11:24.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.272+0000 7ff4ceee4700 1 -- start start 2023-11-24T06:11:24.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ceee4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c814a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ceee4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4c80b0070 0x7ff4c8159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ceee4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c80b4020 0x7ff4c80b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ceee4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8158bd0 con 0x7ff4c80af520 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ceee4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8158050 con 0x7ff4c80b4020 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ceee4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c8158330 con 0x7ff4c80b0070 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c814a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c814a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49750/0 (socket says 172.21.15.97:49750) 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 -- 172.21.15.97:0/757988054 learned_addr learned my addr 172.21.15.97:0/757988054 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 -- 172.21.15.97:0/757988054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4c80b0070 msgr2=0x7ff4c8159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4c80b0070 0x7ff4c8159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 -- 172.21.15.97:0/757988054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c80b4020 msgr2=0x7ff4c80b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c80b4020 0x7ff4c80b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:24.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 -- 172.21.15.97:0/757988054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc009550 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ccc80700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c814a7d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009c40 tx=0x7ff4bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4c57fa700 1 -- 172.21.15.97:0/757988054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0044c0 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4c57fa700 1 -- 172.21.15.97:0/757988054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4bc01f070 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c8158610 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4c57fa700 1 -- 172.21.15.97:0/757988054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0038c0 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.273+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c8159890 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.274+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c8159a80 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.275+0000 7ff4c57fa700 1 -- 172.21.15.97:0/757988054 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff4bc021030 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.275+0000 7ff4c57fa700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4ac06b2c0 0x7ff4ac06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.275+0000 7ff4c57fa700 1 -- 172.21.15.97:0/757988054 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5823+0+0 (secure 0 0 0) 0x7ff4bc08b1d0 con 0x7ff4c80af520 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.275+0000 7ff4c7fff700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4ac06b2c0 0x7ff4ac06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:24.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.276+0000 7ff4c7fff700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4ac06b2c0 0x7ff4ac06d780 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4c814ad10 tx=0x7ff4b4006cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:24.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.280+0000 7ff4c57fa700 1 -- 172.21.15.97:0/757988054 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4bc055ef0 con 0x7ff4c80af520 2023-11-24T06:11:24.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:11:24.342 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:11:24.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:24.521+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff4c815f6a0 con 0x7ff4c80af520 2023-11-24T06:11:25.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.136+0000 7ff4c57fa700 1 -- 172.21.15.97:0/757988054 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff4c815f6a0 con 0x7ff4c80af520 2023-11-24T06:11:25.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4ac06b2c0 msgr2=0x7ff4ac06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4ac06b2c0 0x7ff4ac06d780 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff4c814ad10 tx=0x7ff4b4006cd0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 msgr2=0x7ff4c814a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c814a7d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009c40 tx=0x7ff4bc004750 comp rx=0 tx=0).stop 2023-11-24T06:11:25.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 shutdown_connections 2023-11-24T06:11:25.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff4c80b4020 0x7ff4c80b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4c80af520 0x7ff4c814a7d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff4ac06b2c0 0x7ff4ac06d780 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 --2- 172.21.15.97:0/757988054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff4c80b0070 0x7ff4c8159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 >> 172.21.15.97:0/757988054 conn(0x7ff4c80b28f0 msgr2=0x7ff4c815b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:25.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 shutdown_connections 2023-11-24T06:11:25.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.139+0000 7ff4ceee4700 1 -- 172.21.15.97:0/757988054 wait complete. 2023-11-24T06:11:25.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:11:25.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.284+0000 7fc93edb8700 1 Processor -- start 2023-11-24T06:11:25.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.284+0000 7fc93edb8700 1 -- start start 2023-11-24T06:11:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.285+0000 7fc93edb8700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc9380b01e0 0x7fc93806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.285+0000 7fc93edb8700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380b05c0 con 0x7fc9380b59c0 2023-11-24T06:11:25.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.285+0000 7fc93edb8700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380b4250 con 0x7fc9380746a0 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.285+0000 7fc93edb8700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9381470a0 con 0x7fc9380b01e0 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.285+0000 7fc937fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc9380b59c0 0x7fc938073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42212/0 (socket says 172.21.15.97:42212) 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.285+0000 7fc937fff700 1 -- 172.21.15.97:0/1725297531 learned_addr learned my addr 172.21.15.97:0/1725297531 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.285+0000 7fc93cb54700 1 --2- 172.21.15.97:0/1725297531 >> v2:172.21.15.183:3300/0 conn(0x7fc9380b01e0 0x7fc93806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.285+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772578280 0 0) 0x7fc9380b05c0 con 0x7fc9380b59c0 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9280035b0 con 0x7fc9380b59c0 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694158478 0 0) 0x7fc9280035b0 con 0x7fc9380b59c0 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 >> v2:172.21.15.183:3300/0 conn(0x7fc9380b01e0 msgr2=0x7fc93806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 --2- 172.21.15.97:0/1725297531 >> v2:172.21.15.183:3300/0 conn(0x7fc9380b01e0 0x7fc93806e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 >> v1:172.21.15.183:6789/0 conn(0x7fc9380746a0 legacy=0x7fc93806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93806dc00 con 0x7fc9380b59c0 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fc924001670 con 0x7fc9380b59c0 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 >> v1:172.21.15.97:6789/0 conn(0x7fc9380b59c0 legacy=0x7fc938073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc928003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc928004850 0x7fc928004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928005210 0x7fc92800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9381470a0 con 0x7fc928003a00 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92800c1b0 con 0x7fc928005210 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92800c370 con 0x7fc928004850 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93cb54700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc928003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93d355700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928005210 0x7fc92800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc937fff700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc928004850 0x7fc928004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93cb54700 1 -- 172.21.15.97:0/1725297531 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc928004850 msgr2=0x7fc928004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93cb54700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc928004850 0x7fc928004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93cb54700 1 -- 172.21.15.97:0/1725297531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928005210 msgr2=0x7fc92800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93cb54700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc928005210 0x7fc92800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93d355700 1 -- 172.21.15.97:0/1725297531 reap_dead start 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93cb54700 1 -- 172.21.15.97:0/1725297531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9280035b0 con 0x7fc928003a00 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc93cb54700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc928003e20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009b60 tx=0x7fc928005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbe7c6708bb7991 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.286+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00e6c0 con 0x7fc928003a00 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c00e820 con 0x7fc928003a00 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc9377fe700 1 -- 172.21.15.97:0/1725297531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00eb90 con 0x7fc928003a00 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc93edb8700 1 -- 172.21.15.97:0/1725297531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 msgr2=0x7fc928003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc93edb8700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc928003e20 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009b60 tx=0x7fc928005880 comp rx=0 tx=0).stop 2023-11-24T06:11:25.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc93edb8700 1 -- 172.21.15.97:0/1725297531 shutdown_connections 2023-11-24T06:11:25.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc93edb8700 1 --2- 172.21.15.97:0/1725297531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc928003e20 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc93edb8700 1 -- 172.21.15.97:0/1725297531 >> 172.21.15.97:0/1725297531 conn(0x7fc9380d0b40 msgr2=0x7fc9380d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:25.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc93edb8700 1 -- 172.21.15.97:0/1725297531 shutdown_connections 2023-11-24T06:11:25.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.287+0000 7fc93edb8700 1 -- 172.21.15.97:0/1725297531 wait complete. 2023-11-24T06:11:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.288+0000 7fc93edb8700 1 Processor -- start 2023-11-24T06:11:25.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.288+0000 7fc93edb8700 1 -- start start 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93edb8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc938154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93edb8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9381546b0 0x7fc9380c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93edb8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9380c6000 0x7fc93813d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93edb8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380c6480 con 0x7fc928003a00 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93edb8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93813e190 con 0x7fc9380c6000 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93edb8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93813e470 con 0x7fc9381546b0 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93cb54700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc938154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93cb54700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc938154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49772/0 (socket says 172.21.15.97:49772) 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93cb54700 1 -- 172.21.15.97:0/2657504808 learned_addr learned my addr 172.21.15.97:0/2657504808 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc937fff700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9381546b0 0x7fc9380c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93d355700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9380c6000 0x7fc93813d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.289+0000 7fc93cb54700 1 -- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9381546b0 msgr2=0x7fc9380c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc93cb54700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9381546b0 0x7fc9380c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc93cb54700 1 -- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9380c6000 msgr2=0x7fc93813d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc93cb54700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9380c6000 0x7fc93813d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc93cb54700 1 -- 172.21.15.97:0/2657504808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93813e750 con 0x7fc928003a00 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc93d355700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9380c6000 0x7fc93813d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc93cb54700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc938154170 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c180 tx=0x7fc92c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc9357fa700 1 -- 172.21.15.97:0/2657504808 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c00e6c0 con 0x7fc928003a00 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc9357fa700 1 -- 172.21.15.97:0/2657504808 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c003ad0 con 0x7fc928003a00 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93813ea30 con 0x7fc928003a00 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc9357fa700 1 -- 172.21.15.97:0/2657504808 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c003e80 con 0x7fc928003a00 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.290+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc938153c50 con 0x7fc928003a00 2023-11-24T06:11:25.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.291+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc938067f60 con 0x7fc928003a00 2023-11-24T06:11:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.291+0000 7fc9357fa700 1 -- 172.21.15.97:0/2657504808 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fc92c013470 con 0x7fc928003a00 2023-11-24T06:11:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.292+0000 7fc9357fa700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc91c06b2c0 0x7fc91c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.292+0000 7fc9357fa700 1 -- 172.21.15.97:0/2657504808 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fc92c08b520 con 0x7fc928003a00 2023-11-24T06:11:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.292+0000 7fc937fff700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc91c06b2c0 0x7fc91c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.292+0000 7fc937fff700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc91c06b2c0 0x7fc91c06d780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc924001eb0 tx=0x7fc924009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:25.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.296+0000 7fc9357fa700 1 -- 172.21.15.97:0/2657504808 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc92c056010 con 0x7fc928003a00 2023-11-24T06:11:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:25.490+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc93806c0b0 con 0x7fc928003a00 2023-11-24T06:11:26.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.137+0000 7fc9357fa700 1 -- 172.21.15.97:0/2657504808 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc92c059630 con 0x7fc928003a00 2023-11-24T06:11:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc91c06b2c0 msgr2=0x7fc91c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc91c06b2c0 0x7fc91c06d780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc924001eb0 tx=0x7fc924009040 comp rx=0 tx=0).stop 2023-11-24T06:11:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 msgr2=0x7fc938154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc938154170 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c180 tx=0x7fc92c0047c0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 shutdown_connections 2023-11-24T06:11:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9380c6000 0x7fc93813d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc928003a00 0x7fc938154170 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc91c06b2c0 0x7fc91c06d780 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 --2- 172.21.15.97:0/2657504808 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc9381546b0 0x7fc9380c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 >> 172.21.15.97:0/2657504808 conn(0x7fc9380d0b40 msgr2=0x7fc9380aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 shutdown_connections 2023-11-24T06:11:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.141+0000 7fc93edb8700 1 -- 172.21.15.97:0/2657504808 wait complete. 2023-11-24T06:11:26.145 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:11:26.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:11:26.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.287+0000 7fbefaac9700 1 Processor -- start 2023-11-24T06:11:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.287+0000 7fbefaac9700 1 -- start start 2023-11-24T06:11:26.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.288+0000 7fbefaac9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbef40cfff0 0x7fbef40d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.288+0000 7fbefaac9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbef40ce6a0 0x7fbef40ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.288+0000 7fbefaac9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40ae9f0 con 0x7fbef415be40 2023-11-24T06:11:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.288+0000 7fbefaac9700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef40aefb0 con 0x7fbef40cfff0 2023-11-24T06:11:26.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.288+0000 7fbefaac9700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef40d0970 con 0x7fbef40ce6a0 2023-11-24T06:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.288+0000 7fbef3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbef40cfff0 0x7fbef40d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.288+0000 7fbef3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbef40cfff0 0x7fbef40d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49780/0 (socket says 172.21.15.97:49780) 2023-11-24T06:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.288+0000 7fbef3fff700 1 -- 172.21.15.97:0/3151422821 learned_addr learned my addr 172.21.15.97:0/3151422821 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.289+0000 7fbef9066700 1 --2- 172.21.15.97:0/3151422821 >> v2:172.21.15.183:3300/0 conn(0x7fbef40ce6a0 0x7fbef40ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.289+0000 7fbef9066700 1 -- 172.21.15.97:0/3151422821 >> v2:172.21.15.97:3300/0 conn(0x7fbef40cfff0 msgr2=0x7fbef40d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.289+0000 7fbef9066700 1 --2- 172.21.15.97:0/3151422821 >> v2:172.21.15.97:3300/0 conn(0x7fbef40cfff0 0x7fbef40d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.289+0000 7fbef9066700 1 -- 172.21.15.97:0/3151422821 >> v1:172.21.15.97:6789/0 conn(0x7fbef415be40 legacy=0x7fbef40b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:26.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.289+0000 7fbef9066700 1 -- 172.21.15.97:0/3151422821 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40b3120 con 0x7fbef40ce6a0 2023-11-24T06:11:26.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.289+0000 7fbef9066700 1 --2- 172.21.15.97:0/3151422821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40ce6a0 0x7fbef40ceb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbef40d28f0 tx=0x7fbeec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a4dad287a2a9e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.289+0000 7fbef37fe700 1 -- 172.21.15.97:0/3151422821 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00e070 con 0x7fbef40ce6a0 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbef37fe700 1 -- 172.21.15.97:0/3151422821 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec00bae0 con 0x7fbef40ce6a0 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbef37fe700 1 -- 172.21.15.97:0/3151422821 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00be90 con 0x7fbef40ce6a0 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbefaac9700 1 -- 172.21.15.97:0/3151422821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40ce6a0 msgr2=0x7fbef40ceb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbefaac9700 1 --2- 172.21.15.97:0/3151422821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40ce6a0 0x7fbef40ceb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbef40d28f0 tx=0x7fbeec00b330 comp rx=0 tx=0).stop 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbefaac9700 1 -- 172.21.15.97:0/3151422821 shutdown_connections 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbefaac9700 1 --2- 172.21.15.97:0/3151422821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40ce6a0 0x7fbef40ceb10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbefaac9700 1 --2- 172.21.15.97:0/3151422821 >> v2:172.21.15.97:3300/0 conn(0x7fbef40cfff0 0x7fbef40d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbefaac9700 1 -- 172.21.15.97:0/3151422821 >> 172.21.15.97:0/3151422821 conn(0x7fbef409aa00 msgr2=0x7fbef409ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbefaac9700 1 -- 172.21.15.97:0/3151422821 shutdown_connections 2023-11-24T06:11:26.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.290+0000 7fbefaac9700 1 -- 172.21.15.97:0/3151422821 wait complete. 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.292+0000 7fbefaac9700 1 Processor -- start 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.292+0000 7fbefaac9700 1 -- start start 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.292+0000 7fbefaac9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbef40ce6a0 0x7fbef415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.292+0000 7fbefaac9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40cfff0 0x7fbef406c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.292+0000 7fbefaac9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef415be40 0x7fbef415e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.292+0000 7fbefaac9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4157f20 con 0x7fbef415be40 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.292+0000 7fbefaac9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c4920 con 0x7fbef40cfff0 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.292+0000 7fbefaac9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c4c00 con 0x7fbef40ce6a0 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef415be40 0x7fbef415e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef415be40 0x7fbef415e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49786/0 (socket says 172.21.15.97:49786) 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 -- 172.21.15.97:0/1075574901 learned_addr learned my addr 172.21.15.97:0/1075574901 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef3fff700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40cfff0 0x7fbef406c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef8865700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbef40ce6a0 0x7fbef415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 -- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbef40ce6a0 msgr2=0x7fbef415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbef40ce6a0 0x7fbef415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 -- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40cfff0 msgr2=0x7fbef406c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40cfff0 0x7fbef406c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 -- 172.21.15.97:0/1075574901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec009a70 con 0x7fbef415be40 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef3fff700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40cfff0 0x7fbef406c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:26.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef9066700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef415be40 0x7fbef415e660 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbef40d0ab0 tx=0x7fbeec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbef17fa700 1 -- 172.21.15.97:0/1075574901 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004720 con 0x7fbef415be40 2023-11-24T06:11:26.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.293+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef41578d0 con 0x7fbef415be40 2023-11-24T06:11:26.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.294+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef406cf00 con 0x7fbef415be40 2023-11-24T06:11:26.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.294+0000 7fbef17fa700 1 -- 172.21.15.97:0/1075574901 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec004880 con 0x7fbef415be40 2023-11-24T06:11:26.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.294+0000 7fbef17fa700 1 -- 172.21.15.97:0/1075574901 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec012670 con 0x7fbef415be40 2023-11-24T06:11:26.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.295+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed40052e0 con 0x7fbef415be40 2023-11-24T06:11:26.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.295+0000 7fbef17fa700 1 -- 172.21.15.97:0/1075574901 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbeec0127d0 con 0x7fbef415be40 2023-11-24T06:11:26.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.296+0000 7fbef17fa700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbed806b2c0 0x7fbed806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:26.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.296+0000 7fbef17fa700 1 -- 172.21.15.97:0/1075574901 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5385+0+0 (secure 0 0 0) 0x7fbeec08b6a0 con 0x7fbef415be40 2023-11-24T06:11:26.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.296+0000 7fbef8865700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbed806b2c0 0x7fbed806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:26.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.296+0000 7fbef8865700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbed806b2c0 0x7fbed806d780 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009870 tx=0x7fbee8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:26.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.300+0000 7fbef17fa700 1 -- 172.21.15.97:0/1075574901 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbeec056120 con 0x7fbef415be40 2023-11-24T06:11:26.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:26.495+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbed4005590 con 0x7fbef415be40 2023-11-24T06:11:27.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.142+0000 7fbef17fa700 1 -- 172.21.15.97:0/1075574901 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbeec059740 con 0x7fbef415be40 2023-11-24T06:11:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbed806b2c0 msgr2=0x7fbed806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbed806b2c0 0x7fbed806d780 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009870 tx=0x7fbee8008040 comp rx=0 tx=0).stop 2023-11-24T06:11:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef415be40 msgr2=0x7fbef415e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef415be40 0x7fbef415e660 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbef40d0ab0 tx=0x7fbeec004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 shutdown_connections 2023-11-24T06:11:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbef415be40 0x7fbef415e660 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbed806b2c0 0x7fbed806d780 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbef40ce6a0 0x7fbef415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 --2- 172.21.15.97:0/1075574901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbef40cfff0 0x7fbef406c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.146+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 >> 172.21.15.97:0/1075574901 conn(0x7fbef409aa00 msgr2=0x7fbef40af900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:27.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.147+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 shutdown_connections 2023-11-24T06:11:27.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.147+0000 7fbefaac9700 1 -- 172.21.15.97:0/1075574901 wait complete. 2023-11-24T06:11:27.150 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-11-24T06:11:27.163 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:11:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.284+0000 7fd569bac700 1 Processor -- start 2023-11-24T06:11:27.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.284+0000 7fd569bac700 1 -- start start 2023-11-24T06:11:27.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd569bac700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5640746a0 0x7fd5640b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd569bac700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640afc90 0x7fd56406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd569bac700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640b3c60 con 0x7fd56415e160 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd569bac700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5640b4500 con 0x7fd5640746a0 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd569bac700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640b58c0 con 0x7fd5640afc90 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd562ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5640746a0 0x7fd5640b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd5637fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd56415e160 0x7fd564073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42246/0 (socket says 172.21.15.97:42246) 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd562ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd5640746a0 0x7fd5640b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49796/0 (socket says 172.21.15.97:49796) 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd5637fe700 1 -- 172.21.15.97:0/798425982 learned_addr learned my addr 172.21.15.97:0/798425982 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd563fff700 1 --2- 172.21.15.97:0/798425982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640afc90 0x7fd56406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd562ffd700 1 -- 172.21.15.97:0/798425982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640afc90 msgr2=0x7fd56406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd562ffd700 1 --2- 172.21.15.97:0/798425982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640afc90 0x7fd56406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.290 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.285+0000 7fd562ffd700 1 -- 172.21.15.97:0/798425982 >> v1:172.21.15.97:6789/0 conn(0x7fd56415e160 legacy=0x7fd564073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd562ffd700 1 -- 172.21.15.97:0/798425982 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640b0120 con 0x7fd5640746a0 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd5627fc700 1 -- 172.21.15.97:0/798425982 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394879 0 0) 0x7fd5640b3c60 con 0x7fd56415e160 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd563fff700 1 --2- 172.21.15.97:0/798425982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640afc90 0x7fd56406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd562ffd700 1 --2- 172.21.15.97:0/798425982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5640746a0 0x7fd5640b5da0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd554009840 tx=0x7fd554009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e05a4cd511f900 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd5627fc700 1 -- 172.21.15.97:0/798425982 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554011830 con 0x7fd5640746a0 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd5627fc700 1 -- 172.21.15.97:0/798425982 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd554011990 con 0x7fd5640746a0 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd5627fc700 1 -- 172.21.15.97:0/798425982 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554011d20 con 0x7fd5640746a0 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd569bac700 1 -- 172.21.15.97:0/798425982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5640746a0 msgr2=0x7fd5640b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd569bac700 1 --2- 172.21.15.97:0/798425982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5640746a0 0x7fd5640b5da0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd554009840 tx=0x7fd554009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd569bac700 1 -- 172.21.15.97:0/798425982 shutdown_connections 2023-11-24T06:11:27.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd569bac700 1 --2- 172.21.15.97:0/798425982 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640afc90 0x7fd56406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd569bac700 1 --2- 172.21.15.97:0/798425982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5640746a0 0x7fd5640b5da0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd569bac700 1 -- 172.21.15.97:0/798425982 >> 172.21.15.97:0/798425982 conn(0x7fd5640d0bf0 msgr2=0x7fd5640d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:27.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.286+0000 7fd569bac700 1 -- 172.21.15.97:0/798425982 shutdown_connections 2023-11-24T06:11:27.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.287+0000 7fd569bac700 1 -- 172.21.15.97:0/798425982 wait complete. 2023-11-24T06:11:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.288+0000 7fd569bac700 1 Processor -- start 2023-11-24T06:11:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.288+0000 7fd569bac700 1 -- start start 2023-11-24T06:11:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd569bac700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640746a0 0x7fd564144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd569bac700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5640afc90 0x7fd564160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd569bac700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56415e160 0x7fd5640b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd569bac700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564161650 con 0x7fd56415e160 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd569bac700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564160870 con 0x7fd5640afc90 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd569bac700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd564160ad0 con 0x7fd5640746a0 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56415e160 0x7fd5640b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56415e160 0x7fd5640b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49804/0 (socket says 172.21.15.97:49804) 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 -- 172.21.15.97:0/238756880 learned_addr learned my addr 172.21.15.97:0/238756880 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd5637fe700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640746a0 0x7fd564144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd562ffd700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5640afc90 0x7fd564160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 -- 172.21.15.97:0/238756880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640746a0 msgr2=0x7fd564144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640746a0 0x7fd564144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 -- 172.21.15.97:0/238756880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5640afc90 msgr2=0x7fd564160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5640afc90 0x7fd564160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 -- 172.21.15.97:0/238756880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540094a0 con 0x7fd56415e160 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd563fff700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56415e160 0x7fd5640b1b70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00cef0 tx=0x7fd55c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd55bfff700 1 -- 172.21.15.97:0/238756880 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c00ae50 con 0x7fd56415e160 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd55bfff700 1 -- 172.21.15.97:0/238756880 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55c015440 con 0x7fd56415e160 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd55bfff700 1 -- 172.21.15.97:0/238756880 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c0157f0 con 0x7fd56415e160 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd564160db0 con 0x7fd56415e160 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.289+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd564161370 con 0x7fd56415e160 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.290+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56406ba00 con 0x7fd56415e160 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.291+0000 7fd55bfff700 1 -- 172.21.15.97:0/238756880 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd55c00d430 con 0x7fd56415e160 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.291+0000 7fd55bfff700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd54406b2c0 0x7fd54406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.291+0000 7fd55bfff700 1 -- 172.21.15.97:0/238756880 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fd55c08a020 con 0x7fd56415e160 2023-11-24T06:11:27.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.291+0000 7fd5637fe700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd54406b2c0 0x7fd54406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.291+0000 7fd5637fe700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd54406b2c0 0x7fd54406d780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd54c000f60 tx=0x7fd54c007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.295+0000 7fd55bfff700 1 -- 172.21.15.97:0/238756880 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd55c054e60 con 0x7fd56415e160 2023-11-24T06:11:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.495+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd5640d1500 con 0x7fd56415e160 2023-11-24T06:11:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.498+0000 7fd55bfff700 1 -- 172.21.15.97:0/238756880 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd55c01a0a0 con 0x7fd56415e160 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.500+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd54406b2c0 msgr2=0x7fd54406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd54406b2c0 0x7fd54406d780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd54c000f60 tx=0x7fd54c007c90 comp rx=0 tx=0).stop 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56415e160 msgr2=0x7fd5640b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56415e160 0x7fd5640b1b70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00cef0 tx=0x7fd55c00a340 comp rx=0 tx=0).stop 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 shutdown_connections 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56415e160 0x7fd5640b1b70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd54406b2c0 0x7fd54406d780 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5640746a0 0x7fd564144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 --2- 172.21.15.97:0/238756880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5640afc90 0x7fd564160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 >> 172.21.15.97:0/238756880 conn(0x7fd5640d0bf0 msgr2=0x7fd5641619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:27.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 shutdown_connections 2023-11-24T06:11:27.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.501+0000 7fd569bac700 1 -- 172.21.15.97:0/238756880 wait complete. 2023-11-24T06:11:27.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T06:11:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.633+0000 7f0331e36700 1 Processor -- start 2023-11-24T06:11:27.636 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.633+0000 7f0331e36700 1 -- start start 2023-11-24T06:11:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.634+0000 7f0331e36700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c0b0ba0 0x7f032c0af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.634+0000 7f0331e36700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f032c15ee90 0x7f032c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.634+0000 7f0331e36700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032c0b2bd0 con 0x7f032c0b0f80 2023-11-24T06:11:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.634+0000 7f0331e36700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f032c0b1fc0 con 0x7f032c15ee90 2023-11-24T06:11:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.634+0000 7f0331e36700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0b22a0 con 0x7f032c0b0ba0 2023-11-24T06:11:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.634+0000 7f032b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c0b0ba0 0x7f032c0af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032bfff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f032c15ee90 0x7f032c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c0b0ba0 0x7f032c0af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:36844/0 (socket says 172.21.15.97:36844) 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032b7fe700 1 -- 172.21.15.97:0/4094436781 learned_addr learned my addr 172.21.15.97:0/4094436781 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032bfff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f032c15ee90 0x7f032c15b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:43972/0 (socket says 172.21.15.97:43972) 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032affd700 1 --1- 172.21.15.97:0/4094436781 >> v1:172.21.15.183:6789/0 conn(0x7f032c0b0f80 0x7f032c0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51582/0 (socket says 172.21.15.97:51582) 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032a7fc700 1 -- 172.21.15.97:0/4094436781 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195194457 0 0) 0x7f032c0b2bd0 con 0x7f032c0b0f80 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032a7fc700 1 -- 172.21.15.97:0/4094436781 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03180035b0 con 0x7f032c0b0f80 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032bfff700 1 -- 172.21.15.97:0/4094436781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c0b0ba0 msgr2=0x7f032c0af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032bfff700 1 --2- 172.21.15.97:0/4094436781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c0b0ba0 0x7f032c0af5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032bfff700 1 -- 172.21.15.97:0/4094436781 >> v1:172.21.15.183:6789/0 conn(0x7f032c0b0f80 legacy=0x7f032c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032bfff700 1 -- 172.21.15.97:0/4094436781 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c0b2580 con 0x7f032c15ee90 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032b7fe700 1 --2- 172.21.15.97:0/4094436781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c0b0ba0 0x7f032c0af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032bfff700 1 --2- 172.21.15.97:0/4094436781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c15ee90 0x7f032c15b450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0324009d90 tx=0x7f032400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770293ce38632783 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032a7fc700 1 -- 172.21.15.97:0/4094436781 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400e070 con 0x7f032c15ee90 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032a7fc700 1 -- 172.21.15.97:0/4094436781 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032400bb20 con 0x7f032c15ee90 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.635+0000 7f032a7fc700 1 -- 172.21.15.97:0/4094436781 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400be90 con 0x7f032c15ee90 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.636+0000 7f0331e36700 1 -- 172.21.15.97:0/4094436781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c15ee90 msgr2=0x7f032c15b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.636+0000 7f0331e36700 1 --2- 172.21.15.97:0/4094436781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c15ee90 0x7f032c15b450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0324009d90 tx=0x7f032400b330 comp rx=0 tx=0).stop 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.636+0000 7f0331e36700 1 -- 172.21.15.97:0/4094436781 shutdown_connections 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.636+0000 7f0331e36700 1 --2- 172.21.15.97:0/4094436781 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c15ee90 0x7f032c15b450 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.636+0000 7f0331e36700 1 --2- 172.21.15.97:0/4094436781 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c0b0ba0 0x7f032c0af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.636+0000 7f0331e36700 1 -- 172.21.15.97:0/4094436781 >> 172.21.15.97:0/4094436781 conn(0x7f032c0b57b0 msgr2=0x7f032c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.636+0000 7f0331e36700 1 -- 172.21.15.97:0/4094436781 shutdown_connections 2023-11-24T06:11:27.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.636+0000 7f0331e36700 1 -- 172.21.15.97:0/4094436781 wait complete. 2023-11-24T06:11:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.637+0000 7f0331e36700 1 Processor -- start 2023-11-24T06:11:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.638+0000 7f0331e36700 1 -- start start 2023-11-24T06:11:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0331e36700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c0b0ba0 0x7f032c1e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0331e36700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f032c0b0f80 0x7f032c0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0331e36700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c15ee90 0x7f032c0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0331e36700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0b4340 con 0x7f032c0b0f80 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0331e36700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0b4010 con 0x7f032c0b0ba0 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0331e36700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0add20 con 0x7f032c15ee90 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f032c0b0f80 0x7f032c0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f032c0b0f80 0x7f032c0b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49816/0 (socket says 172.21.15.97:49816) 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 -- 172.21.15.97:0/1528375499 learned_addr learned my addr 172.21.15.97:0/1528375499 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032b7fe700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c0b0ba0 0x7f032c1e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032bfff700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c15ee90 0x7f032c0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 -- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c15ee90 msgr2=0x7f032c0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c15ee90 0x7f032c0b34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 -- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c0b0ba0 msgr2=0x7f032c1e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c0b0ba0 0x7f032c1e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:27.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 -- 172.21.15.97:0/1528375499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031400b7f0 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032b7fe700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c0b0ba0 0x7f032c1e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f032affd700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f032c0b0f80 0x7f032c0b2f60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c500 tx=0x7f031c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0323fff700 1 -- 172.21.15.97:0/1528375499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c005cb0 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0323fff700 1 -- 172.21.15.97:0/1528375499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031c005e10 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0323fff700 1 -- 172.21.15.97:0/1528375499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031c007310 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0324009a70 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.639+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032c158d00 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.640+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032c0b44b0 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.641+0000 7f0323fff700 1 -- 172.21.15.97:0/1528375499 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f031c007470 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.641+0000 7f0323fff700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f030c06b2c0 0x7f030c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.641+0000 7f0323fff700 1 -- 172.21.15.97:0/1528375499 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f031c088fc0 con 0x7f032c0b0f80 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.641+0000 7f032b7fe700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f030c06b2c0 0x7f030c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:27.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.642+0000 7f032b7fe700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f030c06b2c0 0x7f030c06d780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f031400d450 tx=0x7f0314008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:27.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.646+0000 7f0323fff700 1 -- 172.21.15.97:0/1528375499 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f031c053e30 con 0x7f032c0b0f80 2023-11-24T06:11:27.917 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:27.914+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7f032c067f60 con 0x7f032c0b0f80 2023-11-24T06:11:28.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.144+0000 7f0323fff700 1 -- 172.21.15.97:0/1528375499 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v165) v1 ==== 166+0+0 (secure 0 0 0) 0x7f031c017020 con 0x7f032c0b0f80 2023-11-24T06:11:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f030c06b2c0 msgr2=0x7f030c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f030c06b2c0 0x7f030c06d780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f031400d450 tx=0x7f0314008040 comp rx=0 tx=0).stop 2023-11-24T06:11:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f032c0b0f80 msgr2=0x7f032c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f032c0b0f80 0x7f032c0b2f60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f031c00c500 tx=0x7f031c00c8c0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 shutdown_connections 2023-11-24T06:11:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f030c06b2c0 0x7f030c06d780 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f032c15ee90 0x7f032c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032c0b0ba0 0x7f032c1e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 --2- 172.21.15.97:0/1528375499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f032c0b0f80 0x7f032c0b2f60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 >> 172.21.15.97:0/1528375499 conn(0x7f032c0b57b0 msgr2=0x7f032c0cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 shutdown_connections 2023-11-24T06:11:28.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.146+0000 7f0331e36700 1 -- 172.21.15.97:0/1528375499 wait complete. 2023-11-24T06:11:28.161 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ok 2023-11-24T06:11:28.161 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) tests the scenario where 'fs subvolume snapshot info' command 2023-11-24T06:11:28.162 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:11:28.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:28.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.279+0000 7f785c5a6700 1 Processor -- start 2023-11-24T06:11:28.282 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.279+0000 7f785c5a6700 1 -- start start 2023-11-24T06:11:28.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.280+0000 7f785c5a6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f785406c710 0x7f7854165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.280+0000 7f785c5a6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78540af950 con 0x7f78541446a0 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.280+0000 7f785c5a6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78540afc30 con 0x7f78540adf00 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.280+0000 7f785c5a6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7854144a80 con 0x7f785406c710 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f785a342700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f78541446a0 0x7f78540b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42274/0 (socket says 172.21.15.97:42274) 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f785a342700 1 -- 172.21.15.97:0/4230318974 learned_addr learned my addr 172.21.15.97:0/4230318974 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f785ab43700 1 --2- 172.21.15.97:0/4230318974 >> v2:172.21.15.183:3300/0 conn(0x7f785406c710 0x7f7854165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256949240 0 0) 0x7f78540af950 con 0x7f78541446a0 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78400035b0 con 0x7f78541446a0 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 193357517 0 0) 0x7f78400035b0 con 0x7f78541446a0 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 >> v2:172.21.15.183:3300/0 conn(0x7f785406c710 msgr2=0x7f7854165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 --2- 172.21.15.97:0/4230318974 >> v2:172.21.15.183:3300/0 conn(0x7f785406c710 0x7f7854165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 >> v1:172.21.15.183:6789/0 conn(0x7f78540adf00 legacy=0x7f785406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78540aff10 con 0x7f78541446a0 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f7844001670 con 0x7f78541446a0 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 >> v1:172.21.15.97:6789/0 conn(0x7f78541446a0 legacy=0x7f78540b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7840003a00 0x7f7840003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78400048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7840005170 0x7f784000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7854144a80 con 0x7f7840004400 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784000c0b0 con 0x7f7840005170 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784000c270 con 0x7f7840003a00 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f7859b41700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78400048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f785a342700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7840003a00 0x7f7840003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.281+0000 7f785ab43700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7840005170 0x7f784000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859b41700 1 -- 172.21.15.97:0/4230318974 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7840003a00 msgr2=0x7f7840003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859b41700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7840003a00 0x7f7840003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859b41700 1 -- 172.21.15.97:0/4230318974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7840005170 msgr2=0x7f784000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859b41700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7840005170 0x7f784000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859b41700 1 -- 172.21.15.97:0/4230318974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400035b0 con 0x7f7840004400 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f785ab43700 1 -- 172.21.15.97:0/4230318974 reap_dead start 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859b41700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78400048d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f785406f1d0 tx=0x7f7840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34adc131a416e624 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850001480 con 0x7f7840004400 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78500052d0 con 0x7f7840004400 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f7859340700 1 -- 172.21.15.97:0/4230318974 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850003680 con 0x7f7840004400 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f785c5a6700 1 -- 172.21.15.97:0/4230318974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 msgr2=0x7f78400048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f785c5a6700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78400048d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f785406f1d0 tx=0x7f7840005880 comp rx=0 tx=0).stop 2023-11-24T06:11:28.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f785c5a6700 1 -- 172.21.15.97:0/4230318974 shutdown_connections 2023-11-24T06:11:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f785c5a6700 1 --2- 172.21.15.97:0/4230318974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78400048d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f785c5a6700 1 -- 172.21.15.97:0/4230318974 >> 172.21.15.97:0/4230318974 conn(0x7f7854071b90 msgr2=0x7f7854071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.282+0000 7f785c5a6700 1 -- 172.21.15.97:0/4230318974 shutdown_connections 2023-11-24T06:11:28.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.283+0000 7f785c5a6700 1 -- 172.21.15.97:0/4230318974 wait complete. 2023-11-24T06:11:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785c5a6700 1 Processor -- start 2023-11-24T06:11:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785c5a6700 1 -- start start 2023-11-24T06:11:28.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785c5a6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78541f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785c5a6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78540d0190 0x7f785415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785c5a6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785409e1d0 0x7f7854073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785c5a6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785415d060 con 0x7f7840004400 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785c5a6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785415c390 con 0x7f785409e1d0 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785c5a6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78540d0920 con 0x7f78540d0190 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785a342700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78541f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785a342700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78541f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49834/0 (socket says 172.21.15.97:49834) 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.284+0000 7f785a342700 1 -- 172.21.15.97:0/2678657802 learned_addr learned my addr 172.21.15.97:0/2678657802 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f785a342700 1 -- 172.21.15.97:0/2678657802 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78540d0190 msgr2=0x7f785415be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f785a342700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78540d0190 0x7f785415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f785a342700 1 -- 172.21.15.97:0/2678657802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785409e1d0 msgr2=0x7f7854073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.292 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f785a342700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785409e1d0 0x7f7854073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f785a342700 1 -- 172.21.15.97:0/2678657802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785415c6c0 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f785a342700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78541f1c60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7844007290 tx=0x7f78440075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f784affd700 1 -- 172.21.15.97:0/2678657802 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844007780 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f784affd700 1 -- 172.21.15.97:0/2678657802 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784400f7f0 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785415c9a0 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f784affd700 1 -- 172.21.15.97:0/2678657802 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784400fba0 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.285+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78540d0610 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.286+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78540753f0 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.286+0000 7f784affd700 1 -- 172.21.15.97:0/2678657802 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f784400fd00 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.287+0000 7f784affd700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f783806b2c0 0x7f783806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.287+0000 7f784affd700 1 -- 172.21.15.97:0/2678657802 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f7844089810 con 0x7f7840004400 2023-11-24T06:11:28.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.287+0000 7f7859b41700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f783806b2c0 0x7f783806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.287+0000 7f7859b41700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f783806b2c0 0x7f783806d780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f785406f1d0 tx=0x7f785000f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.291+0000 7f784affd700 1 -- 172.21.15.97:0/2678657802 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78440546d0 con 0x7f7840004400 2023-11-24T06:11:28.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.481+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7854067f60 con 0x7f7840004400 2023-11-24T06:11:28.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.482+0000 7f784affd700 1 -- 172.21.15.97:0/2678657802 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+13041 (secure 0 0 0) 0x7f7844057cf0 con 0x7f7840004400 2023-11-24T06:11:28.485 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:28.485 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":64,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:27.138745+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6836/3662495999":"2023-11-25T06:11:24.128422+0000","172.21.15.97:6834/3662495999":"2023-11-25T06:11:24.128422+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f783806b2c0 msgr2=0x7f783806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f783806b2c0 0x7f783806d780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f785406f1d0 tx=0x7f785000f000 comp rx=0 tx=0).stop 2023-11-24T06:11:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 msgr2=0x7f78541f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78541f1c60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7844007290 tx=0x7f78440075a0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 shutdown_connections 2023-11-24T06:11:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f785409e1d0 0x7f7854073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7840004400 0x7f78541f1c60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f783806b2c0 0x7f783806d780 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 --2- 172.21.15.97:0/2678657802 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78540d0190 0x7f785415be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 >> 172.21.15.97:0/2678657802 conn(0x7f7854071b90 msgr2=0x7f78541422a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 shutdown_connections 2023-11-24T06:11:28.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.484+0000 7f785c5a6700 1 -- 172.21.15.97:0/2678657802 wait complete. 2023-11-24T06:11:28.499 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:28.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:28.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.619+0000 7f0800e3f700 1 Processor -- start 2023-11-24T06:11:28.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.619+0000 7f0800e3f700 1 -- start start 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f0800e3f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07fc143b80 0x7f07fc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f0800e3f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 0x7f07fc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f0800e3f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07fc0b46d0 con 0x7f07fc06c540 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f0800e3f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07fc06c9f0 con 0x7f07fc143b80 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f0800e3f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc15faf0 con 0x7f07fc15f4a0 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fa59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07fc143b80 0x7f07fc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fa59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07fc143b80 0x7f07fc06c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49842/0 (socket says 172.21.15.97:49842) 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07f9d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f07fc06c540 0x7f07fc15ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42296/0 (socket says 172.21.15.97:42296) 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fa59c700 1 -- 172.21.15.97:0/3367266098 learned_addr learned my addr 172.21.15.97:0/3367266098 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:28.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fad9d700 1 --2- 172.21.15.97:0/3367266098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 0x7f07fc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fa59c700 1 -- 172.21.15.97:0/3367266098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 msgr2=0x7f07fc06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fa59c700 1 --2- 172.21.15.97:0/3367266098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 0x7f07fc06ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fa59c700 1 -- 172.21.15.97:0/3367266098 >> v1:172.21.15.97:6789/0 conn(0x7f07fc06c540 legacy=0x7f07fc15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07f959a700 1 -- 172.21.15.97:0/3367266098 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321872554 0 0) 0x7f07fc0b46d0 con 0x7f07fc06c540 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fa59c700 1 -- 172.21.15.97:0/3367266098 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc0b3de0 con 0x7f07fc143b80 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07fa59c700 1 --2- 172.21.15.97:0/3367266098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc143b80 0x7f07fc06c000 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009920 tx=0x7f07f0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522269cdda185a65 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07f959a700 1 -- 172.21.15.97:0/3367266098 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f000ba20 con 0x7f07fc143b80 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07f959a700 1 -- 172.21.15.97:0/3367266098 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f000bb80 con 0x7f07fc143b80 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.621+0000 7f07f959a700 1 -- 172.21.15.97:0/3367266098 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0009e70 con 0x7f07fc143b80 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.622+0000 7f0800e3f700 1 -- 172.21.15.97:0/3367266098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc143b80 msgr2=0x7f07fc06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.622+0000 7f0800e3f700 1 --2- 172.21.15.97:0/3367266098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc143b80 0x7f07fc06c000 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009920 tx=0x7f07f0009c30 comp rx=0 tx=0).stop 2023-11-24T06:11:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.622+0000 7f0800e3f700 1 -- 172.21.15.97:0/3367266098 shutdown_connections 2023-11-24T06:11:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.622+0000 7f0800e3f700 1 --2- 172.21.15.97:0/3367266098 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 0x7f07fc06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.622+0000 7f0800e3f700 1 --2- 172.21.15.97:0/3367266098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc143b80 0x7f07fc06c000 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.622+0000 7f0800e3f700 1 -- 172.21.15.97:0/3367266098 >> 172.21.15.97:0/3367266098 conn(0x7f07fc071ac0 msgr2=0x7f07fc071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.622+0000 7f0800e3f700 1 -- 172.21.15.97:0/3367266098 shutdown_connections 2023-11-24T06:11:28.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.622+0000 7f0800e3f700 1 -- 172.21.15.97:0/3367266098 wait complete. 2023-11-24T06:11:28.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.623+0000 7f0800e3f700 1 Processor -- start 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.623+0000 7f0800e3f700 1 -- start start 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f0800e3f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc06c540 0x7f07fc152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f0800e3f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc143b80 0x7f07fc155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f0800e3f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 0x7f07fc073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f0800e3f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc153440 con 0x7f07fc06c540 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f0800e3f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc155560 con 0x7f07fc143b80 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f0800e3f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc1527c0 con 0x7f07fc15f4a0 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fa59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc06c540 0x7f07fc152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fa59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc06c540 0x7f07fc152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49848/0 (socket says 172.21.15.97:49848) 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fa59c700 1 -- 172.21.15.97:0/1254224488 learned_addr learned my addr 172.21.15.97:0/1254224488 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:28.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07f9d9b700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc143b80 0x7f07fc155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fad9d700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 0x7f07fc073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fa59c700 1 -- 172.21.15.97:0/1254224488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 msgr2=0x7f07fc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fa59c700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 0x7f07fc073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fa59c700 1 -- 172.21.15.97:0/1254224488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc143b80 msgr2=0x7f07fc155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fa59c700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc143b80 0x7f07fc155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.624+0000 7f07fa59c700 1 -- 172.21.15.97:0/1254224488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009550 con 0x7f07fc06c540 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.625+0000 7f07fa59c700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc06c540 0x7f07fc152150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009130 tx=0x7f07f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.625+0000 7f07eaffd700 1 -- 172.21.15.97:0/1254224488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0004480 con 0x7f07fc06c540 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.625+0000 7f07eaffd700 1 -- 172.21.15.97:0/1254224488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f001f070 con 0x7f07fc06c540 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.625+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc152aa0 con 0x7f07fc06c540 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.625+0000 7f07eaffd700 1 -- 172.21.15.97:0/1254224488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f00038c0 con 0x7f07fc06c540 2023-11-24T06:11:28.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.625+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc153060 con 0x7f07fc06c540 2023-11-24T06:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.626+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc1536b0 con 0x7f07fc06c540 2023-11-24T06:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.626+0000 7f07eaffd700 1 -- 172.21.15.97:0/1254224488 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f07f0021030 con 0x7f07fc06c540 2023-11-24T06:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.626+0000 7f07eaffd700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06b2c0 0x7f07dc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.627+0000 7f07eaffd700 1 -- 172.21.15.97:0/1254224488 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f07f008ad70 con 0x7f07fc06c540 2023-11-24T06:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.627+0000 7f07f9d9b700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06b2c0 0x7f07dc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.627+0000 7f07f9d9b700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06b2c0 0x7f07dc06d780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0b2890 tx=0x7f07ec007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.631+0000 7f07eaffd700 1 -- 172.21.15.97:0/1254224488 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07f0055cc0 con 0x7f07fc06c540 2023-11-24T06:11:28.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.821+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07fc068480 con 0x7f07fc06c540 2023-11-24T06:11:28.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.822+0000 7f07eaffd700 1 -- 172.21.15.97:0/1254224488 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07f0015020 con 0x7f07fc06c540 2023-11-24T06:11:28.825 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06b2c0 msgr2=0x7f07dc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06b2c0 0x7f07dc06d780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0b2890 tx=0x7f07ec007c90 comp rx=0 tx=0).stop 2023-11-24T06:11:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc06c540 msgr2=0x7f07fc152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc06c540 0x7f07fc152150 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009130 tx=0x7f07f0004750 comp rx=0 tx=0).stop 2023-11-24T06:11:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 shutdown_connections 2023-11-24T06:11:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07dc06b2c0 0x7f07dc06d780 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07fc15f4a0 0x7f07fc073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07fc06c540 0x7f07fc152150 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 --2- 172.21.15.97:0/1254224488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07fc143b80 0x7f07fc155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 >> 172.21.15.97:0/1254224488 conn(0x7f07fc071ac0 msgr2=0x7f07fc1595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 shutdown_connections 2023-11-24T06:11:28.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.824+0000 7f0800e3f700 1 -- 172.21.15.97:0/1254224488 wait complete. 2023-11-24T06:11:28.839 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T06:11:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.957+0000 7f6735f2d700 1 Processor -- start 2023-11-24T06:11:28.959 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.957+0000 7f6735f2d700 1 -- start start 2023-11-24T06:11:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.957+0000 7f6735f2d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f673015bdd0 0x7f67301604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.957+0000 7f6735f2d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67300d1c70 0x7f67300d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.957+0000 7f6735f2d700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67300b22d0 con 0x7f67300b3370 2023-11-24T06:11:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.957+0000 7f6735f2d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67300b2f50 con 0x7f67300d1c70 2023-11-24T06:11:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.957+0000 7f6735f2d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67300d2640 con 0x7f673015bdd0 2023-11-24T06:11:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67300d1c70 0x7f67300d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672effd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f67300b3370 0x7f67300b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42310/0 (socket says 172.21.15.97:42310) 2023-11-24T06:11:28.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672effd700 1 -- 172.21.15.97:0/3106344955 learned_addr learned my addr 172.21.15.97:0/3106344955 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672ffff700 1 -- 172.21.15.97:0/3106344955 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f673015bdd0 msgr2=0x7f67301604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672f7fe700 1 --2- 172.21.15.97:0/3106344955 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f673015bdd0 0x7f67301604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672ffff700 1 --2- 172.21.15.97:0/3106344955 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f673015bdd0 0x7f67301604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672e7fc700 1 -- 172.21.15.97:0/3106344955 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240664698 0 0) 0x7f67300b22d0 con 0x7f67300b3370 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672ffff700 1 -- 172.21.15.97:0/3106344955 >> v1:172.21.15.97:6789/0 conn(0x7f67300b3370 legacy=0x7f67300b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672ffff700 1 -- 172.21.15.97:0/3106344955 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67300b25b0 con 0x7f67300d1c70 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672ffff700 1 --2- 172.21.15.97:0/3106344955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300d1c70 0x7f67300d2100 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6728009d90 tx=0x7f672800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349046a205da0a0b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672e7fc700 1 -- 172.21.15.97:0/3106344955 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800e070 con 0x7f67300d1c70 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672e7fc700 1 -- 172.21.15.97:0/3106344955 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672800bbd0 con 0x7f67300d1c70 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.958+0000 7f672e7fc700 1 -- 172.21.15.97:0/3106344955 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728012430 con 0x7f67300d1c70 2023-11-24T06:11:28.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.959+0000 7f6735f2d700 1 -- 172.21.15.97:0/3106344955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300d1c70 msgr2=0x7f67300d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.959+0000 7f6735f2d700 1 --2- 172.21.15.97:0/3106344955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300d1c70 0x7f67300d2100 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6728009d90 tx=0x7f672800b330 comp rx=0 tx=0).stop 2023-11-24T06:11:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.959+0000 7f6735f2d700 1 -- 172.21.15.97:0/3106344955 shutdown_connections 2023-11-24T06:11:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.959+0000 7f6735f2d700 1 --2- 172.21.15.97:0/3106344955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300d1c70 0x7f67300d2100 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.959+0000 7f6735f2d700 1 --2- 172.21.15.97:0/3106344955 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f673015bdd0 0x7f67301604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.959+0000 7f6735f2d700 1 -- 172.21.15.97:0/3106344955 >> 172.21.15.97:0/3106344955 conn(0x7f67300731a0 msgr2=0x7f673009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.959+0000 7f6735f2d700 1 -- 172.21.15.97:0/3106344955 shutdown_connections 2023-11-24T06:11:28.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.959+0000 7f6735f2d700 1 -- 172.21.15.97:0/3106344955 wait complete. 2023-11-24T06:11:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.960+0000 7f6735f2d700 1 Processor -- start 2023-11-24T06:11:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.960+0000 7f6735f2d700 1 -- start start 2023-11-24T06:11:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f6735f2d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300b3370 0x7f67300cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f6735f2d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f67300d1c70 0x7f67300b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f6735f2d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f673015bdd0 0x7f673006f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f6735f2d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67300b5d30 con 0x7f67300b3370 2023-11-24T06:11:28.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f6735f2d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67300b4e10 con 0x7f673015bdd0 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f6735f2d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67300b50b0 con 0x7f67300d1c70 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300b3370 0x7f67300cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300b3370 0x7f67300cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49862/0 (socket says 172.21.15.97:49862) 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 -- 172.21.15.97:0/33581863 learned_addr learned my addr 172.21.15.97:0/33581863 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672effd700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f67300d1c70 0x7f67300b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 -- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f67300d1c70 msgr2=0x7f67300b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f67300d1c70 0x7f67300b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 -- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f673015bdd0 msgr2=0x7f673006f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672ffff700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f673015bdd0 0x7f673006f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f673015bdd0 0x7f673006f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 -- 172.21.15.97:0/33581863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6728009a70 con 0x7f67300b3370 2023-11-24T06:11:28.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672ffff700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f673015bdd0 0x7f673006f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.961+0000 7f672f7fe700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300b3370 0x7f67300cb5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f672000c7f0 tx=0x7f672000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.962+0000 7f6717fff700 1 -- 172.21.15.97:0/33581863 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720011800 con 0x7f67300b3370 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.962+0000 7f6717fff700 1 -- 172.21.15.97:0/33581863 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6720011960 con 0x7f67300b3370 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.962+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67300b5390 con 0x7f67300b3370 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.962+0000 7f6717fff700 1 -- 172.21.15.97:0/33581863 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720011cf0 con 0x7f67300b3370 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.962+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67300b5950 con 0x7f67300b3370 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.963+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6730067f60 con 0x7f67300b3370 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.963+0000 7f6717fff700 1 -- 172.21.15.97:0/33581863 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6720018360 con 0x7f67300b3370 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.963+0000 7f6717fff700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f671c06b2c0 0x7f671c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.963+0000 7f672effd700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f671c06b2c0 0x7f671c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.964+0000 7f6717fff700 1 -- 172.21.15.97:0/33581863 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6720089ae0 con 0x7f67300b3370 2023-11-24T06:11:28.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.964+0000 7f672effd700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f671c06b2c0 0x7f671c06d780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f673006fd30 tx=0x7f6724009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:28.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:28.968+0000 7f6717fff700 1 -- 172.21.15.97:0/33581863 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6720054780 con 0x7f67300b3370 2023-11-24T06:11:29.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:29.238+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f67300cc180 con 0x7f67300b3370 2023-11-24T06:11:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.150+0000 7f6717fff700 1 -- 172.21.15.97:0/33581863 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v167) v1 ==== 169+0+0 (secure 0 0 0) 0x7f672001f0f0 con 0x7f67300b3370 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f671c06b2c0 msgr2=0x7f671c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f671c06b2c0 0x7f671c06d780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f673006fd30 tx=0x7f6724009040 comp rx=0 tx=0).stop 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300b3370 msgr2=0x7f67300cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300b3370 0x7f67300cb5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f672000c7f0 tx=0x7f672000cbb0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 shutdown_connections 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f673015bdd0 0x7f673006f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300b3370 0x7f67300cb5d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f671c06b2c0 0x7f671c06d780 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 --2- 172.21.15.97:0/33581863 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f67300d1c70 0x7f67300b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 >> 172.21.15.97:0/33581863 conn(0x7f67300731a0 msgr2=0x7f67300b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 shutdown_connections 2023-11-24T06:11:30.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.153+0000 7f6735f2d700 1 -- 172.21.15.97:0/33581863 wait complete. 2023-11-24T06:11:30.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:11:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.297+0000 7f95338e6700 1 Processor -- start 2023-11-24T06:11:30.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.297+0000 7f95338e6700 1 -- start start 2023-11-24T06:11:30.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f95338e6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f952c0b4310 0x7f952c06bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f95338e6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f952c0ce430 0x7f952c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f95338e6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c0cedc0 con 0x7f952c15e080 2023-11-24T06:11:30.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f95338e6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c0d1200 con 0x7f952c0ce430 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f95338e6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c15e460 con 0x7f952c0b4310 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f9531e83700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f952c0ce430 0x7f952c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f9531e83700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f952c0ce430 0x7f952c0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49874/0 (socket says 172.21.15.97:49874) 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f9531e83700 1 -- 172.21.15.97:0/1354752215 learned_addr learned my addr 172.21.15.97:0/1354752215 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f9530e81700 1 --2- 172.21.15.97:0/1354752215 >> v2:172.21.15.183:3300/0 conn(0x7f952c0b4310 0x7f952c06bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f9531e83700 1 -- 172.21.15.97:0/1354752215 >> v2:172.21.15.183:3300/0 conn(0x7f952c0b4310 msgr2=0x7f952c06bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f9531e83700 1 --2- 172.21.15.97:0/1354752215 >> v2:172.21.15.183:3300/0 conn(0x7f952c0b4310 0x7f952c06bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.298+0000 7f9531e83700 1 -- 172.21.15.97:0/1354752215 >> v1:172.21.15.183:6789/0 conn(0x7f952c15e080 legacy=0x7f952c0d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f9531e83700 1 -- 172.21.15.97:0/1354752215 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c06ba50 con 0x7f952c0ce430 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f9531e83700 1 --2- 172.21.15.97:0/1354752215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c0ce880 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f952c072420 tx=0x7f952400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a3c97f517993c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f9523fff700 1 -- 172.21.15.97:0/1354752215 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952400e070 con 0x7f952c0ce430 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f9523fff700 1 -- 172.21.15.97:0/1354752215 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952400b910 con 0x7f952c0ce430 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f9523fff700 1 -- 172.21.15.97:0/1354752215 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952400bcc0 con 0x7f952c0ce430 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f95338e6700 1 -- 172.21.15.97:0/1354752215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 msgr2=0x7f952c0ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f95338e6700 1 --2- 172.21.15.97:0/1354752215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c0ce880 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f952c072420 tx=0x7f952400b330 comp rx=0 tx=0).stop 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f95338e6700 1 -- 172.21.15.97:0/1354752215 shutdown_connections 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f95338e6700 1 --2- 172.21.15.97:0/1354752215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c0ce880 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f95338e6700 1 --2- 172.21.15.97:0/1354752215 >> v2:172.21.15.183:3300/0 conn(0x7f952c0b4310 0x7f952c06bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f95338e6700 1 -- 172.21.15.97:0/1354752215 >> 172.21.15.97:0/1354752215 conn(0x7f952c0b3780 msgr2=0x7f952c0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f95338e6700 1 -- 172.21.15.97:0/1354752215 shutdown_connections 2023-11-24T06:11:30.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.299+0000 7f95338e6700 1 -- 172.21.15.97:0/1354752215 wait complete. 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f95338e6700 1 Processor -- start 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f95338e6700 1 -- start start 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f95338e6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f952c0b4310 0x7f952c157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f95338e6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f95338e6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f952c15e080 0x7f952c0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f95338e6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0c69f0 con 0x7f952c0ce430 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f95338e6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0c9690 con 0x7f952c0b4310 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f95338e6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0c9950 con 0x7f952c15e080 2023-11-24T06:11:30.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f9530e81700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f9530e81700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49878/0 (socket says 172.21.15.97:49878) 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.301+0000 7f9530e81700 1 -- 172.21.15.97:0/2906526146 learned_addr learned my addr 172.21.15.97:0/2906526146 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9531682700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f952c0b4310 0x7f952c157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9531e83700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f952c15e080 0x7f952c0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9530e81700 1 -- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f952c15e080 msgr2=0x7f952c0ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9530e81700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f952c15e080 0x7f952c0ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9530e81700 1 -- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f952c0b4310 msgr2=0x7f952c157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9530e81700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f952c0b4310 0x7f952c157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9530e81700 1 -- 172.21.15.97:0/2906526146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524009b30 con 0x7f952c0ce430 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9531682700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f952c0b4310 0x7f952c157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:30.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9530e81700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c157a10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f952800e980 tx=0x7f952800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9521ffb700 1 -- 172.21.15.97:0/2906526146 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9528009710 con 0x7f952c0ce430 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9521ffb700 1 -- 172.21.15.97:0/2906526146 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952800c850 con 0x7f952c0ce430 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c0c9c30 con 0x7f952c0ce430 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f9521ffb700 1 -- 172.21.15.97:0/2906526146 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800cc00 con 0x7f952c0ce430 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.302+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c0ca1f0 con 0x7f952c0ce430 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.303+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c0ca560 con 0x7f952c0ce430 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.304+0000 7f9521ffb700 1 -- 172.21.15.97:0/2906526146 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9528013470 con 0x7f952c0ce430 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.304+0000 7f9521ffb700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f951006b2c0 0x7f951006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.304+0000 7f9521ffb700 1 -- 172.21.15.97:0/2906526146 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f9528089b30 con 0x7f952c0ce430 2023-11-24T06:11:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.304+0000 7f9531682700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f951006b2c0 0x7f951006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.305+0000 7f9531682700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f951006b2c0 0x7f951006d780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f951c000f60 tx=0x7f951c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.309+0000 7f9521ffb700 1 -- 172.21.15.97:0/2906526146 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95280549a0 con 0x7f952c0ce430 2023-11-24T06:11:30.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.502+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f952c068fb0 con 0x7f952c0ce430 2023-11-24T06:11:30.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.505+0000 7f9521ffb700 1 -- 172.21.15.97:0/2906526146 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f95280170c0 con 0x7f952c0ce430 2023-11-24T06:11:30.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f951006b2c0 msgr2=0x7f951006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f951006b2c0 0x7f951006d780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f951c000f60 tx=0x7f951c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 msgr2=0x7f952c157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c157a10 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f952800e980 tx=0x7f952800ed40 comp rx=0 tx=0).stop 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 shutdown_connections 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f951006b2c0 0x7f951006d780 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f952c15e080 0x7f952c0ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f952c0b4310 0x7f952c157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 --2- 172.21.15.97:0/2906526146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f952c0ce430 0x7f952c157a10 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 >> 172.21.15.97:0/2906526146 conn(0x7f952c0b3780 msgr2=0x7f952c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 shutdown_connections 2023-11-24T06:11:30.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.507+0000 7f95338e6700 1 -- 172.21.15.97:0/2906526146 wait complete. 2023-11-24T06:11:30.522 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:11:30.522 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:11:30.567 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:11:30.609 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:11:30.609 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:30.610 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:30.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:11:30.627 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:30.627 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:30.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:11:30.669 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:30.669 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:30.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:11:30.687 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:30.687 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:30.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:11:30.728 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:11:30.728 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.846+0000 7f78bdd36700 1 Processor -- start 2023-11-24T06:11:30.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.846+0000 7f78bdd36700 1 -- start start 2023-11-24T06:11:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78bdd36700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f78b815bdd0 0x7f78b81604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78bdd36700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f78b80b3370 0x7f78b80b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78bdd36700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80d1c70 0x7f78b80d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78bdd36700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b80b2f50 con 0x7f78b80b3370 2023-11-24T06:11:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78bdd36700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b80d2660 con 0x7f78b815bdd0 2023-11-24T06:11:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78bdd36700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80b2030 con 0x7f78b80d1c70 2023-11-24T06:11:30.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f78b80b3370 0x7f78b80b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f78b80b3370 0x7f78b80b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49886/0 (socket says 172.21.15.97:49886) 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 -- 172.21.15.97:0/2415529558 learned_addr learned my addr 172.21.15.97:0/2415529558 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b7fff700 1 --2- 172.21.15.97:0/2415529558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80d1c70 0x7f78b80d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b77fe700 1 --2- 172.21.15.97:0/2415529558 >> v2:172.21.15.183:3300/0 conn(0x7f78b815bdd0 0x7f78b81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 -- 172.21.15.97:0/2415529558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80d1c70 msgr2=0x7f78b80d2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 --2- 172.21.15.97:0/2415529558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80d1c70 0x7f78b80d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 -- 172.21.15.97:0/2415529558 >> v2:172.21.15.183:3300/0 conn(0x7f78b815bdd0 msgr2=0x7f78b81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 --2- 172.21.15.97:0/2415529558 >> v2:172.21.15.183:3300/0 conn(0x7f78b815bdd0 0x7f78b81604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 -- 172.21.15.97:0/2415529558 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b80b22d0 con 0x7f78b80b3370 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b6ffd700 1 --2- 172.21.15.97:0/2415529558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80b3370 0x7f78b80b1af0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f78b806ec10 tx=0x7f78a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2b621e63c0a6df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.847+0000 7f78b67fc700 1 -- 172.21.15.97:0/2415529558 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a800b870 con 0x7f78b80b3370 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78b67fc700 1 -- 172.21.15.97:0/2415529558 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a800b9d0 con 0x7f78b80b3370 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78b67fc700 1 -- 172.21.15.97:0/2415529558 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a800bd60 con 0x7f78b80b3370 2023-11-24T06:11:30.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 -- 172.21.15.97:0/2415529558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80b3370 msgr2=0x7f78b80b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 --2- 172.21.15.97:0/2415529558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80b3370 0x7f78b80b1af0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f78b806ec10 tx=0x7f78a8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 -- 172.21.15.97:0/2415529558 shutdown_connections 2023-11-24T06:11:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 --2- 172.21.15.97:0/2415529558 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80d1c70 0x7f78b80d2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 --2- 172.21.15.97:0/2415529558 >> v2:172.21.15.183:3300/0 conn(0x7f78b815bdd0 0x7f78b81604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 --2- 172.21.15.97:0/2415529558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80b3370 0x7f78b80b1af0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 -- 172.21.15.97:0/2415529558 >> 172.21.15.97:0/2415529558 conn(0x7f78b80731a0 msgr2=0x7f78b809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 -- 172.21.15.97:0/2415529558 shutdown_connections 2023-11-24T06:11:30.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.848+0000 7f78bdd36700 1 -- 172.21.15.97:0/2415529558 wait complete. 2023-11-24T06:11:30.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.849+0000 7f78bdd36700 1 Processor -- start 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.849+0000 7f78bdd36700 1 -- start start 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78bdd36700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80b3370 0x7f78b8157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78bdd36700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80d1c70 0x7f78b815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78bdd36700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b815bdd0 0x7f78b815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78bdd36700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80c7ef0 con 0x7f78b80d1c70 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78bdd36700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80b75b0 con 0x7f78b815bdd0 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78bdd36700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80c7270 con 0x7f78b80b3370 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78b6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80d1c70 0x7f78b815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78b6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80d1c70 0x7f78b815a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49892/0 (socket says 172.21.15.97:49892) 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78b6ffd700 1 -- 172.21.15.97:0/1358407238 learned_addr learned my addr 172.21.15.97:0/1358407238 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78b77fe700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80b3370 0x7f78b8157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78b7fff700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b815bdd0 0x7f78b815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.850+0000 7f78b6ffd700 1 -- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80b3370 msgr2=0x7f78b8157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f78b6ffd700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80b3370 0x7f78b8157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f78b6ffd700 1 -- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b815bdd0 msgr2=0x7f78b815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f78b6ffd700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b815bdd0 0x7f78b815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f78b6ffd700 1 -- 172.21.15.97:0/1358407238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a8009550 con 0x7f78b80d1c70 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f78b7fff700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b815bdd0 0x7f78b815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f78b6ffd700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80d1c70 0x7f78b815a1c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f78a8013000 tx=0x7f78a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f789ffff700 1 -- 172.21.15.97:0/1358407238 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8004310 con 0x7f78b80d1c70 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f789ffff700 1 -- 172.21.15.97:0/1358407238 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a801f070 con 0x7f78b80d1c70 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f789ffff700 1 -- 172.21.15.97:0/1358407238 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a80036a0 con 0x7f78b80d1c70 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b80c7550 con 0x7f78b80d1c70 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.851+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b80c7b10 con 0x7f78b80d1c70 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.852+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b8067f60 con 0x7f78b80d1c70 2023-11-24T06:11:30.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.852+0000 7f789ffff700 1 -- 172.21.15.97:0/1358407238 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f78a8021030 con 0x7f78b80d1c70 2023-11-24T06:11:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.852+0000 7f789ffff700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f78a406b2c0 0x7f78a406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.853+0000 7f789ffff700 1 -- 172.21.15.97:0/1358407238 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f78a808ada0 con 0x7f78b80d1c70 2023-11-24T06:11:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.853+0000 7f78b77fe700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f78a406b2c0 0x7f78a406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:30.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.853+0000 7f78b77fe700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f78a406b2c0 0x7f78a406d780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00a950 tx=0x7f78ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:30.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:30.857+0000 7f789ffff700 1 -- 172.21.15.97:0/1358407238 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78a8055cf0 con 0x7f78b80d1c70 2023-11-24T06:11:31.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.086+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78b80c8030 con 0x7f78b80d1c70 2023-11-24T06:11:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.086+0000 7f789ffff700 1 -- 172.21.15.97:0/1358407238 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f78b80c8030 con 0x7f78b80d1c70 2023-11-24T06:11:31.089 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:31.089 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":39,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5554,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/3508641674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":3508641674},{"type":"v1","addr":"172.21.15.183:6833","nonce":3508641674}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":5557,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/116508824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":116508824},{"type":"v1","addr":"172.21.15.183:6835","nonce":116508824}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":6000,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2785455707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2785455707},{"type":"v1","addr":"172.21.15.97:6837","nonce":2785455707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6381,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/709421404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":709421404},{"type":"v1","addr":"172.21.15.97:6836","nonce":709421404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[]} 2023-11-24T06:11:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f78a406b2c0 msgr2=0x7f78a406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f78a406b2c0 0x7f78a406d780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00a950 tx=0x7f78ac008040 comp rx=0 tx=0).stop 2023-11-24T06:11:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80d1c70 msgr2=0x7f78b815a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80d1c70 0x7f78b815a1c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f78a8013000 tx=0x7f78a8004750 comp rx=0 tx=0).stop 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 shutdown_connections 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78b815bdd0 0x7f78b815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f78a406b2c0 0x7f78a406d780 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78b80b3370 0x7f78b8157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 --2- 172.21.15.97:0/1358407238 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78b80d1c70 0x7f78b815a1c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 >> 172.21.15.97:0/1358407238 conn(0x7f78b80731a0 msgr2=0x7f78b80b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 shutdown_connections 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.088+0000 7f78bdd36700 1 -- 172.21.15.97:0/1358407238 wait complete. 2023-11-24T06:11:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 39 2023-11-24T06:11:31.103 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:11:31.103 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:11:31.104 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:11:31.104 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:31.104 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:11:31.104 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:11:31.104 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:11:31.104 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:31.104 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:11:31.105 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:11:31.105 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:11:31.105 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:31.105 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:11:31.105 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:11:31.105 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:11:31.105 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:11:31.113 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:11:31.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:11:31.114 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:11:31.114 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:11:31.118 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:11:31.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:11:31.119 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:11:31.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:11:31.156 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:11:31.156 DEBUG:teuthology.parallel:result is None 2023-11-24T06:11:31.156 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:11:31.157 DEBUG:teuthology.parallel:result is None 2023-11-24T06:11:31.161 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:11:31.161 DEBUG:teuthology.parallel:result is None 2023-11-24T06:11:31.162 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:11:31.162 DEBUG:teuthology.parallel:result is None 2023-11-24T06:11:31.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:11:31.235 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-11-24T06:11:31.235 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-11-24T06:11:31.253 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-11-24T06:11:31.253 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-11-24T06:11:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.288+0000 7f7123c8a700 1 Processor -- start 2023-11-24T06:11:31.291 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.288+0000 7f7123c8a700 1 -- start start 2023-11-24T06:11:31.293 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.289+0000 7f7123c8a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c0b50c0 0x7f711c0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.289+0000 7f7123c8a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c06bf50 con 0x7f711c15ac50 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.289+0000 7f7123c8a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c15b030 con 0x7f711c06f460 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.289+0000 7f7123c8a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c06e810 con 0x7f711c0b50c0 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.289+0000 7f7121a26700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f711c15ac50 0x7f711c06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42376/0 (socket says 172.21.15.97:42376) 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.289+0000 7f7121a26700 1 -- 172.21.15.97:0/3417855690 learned_addr learned my addr 172.21.15.97:0/3417855690 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.289+0000 7f7122227700 1 --2- 172.21.15.97:0/3417855690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c0b50c0 0x7f711c0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.289+0000 7f7120a24700 1 -- 172.21.15.97:0/3417855690 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390587933 0 0) 0x7f711c06bf50 con 0x7f711c15ac50 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7120a24700 1 -- 172.21.15.97:0/3417855690 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71080035b0 con 0x7f711c15ac50 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7120a24700 1 -- 172.21.15.97:0/3417855690 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730694104 0 0) 0x7f711c15b030 con 0x7f711c06f460 2023-11-24T06:11:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7120a24700 1 -- 172.21.15.97:0/3417855690 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711c06bf50 con 0x7f711c06f460 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7122227700 1 -- 172.21.15.97:0/3417855690 >> v1:172.21.15.183:6789/0 conn(0x7f711c06f460 legacy=0x7f711c0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7122227700 1 -- 172.21.15.97:0/3417855690 >> v1:172.21.15.97:6789/0 conn(0x7f711c15ac50 legacy=0x7f711c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7122227700 1 -- 172.21.15.97:0/3417855690 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c06b340 con 0x7f711c0b50c0 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7122227700 1 --2- 172.21.15.97:0/3417855690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c0b50c0 0x7f711c0cdf30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7118009870 tx=0x7f7118009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd50e87b2094db5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7120a24700 1 -- 172.21.15.97:0/3417855690 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118011830 con 0x7f711c0b50c0 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7120a24700 1 -- 172.21.15.97:0/3417855690 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7118011990 con 0x7f711c0b50c0 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.290+0000 7f7120a24700 1 -- 172.21.15.97:0/3417855690 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118011d20 con 0x7f711c0b50c0 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.291+0000 7f7123c8a700 1 -- 172.21.15.97:0/3417855690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c0b50c0 msgr2=0x7f711c0cdf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.291+0000 7f7123c8a700 1 --2- 172.21.15.97:0/3417855690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c0b50c0 0x7f711c0cdf30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7118009870 tx=0x7f7118009b80 comp rx=0 tx=0).stop 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.291+0000 7f7123c8a700 1 -- 172.21.15.97:0/3417855690 shutdown_connections 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.291+0000 7f7123c8a700 1 --2- 172.21.15.97:0/3417855690 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c0b50c0 0x7f711c0cdf30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.291+0000 7f7123c8a700 1 -- 172.21.15.97:0/3417855690 >> 172.21.15.97:0/3417855690 conn(0x7f711c0b3130 msgr2=0x7f711c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.291+0000 7f7123c8a700 1 -- 172.21.15.97:0/3417855690 shutdown_connections 2023-11-24T06:11:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.291+0000 7f7123c8a700 1 -- 172.21.15.97:0/3417855690 wait complete. 2023-11-24T06:11:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.292+0000 7f7123c8a700 1 Processor -- start 2023-11-24T06:11:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.292+0000 7f7123c8a700 1 -- start start 2023-11-24T06:11:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7123c8a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f711c06f460 0x7f711c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.298 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7123c8a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c0b50c0 0x7f711c14f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7123c8a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c15ac50 0x7f711c141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7123c8a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c151b80 con 0x7f711c06f460 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7123c8a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c150010 con 0x7f711c0b50c0 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7123c8a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c150f00 con 0x7f711c15ac50 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f711c06f460 0x7f711c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f711c06f460 0x7f711c151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49930/0 (socket says 172.21.15.97:49930) 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 -- 172.21.15.97:0/3370132753 learned_addr learned my addr 172.21.15.97:0/3370132753 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7122227700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c15ac50 0x7f711c141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121225700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c0b50c0 0x7f711c14f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 -- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c15ac50 msgr2=0x7f711c141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c15ac50 0x7f711c141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 -- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c0b50c0 msgr2=0x7f711c14f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c0b50c0 0x7f711c14f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 -- 172.21.15.97:0/3370132753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180094a0 con 0x7f711c06f460 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7122227700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c15ac50 0x7f711c141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121225700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c0b50c0 0x7f711c14f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7121a26700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f711c06f460 0x7f711c151fd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f71140021e0 tx=0x7f711400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f71127fc700 1 -- 172.21.15.97:0/3370132753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114011670 con 0x7f711c06f460 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f71127fc700 1 -- 172.21.15.97:0/3370132753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711400ce90 con 0x7f711c06f460 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c1511e0 con 0x7f711c06f460 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f71127fc700 1 -- 172.21.15.97:0/3370132753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71140119f0 con 0x7f711c06f460 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.293+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c1517a0 con 0x7f711c06f460 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.294+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c067f60 con 0x7f711c06f460 2023-11-24T06:11:31.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.295+0000 7f71127fc700 1 -- 172.21.15.97:0/3370132753 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f7114011b50 con 0x7f711c06f460 2023-11-24T06:11:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.295+0000 7f71127fc700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f710006b2c0 0x7f710006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.295+0000 7f71127fc700 1 -- 172.21.15.97:0/3370132753 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f711408c820 con 0x7f711c06f460 2023-11-24T06:11:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.295+0000 7f7121225700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f710006b2c0 0x7f710006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:31.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.295+0000 7f7121225700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f710006b2c0 0x7f710006d780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f710c007fd0 tx=0x7f710c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:31.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.299+0000 7f71127fc700 1 -- 172.21.15.97:0/3370132753 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f711405af50 con 0x7f711c06f460 2023-11-24T06:11:31.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:31.507+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f711c151cc0 con 0x7f711c06f460 2023-11-24T06:11:32.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.164+0000 7f71127fc700 1 -- 172.21.15.97:0/3370132753 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) v1 ==== 97+0+0 (secure 0 0 0) 0x7f711c151cc0 con 0x7f711c06f460 2023-11-24T06:11:32.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f710006b2c0 msgr2=0x7f710006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f710006b2c0 0x7f710006d780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f710c007fd0 tx=0x7f710c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:32.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f711c06f460 msgr2=0x7f711c151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f711c06f460 0x7f711c151fd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f71140021e0 tx=0x7f711400c550 comp rx=0 tx=0).stop 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 shutdown_connections 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f710006b2c0 0x7f710006d780 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f711c15ac50 0x7f711c141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f711c06f460 0x7f711c151fd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 --2- 172.21.15.97:0/3370132753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f711c0b50c0 0x7f711c14f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 >> 172.21.15.97:0/3370132753 conn(0x7f711c0b3130 msgr2=0x7f711c0cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 shutdown_connections 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.167+0000 7f7123c8a700 1 -- 172.21.15.97:0/3370132753 wait complete. 2023-11-24T06:11:32.171 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-11-24T06:11:32.186 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:11:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.308+0000 7fa6da7ba700 1 Processor -- start 2023-11-24T06:11:32.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.308+0000 7fa6da7ba700 1 -- start start 2023-11-24T06:11:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6da7ba700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 0x7fa6d40b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6da7ba700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d40ccde0 con 0x7fa6d4072df0 2023-11-24T06:11:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6da7ba700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d40cd0c0 con 0x7fa6d415ddf0 2023-11-24T06:11:32.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6da7ba700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40cd3a0 con 0x7fa6d40b0230 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6d37fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa6d4072df0 0x7fa6d40cacb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42394/0 (socket says 172.21.15.97:42394) 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6d37fe700 1 -- 172.21.15.97:0/3133324409 learned_addr learned my addr 172.21.15.97:0/3133324409 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662019422 0 0) 0x7fa6d40ccde0 con 0x7fa6d4072df0 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6c00046c0 con 0x7fa6d4072df0 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.309+0000 7fa6d8d57700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 0x7fa6d40b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3604000393 0 0) 0x7fa6c00046c0 con 0x7fa6d4072df0 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 msgr2=0x7fa6d40b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 0x7fa6d40b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 >> v1:172.21.15.183:6789/0 conn(0x7fa6d415ddf0 legacy=0x7fa6d40726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40cd680 con 0x7fa6d4072df0 2023-11-24T06:11:32.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fa6c4001670 con 0x7fa6d4072df0 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 >> v1:172.21.15.97:6789/0 conn(0x7fa6d4072df0 legacy=0x7fa6d40cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 0x7fa6c0004b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c00050c0 0x7fa6c0005960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6c0005ea0 0x7fa6c0006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40cd3a0 con 0x7fa6c0005ea0 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c000cd60 con 0x7fa6c00050c0 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c000cf20 con 0x7fa6d40b0230 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d8d57700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6c0005ea0 0x7fa6c0006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d37fe700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c00050c0 0x7fa6c0005960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d3fff700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 0x7fa6c0004b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d8d57700 1 -- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 msgr2=0x7fa6c0004b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:32.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d8d57700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 0x7fa6c0004b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d8d57700 1 -- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c00050c0 msgr2=0x7fa6c0005960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d8d57700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c00050c0 0x7fa6c0005960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d8d57700 1 -- 172.21.15.97:0/3133324409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00046c0 con 0x7fa6c0005ea0 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d3fff700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d40b0230 0x7fa6c0004b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d37fe700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c00050c0 0x7fa6c0005960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d8d57700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6c0005ea0 0x7fa6c0006320 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009fc0 tx=0x7fa6c0006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac15e7acf3eae10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00ec30 con 0x7fa6c0005ea0 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc00f040 con 0x7fa6c0005ea0 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.310+0000 7fa6d2ffd700 1 -- 172.21.15.97:0/3133324409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc014480 con 0x7fa6c0005ea0 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.311+0000 7fa6da7ba700 1 -- 172.21.15.97:0/3133324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6c0005ea0 msgr2=0x7fa6c0006320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.311+0000 7fa6da7ba700 1 --2- 172.21.15.97:0/3133324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6c0005ea0 0x7fa6c0006320 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009fc0 tx=0x7fa6c0006990 comp rx=0 tx=0).stop 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.311+0000 7fa6d8d57700 1 -- 172.21.15.97:0/3133324409 reap_dead start 2023-11-24T06:11:32.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.311+0000 7fa6da7ba700 1 -- 172.21.15.97:0/3133324409 shutdown_connections 2023-11-24T06:11:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.311+0000 7fa6da7ba700 1 -- 172.21.15.97:0/3133324409 >> 172.21.15.97:0/3133324409 conn(0x7fa6d40b57b0 msgr2=0x7fa6d40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.311+0000 7fa6da7ba700 1 -- 172.21.15.97:0/3133324409 shutdown_connections 2023-11-24T06:11:32.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.311+0000 7fa6da7ba700 1 -- 172.21.15.97:0/3133324409 wait complete. 2023-11-24T06:11:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.312+0000 7fa6da7ba700 1 Processor -- start 2023-11-24T06:11:32.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.312+0000 7fa6da7ba700 1 -- start start 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6da7ba700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c0005ea0 0x7fa6d4157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6da7ba700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d415c440 0x7fa6d415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6da7ba700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d41e3f30 0x7fa6d41e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6da7ba700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d41e48f0 con 0x7fa6d41e3f30 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6da7ba700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40b3a80 con 0x7fa6c0005ea0 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6da7ba700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d415d590 con 0x7fa6d415c440 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d8d57700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d41e3f30 0x7fa6d41e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d8d57700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d41e3f30 0x7fa6d41e43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49956/0 (socket says 172.21.15.97:49956) 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d8d57700 1 -- 172.21.15.97:0/2202314406 learned_addr learned my addr 172.21.15.97:0/2202314406 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d37fe700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d415c440 0x7fa6d415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d3fff700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c0005ea0 0x7fa6d4157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d8d57700 1 -- 172.21.15.97:0/2202314406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d415c440 msgr2=0x7fa6d415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d8d57700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d415c440 0x7fa6d415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d8d57700 1 -- 172.21.15.97:0/2202314406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c0005ea0 msgr2=0x7fa6d4157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d8d57700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c0005ea0 0x7fa6d4157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.313+0000 7fa6d8d57700 1 -- 172.21.15.97:0/2202314406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d415a280 con 0x7fa6d41e3f30 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.314+0000 7fa6d8d57700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d41e3f30 0x7fa6d41e43b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc016000 tx=0x7fa6cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.314+0000 7fa6d0ff9700 1 -- 172.21.15.97:0/2202314406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc007000 con 0x7fa6d41e3f30 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.314+0000 7fa6d0ff9700 1 -- 172.21.15.97:0/2202314406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc003e50 con 0x7fa6d41e3f30 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.314+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d415cfd0 con 0x7fa6d41e3f30 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.314+0000 7fa6d0ff9700 1 -- 172.21.15.97:0/2202314406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0149f0 con 0x7fa6d41e3f30 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.314+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d40b3710 con 0x7fa6d41e3f30 2023-11-24T06:11:32.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.315+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d4067f60 con 0x7fa6d41e3f30 2023-11-24T06:11:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.315+0000 7fa6d0ff9700 1 -- 172.21.15.97:0/2202314406 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa6cc023020 con 0x7fa6d41e3f30 2023-11-24T06:11:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.316+0000 7fa6d0ff9700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa6b406b2b0 0x7fa6b406d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.316+0000 7fa6d0ff9700 1 -- 172.21.15.97:0/2202314406 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fa6cc08ade0 con 0x7fa6d41e3f30 2023-11-24T06:11:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.316+0000 7fa6d3fff700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa6b406b2b0 0x7fa6b406d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.316+0000 7fa6d3fff700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa6b406b2b0 0x7fa6b406d770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc007fd0 tx=0x7fa6bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.320+0000 7fa6d0ff9700 1 -- 172.21.15.97:0/2202314406 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6cc055c30 con 0x7fa6d41e3f30 2023-11-24T06:11:32.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.589+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa6d40b3bc0 con 0x7fa6d41e3f30 2023-11-24T06:11:32.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.589+0000 7fa6d0ff9700 1 -- 172.21.15.97:0/2202314406 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa6d40b3bc0 con 0x7fa6d41e3f30 2023-11-24T06:11:32.592 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:32.592 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlijU/IxAAp5/+zEd6ER/JFqum0ptk0Q==", 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.593 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLftpLxAAH+Ew2GQv73YtKLzZJjSBew==", 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.594 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlGtxpKhAAiQm5ore3YYETyG+sIKzEcw==", 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.595 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBl05iyNhAAHZYZ5pGghTwe55n84iXXQw==", 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDAPWBlwipJFBAAGLCD8wGcYoSy4yM4RXb0/Q==", 2023-11-24T06:11:32.596 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDDPWBlgW7fExAAQuUSGJn+ZfHSMwSi5r1pQg==", 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDGPWBl3E8uExAAJKjzbRFyYie6dmnRrP7VVw==", 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-11-24T06:11:32.598 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJPWBl5gnRExAAbKbgF6DcdTSv8M84GXyONw==", 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDPPWBlmCfaFRAAsIQWHUnlgDQcRXLndlUZKg==", 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:32.599 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSPWBlPxfqFRAARyMzgivfTfgmFxZRjp54JQ==", 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.600 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVPWBl8cjTFBAAsNfkNY4gGVTJwh+mMtADgg==", 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.601 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDYPWBljh8EFBAA4nVPoyEXigIDTUB6enXsXA==", 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA==", 2023-11-24T06:11:32.602 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlOXAsBhAABOT6CW2RVj7Iv5XMPvQTgQ==", 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:11:32.603 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlJRe4AhAAGQVWmqQr/urD7AefOfmoZQ==", 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:11:32.604 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlwX2YCRAAIM9eB39aohf1xUdEvHr/tA==", 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.605 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlVBcHBhAAbu6IOCKCLUA+5APAYwbRUw==", 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.606 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlOezbHxAAlNbpLHDzRHvk2SXFvQoTXQ==", 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBla7myGxAAHEPHxXHfBzCeqi2nLjhK6g==", 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:11:32.607 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:11:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:11:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:32.608 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-11-24T06:11:32.608 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:11:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa6b406b2b0 msgr2=0x7fa6b406d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa6b406b2b0 0x7fa6b406d770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc007fd0 tx=0x7fa6bc009040 comp rx=0 tx=0).stop 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d41e3f30 msgr2=0x7fa6d41e43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d41e3f30 0x7fa6d41e43b0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc016000 tx=0x7fa6cc009750 comp rx=0 tx=0).stop 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 shutdown_connections 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6d41e3f30 0x7fa6d41e43b0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6c0005ea0 0x7fa6d4157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa6b406b2b0 0x7fa6b406d770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 --2- 172.21.15.97:0/2202314406 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa6d415c440 0x7fa6d415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 >> 172.21.15.97:0/2202314406 conn(0x7fa6d40b57b0 msgr2=0x7fa6d4159750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 shutdown_connections 2023-11-24T06:11:32.610 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.591+0000 7fa6da7ba700 1 -- 172.21.15.97:0/2202314406 wait complete. 2023-11-24T06:11:32.611 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:32.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.728+0000 7f5903175700 1 Processor -- start 2023-11-24T06:11:32.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.728+0000 7f5903175700 1 -- start start 2023-11-24T06:11:32.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5903175700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f58fc15e080 0x7f58fc0cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5903175700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc0ce4d0 0x7f58fc0d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5903175700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58fc0d0150 con 0x7f58fc0ce910 2023-11-24T06:11:32.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5903175700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58fc15e460 con 0x7f58fc15e080 2023-11-24T06:11:32.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5903175700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc0cee50 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5900f11700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f58fc15e080 0x7f58fc0cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f58fbfff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc0ce4d0 0x7f58fc0d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5901712700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f58fc0ce910 0x7f58fc0d05e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51738/0 (socket says 172.21.15.97:51738) 2023-11-24T06:11:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5900f11700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f58fc15e080 0x7f58fc0cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:44120/0 (socket says 172.21.15.97:44120) 2023-11-24T06:11:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f58fbfff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc0ce4d0 0x7f58fc0d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:36998/0 (socket says 172.21.15.97:36998) 2023-11-24T06:11:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.729+0000 7f5901712700 1 -- 172.21.15.97:0/3099121157 learned_addr learned my addr 172.21.15.97:0/3099121157 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fbfff700 1 -- 172.21.15.97:0/3099121157 >> v2:172.21.15.183:3300/0 conn(0x7f58fc15e080 msgr2=0x7f58fc0cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:32.735 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fbfff700 1 --2- 172.21.15.97:0/3099121157 >> v2:172.21.15.183:3300/0 conn(0x7f58fc15e080 0x7f58fc0cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fbfff700 1 -- 172.21.15.97:0/3099121157 >> v1:172.21.15.183:6789/0 conn(0x7f58fc0ce910 legacy=0x7f58fc0d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fbfff700 1 -- 172.21.15.97:0/3099121157 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc0cf130 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fb7fe700 1 -- 172.21.15.97:0/3099121157 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102605924 0 0) 0x7f58fc0d0150 con 0x7f58fc0ce910 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f5900f11700 1 --2- 172.21.15.97:0/3099121157 >> v2:172.21.15.183:3300/0 conn(0x7f58fc15e080 0x7f58fc0cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fbfff700 1 --2- 172.21.15.97:0/3099121157 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc0ce4d0 0x7f58fc0d1be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009840 tx=0x7f58e8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ec8e4c1734bf68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fb7fe700 1 -- 172.21.15.97:0/3099121157 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8011830 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fb7fe700 1 -- 172.21.15.97:0/3099121157 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e8011990 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.730+0000 7f58fb7fe700 1 -- 172.21.15.97:0/3099121157 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8011d20 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.731+0000 7f5903175700 1 -- 172.21.15.97:0/3099121157 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc0ce4d0 msgr2=0x7f58fc0d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.731+0000 7f5903175700 1 --2- 172.21.15.97:0/3099121157 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc0ce4d0 0x7f58fc0d1be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009840 tx=0x7f58e8009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.731+0000 7f5903175700 1 -- 172.21.15.97:0/3099121157 shutdown_connections 2023-11-24T06:11:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.731+0000 7f5903175700 1 --2- 172.21.15.97:0/3099121157 >> v2:172.21.15.183:3300/0 conn(0x7f58fc15e080 0x7f58fc0cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.731+0000 7f5903175700 1 --2- 172.21.15.97:0/3099121157 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc0ce4d0 0x7f58fc0d1be0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.731+0000 7f5903175700 1 -- 172.21.15.97:0/3099121157 >> 172.21.15.97:0/3099121157 conn(0x7f58fc073130 msgr2=0x7f58fc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.731+0000 7f5903175700 1 -- 172.21.15.97:0/3099121157 shutdown_connections 2023-11-24T06:11:32.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.731+0000 7f5903175700 1 -- 172.21.15.97:0/3099121157 wait complete. 2023-11-24T06:11:32.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.732+0000 7f5903175700 1 Processor -- start 2023-11-24T06:11:32.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.732+0000 7f5903175700 1 -- start start 2023-11-24T06:11:32.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5903175700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58fc0ce4d0 0x7f58fc1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5903175700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58fc0ce910 0x7f58fc0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5903175700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc15e080 0x7f58fc06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5903175700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc151820 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5903175700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc0c64a0 con 0x7f58fc0ce910 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5903175700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58fc0c5890 con 0x7f58fc15e080 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58fc0ce4d0 0x7f58fc1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58fc0ce4d0 0x7f58fc1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49966/0 (socket says 172.21.15.97:49966) 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 -- 172.21.15.97:0/2339926505 learned_addr learned my addr 172.21.15.97:0/2339926505 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f58fbfff700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58fc0ce910 0x7f58fc0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5901712700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc15e080 0x7f58fc06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 -- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc15e080 msgr2=0x7f58fc06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc15e080 0x7f58fc06ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 -- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58fc0ce910 msgr2=0x7f58fc0c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58fc0ce910 0x7f58fc0c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 -- 172.21.15.97:0/2339926505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f00094a0 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5901712700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc15e080 0x7f58fc06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f5900f11700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58fc0ce4d0 0x7f58fc1512e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ebe0 tx=0x7f58f000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f58f97fa700 1 -- 172.21.15.97:0/2339926505 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000c850 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.733+0000 7f58f97fa700 1 -- 172.21.15.97:0/2339926505 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f000c9b0 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.734+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e80094a0 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.734+0000 7f58f97fa700 1 -- 172.21.15.97:0/2339926505 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000cd40 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.734+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58fc0c5e50 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.734+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58fc0c6780 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.735+0000 7f58f97fa700 1 -- 172.21.15.97:0/2339926505 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f58f0013470 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.735+0000 7f58f97fa700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58e006b2f0 0x7f58e006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.735+0000 7f58f97fa700 1 -- 172.21.15.97:0/2339926505 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f58f0089ad0 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.735+0000 7f58fbfff700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58e006b2f0 0x7f58e006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.736+0000 7f58fbfff700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58e006b2f0 0x7f58e006d7b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f58e8000c00 tx=0x7f58e8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:32.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.740+0000 7f58f97fa700 1 -- 172.21.15.97:0/2339926505 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58f0054890 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.934 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.931+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58fc0cc760 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.935 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.932+0000 7f58f97fa700 1 -- 172.21.15.97:0/2339926505 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+12912 (secure 0 0 0) 0x7f58f000cea0 con 0x7f58fc0ce4d0 2023-11-24T06:11:32.935 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:32.935 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":68,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:32.171231+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:32.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58e006b2f0 msgr2=0x7f58e006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58e006b2f0 0x7f58e006d7b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f58e8000c00 tx=0x7f58e8008030 comp rx=0 tx=0).stop 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58fc0ce4d0 msgr2=0x7f58fc1512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58fc0ce4d0 0x7f58fc1512e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ebe0 tx=0x7f58f000eef0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 shutdown_connections 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f58e006b2f0 0x7f58e006d7b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58fc15e080 0x7f58fc06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58fc0ce4d0 0x7f58fc1512e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.934+0000 7f5903175700 1 --2- 172.21.15.97:0/2339926505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58fc0ce910 0x7f58fc0c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.935+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 >> 172.21.15.97:0/2339926505 conn(0x7f58fc073130 msgr2=0x7f58fc06fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.935+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 shutdown_connections 2023-11-24T06:11:32.938 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:32.935+0000 7f5903175700 1 -- 172.21.15.97:0/2339926505 wait complete. 2023-11-24T06:11:32.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:32.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:33.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.069+0000 7f1b8c2fb700 1 Processor -- start 2023-11-24T06:11:33.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.070+0000 7f1b8c2fb700 1 -- start start 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.070+0000 7f1b8c2fb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 0x7f1b840b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.070+0000 7f1b8c2fb700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b840746a0 0x7f1b8406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.070+0000 7f1b8c2fb700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b840b3c60 con 0x7f1b840afc90 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.070+0000 7f1b8c2fb700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b840b4500 con 0x7f1b840746a0 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.070+0000 7f1b8c2fb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840b58c0 con 0x7f1b8415e160 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b8a898700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b840746a0 0x7f1b8406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b8a898700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b840746a0 0x7f1b8406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49978/0 (socket says 172.21.15.97:49978) 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b8a898700 1 -- 172.21.15.97:0/1412580515 learned_addr learned my addr 172.21.15.97:0/1412580515 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:33.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b8a898700 1 -- 172.21.15.97:0/1412580515 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 msgr2=0x7f1b840b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:33.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b8a898700 1 --2- 172.21.15.97:0/1412580515 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 0x7f1b840b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b8a898700 1 -- 172.21.15.97:0/1412580515 >> v1:172.21.15.183:6789/0 conn(0x7f1b840afc90 legacy=0x7f1b84073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:33.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b8a898700 1 -- 172.21.15.97:0/1412580515 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8406bdc0 con 0x7f1b840746a0 2023-11-24T06:11:33.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b8a898700 1 --2- 172.21.15.97:0/1412580515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840746a0 0x7f1b8406d930 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009e50 tx=0x7f1b7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f52bfe246467d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b89095700 1 -- 172.21.15.97:0/1412580515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c00e070 con 0x7f1b840746a0 2023-11-24T06:11:33.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b89095700 1 -- 172.21.15.97:0/1412580515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c003a20 con 0x7f1b840746a0 2023-11-24T06:11:33.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.071+0000 7f1b89095700 1 -- 172.21.15.97:0/1412580515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c003dd0 con 0x7f1b840746a0 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.072+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1412580515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840746a0 msgr2=0x7f1b8406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.072+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1412580515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840746a0 0x7f1b8406d930 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009e50 tx=0x7f1b7c00b330 comp rx=0 tx=0).stop 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.072+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1412580515 shutdown_connections 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.072+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1412580515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840746a0 0x7f1b8406d930 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.072+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1412580515 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 0x7f1b840b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.072+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1412580515 >> 172.21.15.97:0/1412580515 conn(0x7f1b840d0bf0 msgr2=0x7f1b840d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.072+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1412580515 shutdown_connections 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.072+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1412580515 wait complete. 2023-11-24T06:11:33.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.073+0000 7f1b8c2fb700 1 Processor -- start 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.073+0000 7f1b8c2fb700 1 -- start start 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8c2fb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b840746a0 0x7f1b84152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8c2fb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840afc90 0x7f1b84151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8c2fb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 0x7f1b840b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8c2fb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84157f40 con 0x7f1b840afc90 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8c2fb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84151f50 con 0x7f1b840746a0 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8c2fb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84157b80 con 0x7f1b8415e160 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840afc90 0x7f1b84151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840afc90 0x7f1b84151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49982/0 (socket says 172.21.15.97:49982) 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 -- 172.21.15.97:0/1444902553 learned_addr learned my addr 172.21.15.97:0/1444902553 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8a097700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b840746a0 0x7f1b84152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8a898700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 0x7f1b840b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 -- 172.21.15.97:0/1444902553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 msgr2=0x7f1b840b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 0x7f1b840b1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 -- 172.21.15.97:0/1444902553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b840746a0 msgr2=0x7f1b84152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b840746a0 0x7f1b84152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 -- 172.21.15.97:0/1444902553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c009b30 con 0x7f1b840afc90 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b89896700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840afc90 0x7f1b84151a10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c4d0 tx=0x7f1b8000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b7affd700 1 -- 172.21.15.97:0/1444902553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80011790 con 0x7f1b840afc90 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b7affd700 1 -- 172.21.15.97:0/1444902553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b800118f0 con 0x7f1b840afc90 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b841543c0 con 0x7f1b840afc90 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.074+0000 7f1b7affd700 1 -- 172.21.15.97:0/1444902553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80011ca0 con 0x7f1b840afc90 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.075+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b84154980 con 0x7f1b840afc90 2023-11-24T06:11:33.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.075+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b84158080 con 0x7f1b840afc90 2023-11-24T06:11:33.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.076+0000 7f1b7affd700 1 -- 172.21.15.97:0/1444902553 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1b8001c460 con 0x7f1b840afc90 2023-11-24T06:11:33.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.076+0000 7f1b7affd700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b6806b2c0 0x7f1b6806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.076+0000 7f1b8a097700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b6806b2c0 0x7f1b6806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.076+0000 7f1b7affd700 1 -- 172.21.15.97:0/1444902553 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f1b80089810 con 0x7f1b840afc90 2023-11-24T06:11:33.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.077+0000 7f1b8a097700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b6806b2c0 0x7f1b6806d780 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1b8406b760 tx=0x7f1b74009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.081+0000 7f1b7affd700 1 -- 172.21.15.97:0/1444902553 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b800545d0 con 0x7f1b840afc90 2023-11-24T06:11:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.274+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b84068fb0 con 0x7f1b840afc90 2023-11-24T06:11:33.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.275+0000 7f1b7affd700 1 -- 172.21.15.97:0/1444902553 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b80057bf0 con 0x7f1b840afc90 2023-11-24T06:11:33.278 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b6806b2c0 msgr2=0x7f1b6806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b6806b2c0 0x7f1b6806d780 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1b8406b760 tx=0x7f1b74009110 comp rx=0 tx=0).stop 2023-11-24T06:11:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840afc90 msgr2=0x7f1b84151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840afc90 0x7f1b84151a10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c4d0 tx=0x7f1b8000c7e0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 shutdown_connections 2023-11-24T06:11:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b6806b2c0 0x7f1b6806d780 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8415e160 0x7f1b840b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b840746a0 0x7f1b84152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 --2- 172.21.15.97:0/1444902553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b840afc90 0x7f1b84151a10 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 >> 172.21.15.97:0/1444902553 conn(0x7f1b840d0bf0 msgr2=0x7f1b84073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 shutdown_connections 2023-11-24T06:11:33.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.277+0000 7f1b8c2fb700 1 -- 172.21.15.97:0/1444902553 wait complete. 2023-11-24T06:11:33.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:11:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.410+0000 7fe2b2612700 1 Processor -- start 2023-11-24T06:11:33.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.410+0000 7fe2b2612700 1 -- start start 2023-11-24T06:11:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2b2612700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe2ac144660 0x7fe2ac0b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2b2612700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac15f520 con 0x7fe2ac15ea60 2023-11-24T06:11:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2b2612700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac15f800 con 0x7fe2ac06c710 2023-11-24T06:11:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2b2612700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ac0b4230 con 0x7fe2ac144660 2023-11-24T06:11:33.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2ab7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe2ac15ea60 0x7fe2ac06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42444/0 (socket says 172.21.15.97:42444) 2023-11-24T06:11:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2ab7fe700 1 -- 172.21.15.97:0/1417109635 learned_addr learned my addr 172.21.15.97:0/1417109635 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2abfff700 1 --2- 172.21.15.97:0/1417109635 >> v2:172.21.15.183:3300/0 conn(0x7fe2ac144660 0x7fe2ac0b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2aaffd700 1 -- 172.21.15.97:0/1417109635 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004971629 0 0) 0x7fe2ac15f520 con 0x7fe2ac15ea60 2023-11-24T06:11:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2aaffd700 1 -- 172.21.15.97:0/1417109635 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2980035b0 con 0x7fe2ac15ea60 2023-11-24T06:11:33.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2aaffd700 1 -- 172.21.15.97:0/1417109635 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3297926067 0 0) 0x7fe2980035b0 con 0x7fe2ac15ea60 2023-11-24T06:11:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2abfff700 1 -- 172.21.15.97:0/1417109635 >> v1:172.21.15.183:6789/0 conn(0x7fe2ac06c710 legacy=0x7fe2ac06ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2abfff700 1 -- 172.21.15.97:0/1417109635 >> v1:172.21.15.97:6789/0 conn(0x7fe2ac15ea60 legacy=0x7fe2ac06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.411+0000 7fe2abfff700 1 -- 172.21.15.97:0/1417109635 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac0b3e50 con 0x7fe2ac144660 2023-11-24T06:11:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2abfff700 1 --2- 172.21.15.97:0/1417109635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac144660 0x7fe2ac0b4650 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009840 tx=0x7fe29c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82e3cde06868fff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2aaffd700 1 -- 172.21.15.97:0/1417109635 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294001670 con 0x7fe2ac144660 2023-11-24T06:11:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2aaffd700 1 -- 172.21.15.97:0/1417109635 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c004030 con 0x7fe2ac144660 2023-11-24T06:11:33.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2aaffd700 1 -- 172.21.15.97:0/1417109635 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0043c0 con 0x7fe2ac144660 2023-11-24T06:11:33.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2b2612700 1 -- 172.21.15.97:0/1417109635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac144660 msgr2=0x7fe2ac0b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2b2612700 1 --2- 172.21.15.97:0/1417109635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac144660 0x7fe2ac0b4650 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009840 tx=0x7fe29c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:33.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2b2612700 1 -- 172.21.15.97:0/1417109635 shutdown_connections 2023-11-24T06:11:33.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2b2612700 1 --2- 172.21.15.97:0/1417109635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac144660 0x7fe2ac0b4650 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2b2612700 1 -- 172.21.15.97:0/1417109635 >> 172.21.15.97:0/1417109635 conn(0x7fe2ac071ac0 msgr2=0x7fe2ac071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:33.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2b2612700 1 -- 172.21.15.97:0/1417109635 shutdown_connections 2023-11-24T06:11:33.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.412+0000 7fe2b2612700 1 -- 172.21.15.97:0/1417109635 wait complete. 2023-11-24T06:11:33.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.413+0000 7fe2b2612700 1 Processor -- start 2023-11-24T06:11:33.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2b2612700 1 -- start start 2023-11-24T06:11:33.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2b2612700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2ac06c710 0x7fe2ac1f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2b2612700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2ac144660 0x7fe2ac13a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2b2612700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac15ea60 0x7fe2ac0ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2b2612700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0b0500 con 0x7fe2ac06c710 2023-11-24T06:11:33.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2b2612700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0abde0 con 0x7fe2ac15ea60 2023-11-24T06:11:33.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2b2612700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac1f2230 con 0x7fe2ac144660 2023-11-24T06:11:33.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2abfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2ac06c710 0x7fe2ac1f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2abfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2ac06c710 0x7fe2ac1f1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49996/0 (socket says 172.21.15.97:49996) 2023-11-24T06:11:33.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.414+0000 7fe2abfff700 1 -- 172.21.15.97:0/3402163029 learned_addr learned my addr 172.21.15.97:0/3402163029 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:33.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2ab7fe700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2ac144660 0x7fe2ac13a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2b0baf700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac15ea60 0x7fe2ac0ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2abfff700 1 -- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2ac144660 msgr2=0x7fe2ac13a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2abfff700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2ac144660 0x7fe2ac13a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2abfff700 1 -- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac15ea60 msgr2=0x7fe2ac0ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2abfff700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac15ea60 0x7fe2ac0ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2abfff700 1 -- 172.21.15.97:0/3402163029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c0094a0 con 0x7fe2ac06c710 2023-11-24T06:11:33.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2b0baf700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac15ea60 0x7fe2ac0ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:33.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2abfff700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2ac06c710 0x7fe2ac1f1cf0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0090b0 tx=0x7fe29c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2a8ff9700 1 -- 172.21.15.97:0/3402163029 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c011830 con 0x7fe2ac06c710 2023-11-24T06:11:33.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2a8ff9700 1 -- 172.21.15.97:0/3402163029 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29c01f070 con 0x7fe2ac06c710 2023-11-24T06:11:33.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac0afc60 con 0x7fe2ac06c710 2023-11-24T06:11:33.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2a8ff9700 1 -- 172.21.15.97:0/3402163029 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29c011bd0 con 0x7fe2ac06c710 2023-11-24T06:11:33.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.415+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac0b0220 con 0x7fe2ac06c710 2023-11-24T06:11:33.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.416+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac15eeb0 con 0x7fe2ac06c710 2023-11-24T06:11:33.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.417+0000 7fe2a8ff9700 1 -- 172.21.15.97:0/3402163029 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe29c011d30 con 0x7fe2ac06c710 2023-11-24T06:11:33.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.417+0000 7fe2a8ff9700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe28c06b2c0 0x7fe28c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.417+0000 7fe2a8ff9700 1 -- 172.21.15.97:0/3402163029 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fe29c021070 con 0x7fe2ac06c710 2023-11-24T06:11:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.417+0000 7fe2ab7fe700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe28c06b2c0 0x7fe28c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.417+0000 7fe2ab7fe700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe28c06b2c0 0x7fe28c06d780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac15a560 tx=0x7fe294001bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.421+0000 7fe2a8ff9700 1 -- 172.21.15.97:0/3402163029 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe29c056240 con 0x7fe2ac06c710 2023-11-24T06:11:33.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.612+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe2ac0ac050 con 0x7fe2ac06c710 2023-11-24T06:11:33.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.612+0000 7fe2a8ff9700 1 -- 172.21.15.97:0/3402163029 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe29c059860 con 0x7fe2ac06c710 2023-11-24T06:11:33.615 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:33.615 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:11:33.615 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:33.615 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-11-24T06:11:33.616 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-11-24T06:11:33.616 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:33.616 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:11:33.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe28c06b2c0 msgr2=0x7fe28c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe28c06b2c0 0x7fe28c06d780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac15a560 tx=0x7fe294001bc0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2ac06c710 msgr2=0x7fe2ac1f1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2ac06c710 0x7fe2ac1f1cf0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0090b0 tx=0x7fe29c004750 comp rx=0 tx=0).stop 2023-11-24T06:11:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 shutdown_connections 2023-11-24T06:11:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac15ea60 0x7fe2ac0ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2ac06c710 0x7fe2ac1f1cf0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe28c06b2c0 0x7fe28c06d780 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 --2- 172.21.15.97:0/3402163029 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe2ac144660 0x7fe2ac13a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.614+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 >> 172.21.15.97:0/3402163029 conn(0x7fe2ac071ac0 msgr2=0x7fe2ac0cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:33.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.615+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 shutdown_connections 2023-11-24T06:11:33.620 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.615+0000 7fe2b2612700 1 -- 172.21.15.97:0/3402163029 wait complete. 2023-11-24T06:11:33.630 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:11:33.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:11:33.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.752+0000 7fc830419700 1 Processor -- start 2023-11-24T06:11:33.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.752+0000 7fc830419700 1 -- start start 2023-11-24T06:11:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.753+0000 7fc830419700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc828145620 0x7fc82806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.753+0000 7fc830419700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8280cf530 con 0x7fc8280cd810 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.753+0000 7fc830419700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82806b8f0 con 0x7fc828073f90 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.753+0000 7fc830419700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82806b700 con 0x7fc828145620 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.753+0000 7fc82e1b5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc828145620 0x7fc82806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.753+0000 7fc82d9b4700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc8280cd810 0x7fc82806aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42460/0 (socket says 172.21.15.97:42460) 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.753+0000 7fc82e1b5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc828145620 0x7fc82806dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50004/0 (socket says 172.21.15.97:50004) 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.753+0000 7fc82d9b4700 1 -- 172.21.15.97:0/1238549133 learned_addr learned my addr 172.21.15.97:0/1238549133 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc82e1b5700 1 -- 172.21.15.97:0/1238549133 >> v1:172.21.15.183:6789/0 conn(0x7fc828073f90 legacy=0x7fc828074390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc82e1b5700 1 -- 172.21.15.97:0/1238549133 >> v1:172.21.15.97:6789/0 conn(0x7fc8280cd810 legacy=0x7fc82806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc82e1b5700 1 -- 172.21.15.97:0/1238549133 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82806bbd0 con 0x7fc828145620 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc82e1b5700 1 --2- 172.21.15.97:0/1238549133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828145620 0x7fc82806dc00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc818009840 tx=0x7fc818009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea28c1188ad2810 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc82d1b3700 1 -- 172.21.15.97:0/1238549133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818011830 con 0x7fc828145620 2023-11-24T06:11:33.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc82d1b3700 1 -- 172.21.15.97:0/1238549133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818011990 con 0x7fc828145620 2023-11-24T06:11:33.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc82d1b3700 1 -- 172.21.15.97:0/1238549133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818011d20 con 0x7fc828145620 2023-11-24T06:11:33.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc830419700 1 -- 172.21.15.97:0/1238549133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828145620 msgr2=0x7fc82806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc830419700 1 --2- 172.21.15.97:0/1238549133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828145620 0x7fc82806dc00 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc818009840 tx=0x7fc818009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc830419700 1 -- 172.21.15.97:0/1238549133 shutdown_connections 2023-11-24T06:11:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc830419700 1 --2- 172.21.15.97:0/1238549133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc828145620 0x7fc82806dc00 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.754+0000 7fc830419700 1 -- 172.21.15.97:0/1238549133 >> 172.21.15.97:0/1238549133 conn(0x7fc8280d0b40 msgr2=0x7fc8280d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.755+0000 7fc830419700 1 -- 172.21.15.97:0/1238549133 shutdown_connections 2023-11-24T06:11:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.755+0000 7fc830419700 1 -- 172.21.15.97:0/1238549133 wait complete. 2023-11-24T06:11:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.756+0000 7fc830419700 1 Processor -- start 2023-11-24T06:11:33.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.756+0000 7fc830419700 1 -- start start 2023-11-24T06:11:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.756+0000 7fc830419700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc828073f90 0x7fc828159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.756+0000 7fc830419700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8280cd810 0x7fc828138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.756+0000 7fc830419700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc828145620 0x7fc828139480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.756+0000 7fc830419700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b5900 con 0x7fc8280cd810 2023-11-24T06:11:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.756+0000 7fc830419700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828143760 con 0x7fc828073f90 2023-11-24T06:11:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.756+0000 7fc830419700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b4d10 con 0x7fc828145620 2023-11-24T06:11:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8280cd810 0x7fc828138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8280cd810 0x7fc828138f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50012/0 (socket says 172.21.15.97:50012) 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 -- 172.21.15.97:0/132442343 learned_addr learned my addr 172.21.15.97:0/132442343 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 -- 172.21.15.97:0/132442343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc828145620 msgr2=0x7fc828139480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc828145620 0x7fc828139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 -- 172.21.15.97:0/132442343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc828073f90 msgr2=0x7fc828159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82e1b5700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc828073f90 0x7fc828159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc828073f90 0x7fc828159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 -- 172.21.15.97:0/132442343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180094a0 con 0x7fc8280cd810 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc82d9b4700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8280cd810 0x7fc828138f40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c500 tx=0x7fc82400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc81effd700 1 -- 172.21.15.97:0/132442343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824005d70 con 0x7fc8280cd810 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc81effd700 1 -- 172.21.15.97:0/132442343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8240070d0 con 0x7fc8280cd810 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8280b4f60 con 0x7fc8280cd810 2023-11-24T06:11:33.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc81effd700 1 -- 172.21.15.97:0/132442343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824007440 con 0x7fc8280cd810 2023-11-24T06:11:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.757+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8280b5520 con 0x7fc8280cd810 2023-11-24T06:11:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.758+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc828067f60 con 0x7fc8280cd810 2023-11-24T06:11:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.759+0000 7fc81effd700 1 -- 172.21.15.97:0/132442343 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fc8240075a0 con 0x7fc8280cd810 2023-11-24T06:11:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.759+0000 7fc81effd700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc80c06b2c0 0x7fc80c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.759+0000 7fc81effd700 1 -- 172.21.15.97:0/132442343 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fc824088db0 con 0x7fc8280cd810 2023-11-24T06:11:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.759+0000 7fc82e1b5700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc80c06b2c0 0x7fc80c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:33.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.759+0000 7fc82e1b5700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc80c06b2c0 0x7fc80c06d780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc82813a6e0 tx=0x7fc818008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:33.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.763+0000 7fc81effd700 1 -- 172.21.15.97:0/132442343 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc82408d070 con 0x7fc8280cd810 2023-11-24T06:11:33.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:33.954+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc8280cf9a0 con 0x7fc8280cd810 2023-11-24T06:11:34.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.175+0000 7fc81effd700 1 -- 172.21.15.97:0/132442343 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc8280cf9a0 con 0x7fc8280cd810 2023-11-24T06:11:34.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc80c06b2c0 msgr2=0x7fc80c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc80c06b2c0 0x7fc80c06d780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc82813a6e0 tx=0x7fc818008030 comp rx=0 tx=0).stop 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8280cd810 msgr2=0x7fc828138f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8280cd810 0x7fc828138f40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c500 tx=0x7fc82400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 shutdown_connections 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc80c06b2c0 0x7fc80c06d780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc828145620 0x7fc828139480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc828073f90 0x7fc828159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 --2- 172.21.15.97:0/132442343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8280cd810 0x7fc828138f40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 >> 172.21.15.97:0/132442343 conn(0x7fc8280d0b40 msgr2=0x7fc828074390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 shutdown_connections 2023-11-24T06:11:34.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.177+0000 7fc830419700 1 -- 172.21.15.97:0/132442343 wait complete. 2023-11-24T06:11:34.182 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-11-24T06:11:34.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:11:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.320+0000 7f8d8c5f6700 1 Processor -- start 2023-11-24T06:11:34.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.320+0000 7f8d8c5f6700 1 -- start start 2023-11-24T06:11:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8c5f6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f8d8415e290 0x7f8d840ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8c5f6700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d840ad870 0x7f8d8406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8c5f6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8406e400 con 0x7f8d840aedc0 2023-11-24T06:11:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8c5f6700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8406e630 con 0x7f8d840ad870 2023-11-24T06:11:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8c5f6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8406e910 con 0x7f8d8415e290 2023-11-24T06:11:34.325 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8ab93700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d840ad870 0x7f8d8406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8ab93700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d840ad870 0x7f8d8406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50018/0 (socket says 172.21.15.97:50018) 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8ab93700 1 -- 172.21.15.97:0/1290693231 learned_addr learned my addr 172.21.15.97:0/1290693231 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8ab93700 1 -- 172.21.15.97:0/1290693231 >> v2:172.21.15.183:3300/0 conn(0x7f8d8415e290 msgr2=0x7f8d840ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8ab93700 1 --2- 172.21.15.97:0/1290693231 >> v2:172.21.15.183:3300/0 conn(0x7f8d8415e290 0x7f8d840ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8ab93700 1 -- 172.21.15.97:0/1290693231 >> v1:172.21.15.183:6789/0 conn(0x7f8d840aedc0 legacy=0x7f8d840af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.321+0000 7f8d8ab93700 1 -- 172.21.15.97:0/1290693231 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8406ebf0 con 0x7f8d840ad870 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8ab93700 1 --2- 172.21.15.97:0/1290693231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d840ad870 0x7f8d8406f6d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8d840712f0 tx=0x7f8d7c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a086c4e7ddba1344 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d89390700 1 -- 172.21.15.97:0/1290693231 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c00e070 con 0x7f8d840ad870 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d89390700 1 -- 172.21.15.97:0/1290693231 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c003a20 con 0x7f8d840ad870 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d89390700 1 -- 172.21.15.97:0/1290693231 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c003dd0 con 0x7f8d840ad870 2023-11-24T06:11:34.326 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/1290693231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d840ad870 msgr2=0x7f8d8406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/1290693231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d840ad870 0x7f8d8406f6d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f8d840712f0 tx=0x7f8d7c00b330 comp rx=0 tx=0).stop 2023-11-24T06:11:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/1290693231 shutdown_connections 2023-11-24T06:11:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/1290693231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d840ad870 0x7f8d8406f6d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/1290693231 >> v2:172.21.15.183:3300/0 conn(0x7f8d8415e290 0x7f8d840ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/1290693231 >> 172.21.15.97:0/1290693231 conn(0x7f8d840b2050 msgr2=0x7f8d840b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/1290693231 shutdown_connections 2023-11-24T06:11:34.327 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.322+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/1290693231 wait complete. 2023-11-24T06:11:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8c5f6700 1 Processor -- start 2023-11-24T06:11:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8c5f6700 1 -- start start 2023-11-24T06:11:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8c5f6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d840ad870 0x7f8d8414b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8c5f6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d840aedc0 0x7f8d840cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8c5f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d8415e290 0x7f8d840b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8c5f6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8414b570 con 0x7f8d8415e290 2023-11-24T06:11:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8c5f6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840ce3a0 con 0x7f8d840aedc0 2023-11-24T06:11:34.330 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8c5f6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84152d30 con 0x7f8d840ad870 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8ab93700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d8415e290 0x7f8d840b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8ab93700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d8415e290 0x7f8d840b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50026/0 (socket says 172.21.15.97:50026) 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.324+0000 7f8d8ab93700 1 -- 172.21.15.97:0/2644737004 learned_addr learned my addr 172.21.15.97:0/2644737004 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8ab93700 1 -- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d840ad870 msgr2=0x7f8d8414b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8a392700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d840ad870 0x7f8d8414b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d89b91700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d840aedc0 0x7f8d840cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8ab93700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d840ad870 0x7f8d8414b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8ab93700 1 -- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d840aedc0 msgr2=0x7f8d840cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8ab93700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d840aedc0 0x7f8d840cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8ab93700 1 -- 172.21.15.97:0/2644737004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c009b30 con 0x7f8d8415e290 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8a392700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d840ad870 0x7f8d8414b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d89b91700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d840aedc0 0x7f8d840cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:34.331 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8ab93700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d8415e290 0x7f8d840b0bc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c015040 tx=0x7f8d7c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d7affd700 1 -- 172.21.15.97:0/2644737004 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c004240 con 0x7f8d8415e290 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d7affd700 1 -- 172.21.15.97:0/2644737004 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c0043a0 con 0x7f8d8415e290 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d84153010 con 0x7f8d8415e290 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d7affd700 1 -- 172.21.15.97:0/2644737004 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c004750 con 0x7f8d8415e290 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.325+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d840ce680 con 0x7f8d8415e290 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.326+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d84153660 con 0x7f8d8415e290 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.327+0000 7f8d7affd700 1 -- 172.21.15.97:0/2644737004 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f8d7c019070 con 0x7f8d8415e290 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.327+0000 7f8d7affd700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8d6806b2c0 0x7f8d6806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.327+0000 7f8d7affd700 1 -- 172.21.15.97:0/2644737004 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f8d7c08ad00 con 0x7f8d8415e290 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.327+0000 7f8d8a392700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8d6806b2c0 0x7f8d6806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:34.332 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.327+0000 7f8d8a392700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8d6806b2c0 0x7f8d6806d780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d74001750 tx=0x7f8d74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:34.334 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.332+0000 7f8d7affd700 1 -- 172.21.15.97:0/2644737004 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d7c059320 con 0x7f8d8415e290 2023-11-24T06:11:34.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:34.522+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8d84068d90 con 0x7f8d8415e290 2023-11-24T06:11:35.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.180+0000 7f8d7affd700 1 -- 172.21.15.97:0/2644737004 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8d7c0200d0 con 0x7f8d8415e290 2023-11-24T06:11:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8d6806b2c0 msgr2=0x7f8d6806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8d6806b2c0 0x7f8d6806d780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d74001750 tx=0x7f8d74009040 comp rx=0 tx=0).stop 2023-11-24T06:11:35.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d8415e290 msgr2=0x7f8d840b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d8415e290 0x7f8d840b0bc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c015040 tx=0x7f8d7c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 shutdown_connections 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d8415e290 0x7f8d840b0bc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8d6806b2c0 0x7f8d6806d780 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8d840ad870 0x7f8d8414b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 --2- 172.21.15.97:0/2644737004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d840aedc0 0x7f8d840cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 >> 172.21.15.97:0/2644737004 conn(0x7f8d840b2050 msgr2=0x7f8d8415b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 shutdown_connections 2023-11-24T06:11:35.186 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.182+0000 7f8d8c5f6700 1 -- 172.21.15.97:0/2644737004 wait complete. 2023-11-24T06:11:35.187 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-11-24T06:11:35.208 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:11:35.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.330+0000 7f608f1cb700 1 Processor -- start 2023-11-24T06:11:35.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.330+0000 7f608f1cb700 1 -- start start 2023-11-24T06:11:35.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f608f1cb700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f608815e160 0x7f60880b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f608f1cb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880afc90 0x7f6088073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f608f1cb700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60880b3c60 con 0x7f60880744d0 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f608f1cb700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60880b4500 con 0x7f608815e160 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f608f1cb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6088074960 con 0x7f60880afc90 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f608d768700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f60880744d0 0x7f608806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51808/0 (socket says 172.21.15.97:51808) 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f608d768700 1 -- 172.21.15.97:0/3958467894 learned_addr learned my addr 172.21.15.97:0/3958467894 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f608cf67700 1 --2- 172.21.15.97:0/3958467894 >> v2:172.21.15.183:3300/0 conn(0x7f608815e160 0x7f60880b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.331+0000 7f6087fff700 1 --2- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880afc90 0x7f6088073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f60877fe700 1 -- 172.21.15.97:0/3958467894 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765308093 0 0) 0x7f60880b3c60 con 0x7f60880744d0 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f60877fe700 1 -- 172.21.15.97:0/3958467894 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60780035b0 con 0x7f60880744d0 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f608cf67700 1 -- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880afc90 msgr2=0x7f6088073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f608cf67700 1 --2- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880afc90 0x7f6088073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f608cf67700 1 -- 172.21.15.97:0/3958467894 >> v1:172.21.15.183:6789/0 conn(0x7f60880744d0 legacy=0x7f608806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f60877fe700 1 -- 172.21.15.97:0/3958467894 <== mon.1 v1:172.21.15.183:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970806462 0 0) 0x7f60780035b0 con 0x7f60880744d0 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f608cf67700 1 -- 172.21.15.97:0/3958467894 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60880b58c0 con 0x7f608815e160 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f6087fff700 1 --2- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880afc90 0x7f6088073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f608cf67700 1 --2- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 0x7f60880b5da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f607c009840 tx=0x7f607c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=888755bb37ffd0a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f60877fe700 1 -- 172.21.15.97:0/3958467894 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60800019f0 con 0x7f608815e160 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f60877fe700 1 -- 172.21.15.97:0/3958467894 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607c004030 con 0x7f608815e160 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.332+0000 7f60877fe700 1 -- 172.21.15.97:0/3958467894 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607c0043c0 con 0x7f608815e160 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.333+0000 7f608f1cb700 1 -- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 msgr2=0x7f60880b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.333+0000 7f608f1cb700 1 --2- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 0x7f60880b5da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f607c009840 tx=0x7f607c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.333+0000 7f608f1cb700 1 -- 172.21.15.97:0/3958467894 shutdown_connections 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.333+0000 7f608f1cb700 1 --2- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 0x7f60880b5da0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.333+0000 7f608f1cb700 1 --2- 172.21.15.97:0/3958467894 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880afc90 0x7f6088073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.333+0000 7f608f1cb700 1 -- 172.21.15.97:0/3958467894 >> 172.21.15.97:0/3958467894 conn(0x7f60880d0bf0 msgr2=0x7f60880d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:35.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.333+0000 7f608f1cb700 1 -- 172.21.15.97:0/3958467894 shutdown_connections 2023-11-24T06:11:35.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.333+0000 7f608f1cb700 1 -- 172.21.15.97:0/3958467894 wait complete. 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.334+0000 7f608f1cb700 1 Processor -- start 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.334+0000 7f608f1cb700 1 -- start start 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f608f1cb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880744d0 0x7f608815f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f608f1cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60880afc90 0x7f608815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f608f1cb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 0x7f60881604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f608f1cb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60881567d0 con 0x7f60880afc90 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f608f1cb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6088155f00 con 0x7f608815e160 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f608f1cb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6088147f20 con 0x7f60880744d0 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60880afc90 0x7f608815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60880afc90 0x7f608815f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50038/0 (socket says 172.21.15.97:50038) 2023-11-24T06:11:35.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 -- 172.21.15.97:0/381712367 learned_addr learned my addr 172.21.15.97:0/381712367 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f608d768700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 0x7f60881604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 -- 172.21.15.97:0/381712367 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880744d0 msgr2=0x7f608815f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880744d0 0x7f608815f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 -- 172.21.15.97:0/381712367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 msgr2=0x7f60881604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 0x7f60881604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 -- 172.21.15.97:0/381712367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60740094a0 con 0x7f60880afc90 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.335+0000 7f6087fff700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60880afc90 0x7f608815f640 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f607400ebe0 tx=0x7f607400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.336+0000 7f60857fa700 1 -- 172.21.15.97:0/381712367 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074009800 con 0x7f60880afc90 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.336+0000 7f60857fa700 1 -- 172.21.15.97:0/381712367 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607400c910 con 0x7f60880afc90 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.336+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c0094a0 con 0x7f60880afc90 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.336+0000 7f60857fa700 1 -- 172.21.15.97:0/381712367 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400cc80 con 0x7f60880afc90 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.336+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6088156110 con 0x7f60880afc90 2023-11-24T06:11:35.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.337+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60880b51c0 con 0x7f60880afc90 2023-11-24T06:11:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.337+0000 7f60857fa700 1 -- 172.21.15.97:0/381712367 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6074013470 con 0x7f60880afc90 2023-11-24T06:11:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.338+0000 7f60857fa700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f606c06b2c0 0x7f606c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.338+0000 7f60857fa700 1 -- 172.21.15.97:0/381712367 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f6074089dd0 con 0x7f60880afc90 2023-11-24T06:11:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.338+0000 7f608cf67700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f606c06b2c0 0x7f606c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:35.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.338+0000 7f608cf67700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f606c06b2c0 0x7f606c06d780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f607c000c00 tx=0x7f607c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:35.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.343+0000 7f60857fa700 1 -- 172.21.15.97:0/381712367 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6074054970 con 0x7f60880afc90 2023-11-24T06:11:35.582 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:35.579+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6088068fb0 con 0x7f60880afc90 2023-11-24T06:11:36.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.181+0000 7f60857fa700 1 -- 172.21.15.97:0/381712367 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7f60740170c0 con 0x7f60880afc90 2023-11-24T06:11:36.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.184+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f606c06b2c0 msgr2=0x7f606c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.184+0000 7f608f1cb700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f606c06b2c0 0x7f606c06d780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f607c000c00 tx=0x7f607c010000 comp rx=0 tx=0).stop 2023-11-24T06:11:36.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.184+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60880afc90 msgr2=0x7f608815f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.184+0000 7f608f1cb700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60880afc90 0x7f608815f640 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f607400ebe0 tx=0x7f607400eef0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.185+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 shutdown_connections 2023-11-24T06:11:36.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.185+0000 7f608f1cb700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f608815e160 0x7f60881604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.185+0000 7f608f1cb700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f606c06b2c0 0x7f606c06d780 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.185+0000 7f608f1cb700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f60880744d0 0x7f608815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.185+0000 7f608f1cb700 1 --2- 172.21.15.97:0/381712367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60880afc90 0x7f608815f640 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.185+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 >> 172.21.15.97:0/381712367 conn(0x7f60880d0bf0 msgr2=0x7f608815c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:36.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.185+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 shutdown_connections 2023-11-24T06:11:36.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.185+0000 7f608f1cb700 1 -- 172.21.15.97:0/381712367 wait complete. 2023-11-24T06:11:36.194 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 10 and data pool 11 2023-11-24T06:11:36.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:36.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.340+0000 7f4b8f341700 1 Processor -- start 2023-11-24T06:11:36.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.341+0000 7f4b8f341700 1 -- start start 2023-11-24T06:11:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.341+0000 7f4b8f341700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4b8815e080 0x7f4b880b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.341+0000 7f4b8f341700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4b8806c710 0x7f4b880afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.341+0000 7f4b8f341700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b880ae430 con 0x7f4b880af4d0 2023-11-24T06:11:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.341+0000 7f4b8f341700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880af0b0 con 0x7f4b8806c710 2023-11-24T06:11:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8f341700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b880b05a0 con 0x7f4b8815e080 2023-11-24T06:11:36.346 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d8de700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4b8806c710 0x7f4b880afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d8de700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4b8806c710 0x7f4b880afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50048/0 (socket says 172.21.15.97:50048) 2023-11-24T06:11:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d8de700 1 -- 172.21.15.97:0/2052700041 learned_addr learned my addr 172.21.15.97:0/2052700041 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d0dd700 1 --2- 172.21.15.97:0/2052700041 >> v2:172.21.15.183:3300/0 conn(0x7f4b8815e080 0x7f4b880b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d8de700 1 -- 172.21.15.97:0/2052700041 >> v2:172.21.15.183:3300/0 conn(0x7f4b8815e080 msgr2=0x7f4b880b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d8de700 1 --2- 172.21.15.97:0/2052700041 >> v2:172.21.15.183:3300/0 conn(0x7f4b8815e080 0x7f4b880b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d8de700 1 -- 172.21.15.97:0/2052700041 >> v1:172.21.15.183:6789/0 conn(0x7f4b880af4d0 legacy=0x7f4b8806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d8de700 1 -- 172.21.15.97:0/2052700041 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b880ae710 con 0x7f4b8806c710 2023-11-24T06:11:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b8d8de700 1 --2- 172.21.15.97:0/2052700041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8806c710 0x7f4b880afb90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4b88072a90 tx=0x7f4b8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa613ff303ff088 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b7ffff700 1 -- 172.21.15.97:0/2052700041 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000e070 con 0x7f4b8806c710 2023-11-24T06:11:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b7ffff700 1 -- 172.21.15.97:0/2052700041 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8000b980 con 0x7f4b8806c710 2023-11-24T06:11:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.342+0000 7f4b7ffff700 1 -- 172.21.15.97:0/2052700041 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000bd30 con 0x7f4b8806c710 2023-11-24T06:11:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.343+0000 7f4b8f341700 1 -- 172.21.15.97:0/2052700041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8806c710 msgr2=0x7f4b880afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.343+0000 7f4b8f341700 1 --2- 172.21.15.97:0/2052700041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8806c710 0x7f4b880afb90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4b88072a90 tx=0x7f4b8000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.343+0000 7f4b8f341700 1 -- 172.21.15.97:0/2052700041 shutdown_connections 2023-11-24T06:11:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.343+0000 7f4b8f341700 1 --2- 172.21.15.97:0/2052700041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8806c710 0x7f4b880afb90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.343+0000 7f4b8f341700 1 --2- 172.21.15.97:0/2052700041 >> v2:172.21.15.183:3300/0 conn(0x7f4b8815e080 0x7f4b880b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.343+0000 7f4b8f341700 1 -- 172.21.15.97:0/2052700041 >> 172.21.15.97:0/2052700041 conn(0x7f4b88071b90 msgr2=0x7f4b88071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.343+0000 7f4b8f341700 1 -- 172.21.15.97:0/2052700041 shutdown_connections 2023-11-24T06:11:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.343+0000 7f4b8f341700 1 -- 172.21.15.97:0/2052700041 wait complete. 2023-11-24T06:11:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.344+0000 7f4b8f341700 1 Processor -- start 2023-11-24T06:11:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.345+0000 7f4b8f341700 1 -- start start 2023-11-24T06:11:36.351 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.345+0000 7f4b8f341700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b8806c710 0x7f4b88141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.345+0000 7f4b8f341700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b880af4d0 0x7f4b88160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.345+0000 7f4b8f341700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8815e080 0x7f4b88160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.345+0000 7f4b8f341700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8815a8f0 con 0x7f4b8815e080 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.345+0000 7f4b8f341700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88141fb0 con 0x7f4b8806c710 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.345+0000 7f4b8f341700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8815ab40 con 0x7f4b880af4d0 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8815e080 0x7f4b88160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8815e080 0x7f4b88160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50054/0 (socket says 172.21.15.97:50054) 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 -- 172.21.15.97:0/1251439557 learned_addr learned my addr 172.21.15.97:0/1251439557 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 -- 172.21.15.97:0/1251439557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b880af4d0 msgr2=0x7f4b88160ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b880af4d0 0x7f4b88160ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 -- 172.21.15.97:0/1251439557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b8806c710 msgr2=0x7f4b88141a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:11:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b8806c710 0x7f4b88141a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 -- 172.21.15.97:0/1251439557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80009b30 con 0x7f4b8815e080 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8d8de700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8815e080 0x7f4b88160ff0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4b80012000 tx=0x7f4b80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b7dffb700 1 -- 172.21.15.97:0/1251439557 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b800092e0 con 0x7f4b8815e080 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b7dffb700 1 -- 172.21.15.97:0/1251439557 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b80004960 con 0x7f4b8815e080 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8815ae20 con 0x7f4b8815e080 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b7dffb700 1 -- 172.21.15.97:0/1251439557 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8001c680 con 0x7f4b8815e080 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.346+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8815b3e0 con 0x7f4b8815e080 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.347+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b880753f0 con 0x7f4b8815e080 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.348+0000 7f4b7dffb700 1 -- 172.21.15.97:0/1251439557 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f4b8001c7e0 con 0x7f4b8815e080 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.348+0000 7f4b7dffb700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b6c06b2c0 0x7f4b6c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.348+0000 7f4b7dffb700 1 -- 172.21.15.97:0/1251439557 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4b8008b6c0 con 0x7f4b8815e080 2023-11-24T06:11:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.348+0000 7f4b8d0dd700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b6c06b2c0 0x7f4b6c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.349+0000 7f4b8d0dd700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b6c06b2c0 0x7f4b6c06d780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b881f1480 tx=0x7f4b78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.355 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.353+0000 7f4b7dffb700 1 -- 172.21.15.97:0/1251439557 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b800560f0 con 0x7f4b8815e080 2023-11-24T06:11:36.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.543+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b88067f60 con 0x7f4b8815e080 2023-11-24T06:11:36.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.545+0000 7f4b7dffb700 1 -- 172.21.15.97:0/1251439557 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16636 (secure 0 0 0) 0x7f4b80059710 con 0x7f4b8815e080 2023-11-24T06:11:36.548 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:36.548 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:35.580702+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:33.973262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:34.541598+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b6c06b2c0 msgr2=0x7f4b6c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b6c06b2c0 0x7f4b6c06d780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4b881f1480 tx=0x7f4b78008040 comp rx=0 tx=0).stop 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8815e080 msgr2=0x7f4b88160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8815e080 0x7f4b88160ff0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4b80012000 tx=0x7f4b80004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 shutdown_connections 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b8815e080 0x7f4b88160ff0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b8806c710 0x7f4b88141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4b6c06b2c0 0x7f4b6c06d780 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 --2- 172.21.15.97:0/1251439557 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4b880af4d0 0x7f4b88160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 >> 172.21.15.97:0/1251439557 conn(0x7f4b88071b90 msgr2=0x7f4b880b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 shutdown_connections 2023-11-24T06:11:36.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.547+0000 7f4b8f341700 1 -- 172.21.15.97:0/1251439557 wait complete. 2023-11-24T06:11:36.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:36.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.679+0000 7fef6f2c4700 1 Processor -- start 2023-11-24T06:11:36.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.679+0000 7fef6f2c4700 1 -- start start 2023-11-24T06:11:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6f2c4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef680b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6f2c4700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef680af4d0 0x7fef680afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6f2c4700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680ae430 con 0x7fef6815e080 2023-11-24T06:11:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6f2c4700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef680af0b0 con 0x7fef680af4d0 2023-11-24T06:11:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6f2c4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef680b05a0 con 0x7fef6806c710 2023-11-24T06:11:36.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6d861700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef680af4d0 0x7fef680afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6d060700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fef6815e080 0x7fef6806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42510/0 (socket says 172.21.15.97:42510) 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6d060700 1 -- 172.21.15.97:0/1482595008 learned_addr learned my addr 172.21.15.97:0/1482595008 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6c85f700 1 --2- 172.21.15.97:0/1482595008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef680b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6d861700 1 -- 172.21.15.97:0/1482595008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 msgr2=0x7fef680b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6d861700 1 --2- 172.21.15.97:0/1482595008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef680b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6d861700 1 -- 172.21.15.97:0/1482595008 >> v1:172.21.15.97:6789/0 conn(0x7fef6815e080 legacy=0x7fef6806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef5ffff700 1 -- 172.21.15.97:0/1482595008 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979233658 0 0) 0x7fef680ae430 con 0x7fef6815e080 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6d861700 1 -- 172.21.15.97:0/1482595008 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680ae710 con 0x7fef680af4d0 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.680+0000 7fef6d861700 1 --2- 172.21.15.97:0/1482595008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef680af4d0 0x7fef680afb90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fef60009dc0 tx=0x7fef6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898bdd04d489896e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef5ffff700 1 -- 172.21.15.97:0/1482595008 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6000e070 con 0x7fef680af4d0 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef5ffff700 1 -- 172.21.15.97:0/1482595008 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6000bb90 con 0x7fef680af4d0 2023-11-24T06:11:36.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef5ffff700 1 -- 172.21.15.97:0/1482595008 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60012430 con 0x7fef680af4d0 2023-11-24T06:11:36.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef6f2c4700 1 -- 172.21.15.97:0/1482595008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef680af4d0 msgr2=0x7fef680afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/1482595008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef680af4d0 0x7fef680afb90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fef60009dc0 tx=0x7fef6000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:36.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef6f2c4700 1 -- 172.21.15.97:0/1482595008 shutdown_connections 2023-11-24T06:11:36.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/1482595008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef680af4d0 0x7fef680afb90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/1482595008 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef680b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef6f2c4700 1 -- 172.21.15.97:0/1482595008 >> 172.21.15.97:0/1482595008 conn(0x7fef68071b90 msgr2=0x7fef68071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:36.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef6f2c4700 1 -- 172.21.15.97:0/1482595008 shutdown_connections 2023-11-24T06:11:36.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.681+0000 7fef6f2c4700 1 -- 172.21.15.97:0/1482595008 wait complete. 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.682+0000 7fef6f2c4700 1 Processor -- start 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6f2c4700 1 -- start start 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6f2c4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef681f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6f2c4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef680af4d0 0x7fef68160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6f2c4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6815e080 0x7fef681492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6f2c4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef681556e0 con 0x7fef6815e080 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6f2c4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68152ec0 con 0x7fef680af4d0 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6f2c4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef68161140 con 0x7fef6806c710 2023-11-24T06:11:36.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6d861700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6815e080 0x7fef681492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6d861700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6815e080 0x7fef681492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50066/0 (socket says 172.21.15.97:50066) 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.683+0000 7fef6d861700 1 -- 172.21.15.97:0/2943143594 learned_addr learned my addr 172.21.15.97:0/2943143594 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6c85f700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef680af4d0 0x7fef68160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6d060700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef681f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6d861700 1 -- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 msgr2=0x7fef681f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6d861700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef681f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6d861700 1 -- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef680af4d0 msgr2=0x7fef68160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6d861700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef680af4d0 0x7fef68160c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6d861700 1 -- 172.21.15.97:0/2943143594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef60009a70 con 0x7fef6815e080 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6c85f700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef680af4d0 0x7fef68160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:36.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6d060700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef681f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6d861700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6815e080 0x7fef681492a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fef60014040 tx=0x7fef60004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef5dffb700 1 -- 172.21.15.97:0/2943143594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60012430 con 0x7fef6815e080 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef5dffb700 1 -- 172.21.15.97:0/2943143594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60020070 con 0x7fef6815e080 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef5dffb700 1 -- 172.21.15.97:0/2943143594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60004280 con 0x7fef6815e080 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef68154d40 con 0x7fef6815e080 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.684+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef68155300 con 0x7fef6815e080 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.685+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef680753f0 con 0x7fef6815e080 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.686+0000 7fef5dffb700 1 -- 172.21.15.97:0/2943143594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fef600043e0 con 0x7fef6815e080 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.686+0000 7fef5dffb700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fef4c06b2c0 0x7fef4c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.686+0000 7fef5dffb700 1 -- 172.21.15.97:0/2943143594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fef60022070 con 0x7fef6815e080 2023-11-24T06:11:36.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.686+0000 7fef6d060700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fef4c06b2c0 0x7fef4c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:36.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.687+0000 7fef6d060700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fef4c06b2c0 0x7fef4c06d780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fef64007fd0 tx=0x7fef64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:36.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.691+0000 7fef5dffb700 1 -- 172.21.15.97:0/2943143594 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef60056a00 con 0x7fef6815e080 2023-11-24T06:11:36.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.882+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef68155820 con 0x7fef6815e080 2023-11-24T06:11:36.885 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.883+0000 7fef5dffb700 1 -- 172.21.15.97:0/2943143594 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16636 (secure 0 0 0) 0x7fef68155820 con 0x7fef6815e080 2023-11-24T06:11:36.886 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:36.886 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":71,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:35.580702+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:33.973262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:34.541598+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:36.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fef4c06b2c0 msgr2=0x7fef4c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fef4c06b2c0 0x7fef4c06d780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fef64007fd0 tx=0x7fef64009040 comp rx=0 tx=0).stop 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6815e080 msgr2=0x7fef681492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6815e080 0x7fef681492a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fef60014040 tx=0x7fef60004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 shutdown_connections 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6815e080 0x7fef681492a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fef4c06b2c0 0x7fef4c06d780 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fef6806c710 0x7fef681f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 --2- 172.21.15.97:0/2943143594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef680af4d0 0x7fef68160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.885+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 >> 172.21.15.97:0/2943143594 conn(0x7fef68071b90 msgr2=0x7fef680cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.886+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 shutdown_connections 2023-11-24T06:11:36.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:36.886+0000 7fef6f2c4700 1 -- 172.21.15.97:0/2943143594 wait complete. 2023-11-24T06:11:36.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:11:37.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.021+0000 7f3961f4f700 1 Processor -- start 2023-11-24T06:11:37.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.021+0000 7f3961f4f700 1 -- start start 2023-11-24T06:11:37.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f3961f4f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f395c15a620 0x7f395c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f3961f4f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 0x7f395c0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f3961f4f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f395c0b2f20 con 0x7f395c0cc540 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f3961f4f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395c0ce690 con 0x7f395c15a620 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f3961f4f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395c15aa00 con 0x7f395c06ecf0 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f395b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f395c15a620 0x7f395c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f3953fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f395c0cc540 0x7f395c06e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42524/0 (socket says 172.21.15.97:42524) 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f3953fff700 1 -- 172.21.15.97:0/3048718162 learned_addr learned my addr 172.21.15.97:0/3048718162 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f395b7fe700 1 -- 172.21.15.97:0/3048718162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 msgr2=0x7f395c0b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f395b7fe700 1 --2- 172.21.15.97:0/3048718162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 0x7f395c0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f395b7fe700 1 -- 172.21.15.97:0/3048718162 >> v1:172.21.15.97:6789/0 conn(0x7f395c0cc540 legacy=0x7f395c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f395affd700 1 -- 172.21.15.97:0/3048718162 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968452959 0 0) 0x7f395c0b2f20 con 0x7f395c0cc540 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.022+0000 7f395b7fe700 1 -- 172.21.15.97:0/3048718162 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c0ce960 con 0x7f395c15a620 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f395b7fe700 1 --2- 172.21.15.97:0/3048718162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c0ce150 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f394c009840 tx=0x7f394c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556fc8dce61a851e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f395affd700 1 -- 172.21.15.97:0/3048718162 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c011830 con 0x7f395c15a620 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f395affd700 1 -- 172.21.15.97:0/3048718162 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c011990 con 0x7f395c15a620 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f395affd700 1 -- 172.21.15.97:0/3048718162 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c011d20 con 0x7f395c15a620 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f3961f4f700 1 -- 172.21.15.97:0/3048718162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 msgr2=0x7f395c0ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3048718162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c0ce150 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f394c009840 tx=0x7f394c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f3961f4f700 1 -- 172.21.15.97:0/3048718162 shutdown_connections 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3048718162 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 0x7f395c0b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3048718162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c0ce150 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f3961f4f700 1 -- 172.21.15.97:0/3048718162 >> 172.21.15.97:0/3048718162 conn(0x7f395c0b2050 msgr2=0x7f395c0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f3961f4f700 1 -- 172.21.15.97:0/3048718162 shutdown_connections 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.023+0000 7f3961f4f700 1 -- 172.21.15.97:0/3048718162 wait complete. 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f3961f4f700 1 Processor -- start 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f3961f4f700 1 -- start start 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f3961f4f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 0x7f395c0aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f3961f4f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395c0cc540 0x7f395c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f3961f4f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f3961f4f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c15c6c0 con 0x7f395c15a620 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f3961f4f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c15c9a0 con 0x7f395c0cc540 2023-11-24T06:11:37.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f3961f4f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395c15cc80 con 0x7f395c06ecf0 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.025+0000 7f395bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50080/0 (socket says 172.21.15.97:50080) 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395bfff700 1 -- 172.21.15.97:0/3368647784 learned_addr learned my addr 172.21.15.97:0/3368647784 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f3953fff700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395c0cc540 0x7f395c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395bfff700 1 -- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 msgr2=0x7f395c0aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395b7fe700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 0x7f395c0aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395bfff700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 0x7f395c0aa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395bfff700 1 -- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395c0cc540 msgr2=0x7f395c15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395bfff700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395c0cc540 0x7f395c15bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395bfff700 1 -- 172.21.15.97:0/3368647784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c0094a0 con 0x7f395c15a620 2023-11-24T06:11:37.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f3953fff700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395c0cc540 0x7f395c15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f395bfff700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c144660 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f395400b9e0 tx=0x7f395400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f3958ff9700 1 -- 172.21.15.97:0/3368647784 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395400c9b0 con 0x7f395c15a620 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f3958ff9700 1 -- 172.21.15.97:0/3368647784 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395400cb10 con 0x7f395c15a620 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c15cf60 con 0x7f395c15a620 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f3958ff9700 1 -- 172.21.15.97:0/3368647784 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3954014440 con 0x7f395c15a620 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.026+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c15c300 con 0x7f395c15a620 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.027+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c068d90 con 0x7f395c15a620 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.028+0000 7f3958ff9700 1 -- 172.21.15.97:0/3368647784 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f395401d3b0 con 0x7f395c15a620 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.028+0000 7f3958ff9700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f393c06b2f0 0x7f393c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.028+0000 7f395b7fe700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f393c06b2f0 0x7f393c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.028+0000 7f3958ff9700 1 -- 172.21.15.97:0/3368647784 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3954006e40 con 0x7f395c15a620 2023-11-24T06:11:37.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.028+0000 7f395b7fe700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f393c06b2f0 0x7f393c06d7b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f394c0090b0 tx=0x7f394c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:37.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.033+0000 7f3958ff9700 1 -- 172.21.15.97:0/3368647784 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3954054470 con 0x7f395c15a620 2023-11-24T06:11:37.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:37.273+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f395c068fb0 con 0x7f395c15a620 2023-11-24T06:11:38.171 INFO:tasks.ceph.mds.b.smithi183.stderr:2023-11-24T06:11:38.169+0000 7f107c364700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:11:38.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.215+0000 7f3958ff9700 1 -- 172.21.15.97:0/3368647784 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3954057a90 con 0x7f395c15a620 2023-11-24T06:11:38.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f393c06b2f0 msgr2=0x7f393c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f393c06b2f0 0x7f393c06d7b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f394c0090b0 tx=0x7f394c008030 comp rx=0 tx=0).stop 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 msgr2=0x7f395c144660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c144660 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f395400b9e0 tx=0x7f395400bda0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 shutdown_connections 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f395c15a620 0x7f395c144660 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f393c06b2f0 0x7f393c06d7b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f395c06ecf0 0x7f395c0aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 --2- 172.21.15.97:0/3368647784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395c0cc540 0x7f395c15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 >> 172.21.15.97:0/3368647784 conn(0x7f395c0b2050 msgr2=0x7f395c14e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 shutdown_connections 2023-11-24T06:11:38.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.218+0000 7f3961f4f700 1 -- 172.21.15.97:0/3368647784 wait complete. 2023-11-24T06:11:38.234 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:11:38.235 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.361+0000 7fd111945700 1 Processor -- start 2023-11-24T06:11:38.364 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.361+0000 7fd111945700 1 -- start start 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd111945700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd10c0d45d0 0x7fd10c0b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd111945700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd10c0d39d0 0x7fd10c0b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd111945700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10c0d2930 con 0x7fd10c15e140 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd111945700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10c15e520 con 0x7fd10c0d39d0 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd111945700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10c0d2c10 con 0x7fd10c0d45d0 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd10b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd10c0d39d0 0x7fd10c0b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd10affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd10c15e140 0x7fd10c0d3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42536/0 (socket says 172.21.15.97:42536) 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd10affd700 1 -- 172.21.15.97:0/2608608300 learned_addr learned my addr 172.21.15.97:0/2608608300 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd10b7fe700 1 -- 172.21.15.97:0/2608608300 >> v2:172.21.15.183:3300/0 conn(0x7fd10c0d45d0 msgr2=0x7fd10c0b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:38.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd10b7fe700 1 --2- 172.21.15.97:0/2608608300 >> v2:172.21.15.183:3300/0 conn(0x7fd10c0d45d0 0x7fd10c0b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd10b7fe700 1 -- 172.21.15.97:0/2608608300 >> v1:172.21.15.97:6789/0 conn(0x7fd10c15e140 legacy=0x7fd10c0d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd10b7fe700 1 -- 172.21.15.97:0/2608608300 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0d2ef0 con 0x7fd10c0d39d0 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd109ffb700 1 -- 172.21.15.97:0/2608608300 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172441621 0 0) 0x7fd10c0d2930 con 0x7fd10c15e140 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd10b7fe700 1 --2- 172.21.15.97:0/2608608300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d39d0 0x7fd10c0b7610 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0d4b20 tx=0x7fd10400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb6bdb205e54988 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.362+0000 7fd109ffb700 1 -- 172.21.15.97:0/2608608300 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10400e070 con 0x7fd10c0d39d0 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd109ffb700 1 -- 172.21.15.97:0/2608608300 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10400bbd0 con 0x7fd10c0d39d0 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd109ffb700 1 -- 172.21.15.97:0/2608608300 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104012430 con 0x7fd10c0d39d0 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd111945700 1 -- 172.21.15.97:0/2608608300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d39d0 msgr2=0x7fd10c0b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd111945700 1 --2- 172.21.15.97:0/2608608300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d39d0 0x7fd10c0b7610 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0d4b20 tx=0x7fd10400b330 comp rx=0 tx=0).stop 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd111945700 1 -- 172.21.15.97:0/2608608300 shutdown_connections 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd111945700 1 --2- 172.21.15.97:0/2608608300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d39d0 0x7fd10c0b7610 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd111945700 1 --2- 172.21.15.97:0/2608608300 >> v2:172.21.15.183:3300/0 conn(0x7fd10c0d45d0 0x7fd10c0b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd111945700 1 -- 172.21.15.97:0/2608608300 >> 172.21.15.97:0/2608608300 conn(0x7fd10c06b410 msgr2=0x7fd10c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd111945700 1 -- 172.21.15.97:0/2608608300 shutdown_connections 2023-11-24T06:11:38.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.363+0000 7fd111945700 1 -- 172.21.15.97:0/2608608300 wait complete. 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.364+0000 7fd111945700 1 Processor -- start 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd111945700 1 -- start start 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd111945700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10c0d39d0 0x7fd10c15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd111945700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d45d0 0x7fd10c1f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd111945700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd10c15e140 0x7fd10c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd111945700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0c9730 con 0x7fd10c0d45d0 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd111945700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0c8ab0 con 0x7fd10c0d39d0 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd111945700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10c0c8d90 con 0x7fd10c15e140 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd10a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d45d0 0x7fd10c1f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd10a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d45d0 0x7fd10c1f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50090/0 (socket says 172.21.15.97:50090) 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.365+0000 7fd10a7fc700 1 -- 172.21.15.97:0/1896557568 learned_addr learned my addr 172.21.15.97:0/1896557568 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd10a7fc700 1 -- 172.21.15.97:0/1896557568 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd10c15e140 msgr2=0x7fd10c0cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd10b7fe700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd10c15e140 0x7fd10c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd10affd700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10c0d39d0 0x7fd10c15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd10a7fc700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd10c15e140 0x7fd10c0cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd10a7fc700 1 -- 172.21.15.97:0/1896557568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10c0d39d0 msgr2=0x7fd10c15a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd10a7fc700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10c0d39d0 0x7fd10c15a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd10a7fc700 1 -- 172.21.15.97:0/1896557568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104009a70 con 0x7fd10c0d45d0 2023-11-24T06:11:38.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd10a7fc700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d45d0 0x7fd10c1f36f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b530 tx=0x7fd0fc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd0f3fff700 1 -- 172.21.15.97:0/1896557568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc011830 con 0x7fd10c0d45d0 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd0f3fff700 1 -- 172.21.15.97:0/1896557568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0fc0119b0 con 0x7fd10c0d45d0 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10c0c9070 con 0x7fd10c0d45d0 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd0f3fff700 1 -- 172.21.15.97:0/1896557568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc011d60 con 0x7fd10c0d45d0 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.366+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10c15a840 con 0x7fd10c0d45d0 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.367+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd10c068fb0 con 0x7fd10c0d45d0 2023-11-24T06:11:38.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.367+0000 7fd0f3fff700 1 -- 172.21.15.97:0/1896557568 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd0fc01a400 con 0x7fd10c0d45d0 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.368+0000 7fd0f3fff700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0f806b2c0 0x7fd0f806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.368+0000 7fd0f3fff700 1 -- 172.21.15.97:0/1896557568 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd0fc089de0 con 0x7fd10c0d45d0 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.368+0000 7fd10affd700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0f806b2c0 0x7fd0f806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:38.373 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.368+0000 7fd10affd700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0f806b2c0 0x7fd0f806d780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd100000f60 tx=0x7fd100009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:38.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.372+0000 7fd0f3fff700 1 -- 172.21.15.97:0/1896557568 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0fc0548e0 con 0x7fd10c0d45d0 2023-11-24T06:11:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:38.606+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd10c0d1bb0 con 0x7fd10c0d45d0 2023-11-24T06:11:39.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.220+0000 7fd0f3fff700 1 -- 172.21.15.97:0/1896557568 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd0fc015020 con 0x7fd10c0d45d0 2023-11-24T06:11:39.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.223+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0f806b2c0 msgr2=0x7fd0f806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.223+0000 7fd111945700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0f806b2c0 0x7fd0f806d780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd100000f60 tx=0x7fd100009040 comp rx=0 tx=0).stop 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.223+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d45d0 msgr2=0x7fd10c1f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.223+0000 7fd111945700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d45d0 0x7fd10c1f36f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00b530 tx=0x7fd0fc00b8f0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.224+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 shutdown_connections 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.224+0000 7fd111945700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd0f806b2c0 0x7fd0f806d780 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.224+0000 7fd111945700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd10c15e140 0x7fd10c0cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.224+0000 7fd111945700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd10c0d39d0 0x7fd10c15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.224+0000 7fd111945700 1 --2- 172.21.15.97:0/1896557568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd10c0d45d0 0x7fd10c1f36f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.224+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 >> 172.21.15.97:0/1896557568 conn(0x7fd10c06b410 msgr2=0x7fd10c06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:39.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.224+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 shutdown_connections 2023-11-24T06:11:39.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.224+0000 7fd111945700 1 -- 172.21.15.97:0/1896557568 wait complete. 2023-11-24T06:11:39.240 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:11:39.240 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:39.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.360+0000 7f386f06f700 1 Processor -- start 2023-11-24T06:11:39.362 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.360+0000 7f386f06f700 1 -- start start 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.360+0000 7f386f06f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3868144660 0x7f38680b4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.360+0000 7f386f06f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f386806c710 0x7f386815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386f06f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f386815f520 con 0x7f386815ea60 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386f06f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38680b4230 con 0x7f386806c710 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386f06f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38680b3c40 con 0x7f3868144660 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386d60c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f386806c710 0x7f386815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f3867fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f386815ea60 0x7f386806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42548/0 (socket says 172.21.15.97:42548) 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386d60c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f386806c710 0x7f386815eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50100/0 (socket says 172.21.15.97:50100) 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f3867fff700 1 -- 172.21.15.97:0/2329322539 learned_addr learned my addr 172.21.15.97:0/2329322539 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386ce0b700 1 --2- 172.21.15.97:0/2329322539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3868144660 0x7f38680b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386d60c700 1 -- 172.21.15.97:0/2329322539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3868144660 msgr2=0x7f38680b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386d60c700 1 --2- 172.21.15.97:0/2329322539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3868144660 0x7f38680b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386d60c700 1 -- 172.21.15.97:0/2329322539 >> v1:172.21.15.97:6789/0 conn(0x7f386815ea60 legacy=0x7f386806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386d60c700 1 -- 172.21.15.97:0/2329322539 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386815f800 con 0x7f386806c710 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f38677fe700 1 -- 172.21.15.97:0/2329322539 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991488883 0 0) 0x7f386815f520 con 0x7f386815ea60 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386ce0b700 1 --2- 172.21.15.97:0/2329322539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3868144660 0x7f38680b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f386d60c700 1 --2- 172.21.15.97:0/2329322539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386806c710 0x7f386815eee0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3860009d90 tx=0x7f386000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b4281f0c9ed038 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f38677fe700 1 -- 172.21.15.97:0/2329322539 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386000e070 con 0x7f386806c710 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f38677fe700 1 -- 172.21.15.97:0/2329322539 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386000bbd0 con 0x7f386806c710 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.361+0000 7f38677fe700 1 -- 172.21.15.97:0/2329322539 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860012430 con 0x7f386806c710 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.362+0000 7f386f06f700 1 -- 172.21.15.97:0/2329322539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386806c710 msgr2=0x7f386815eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.362+0000 7f386f06f700 1 --2- 172.21.15.97:0/2329322539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386806c710 0x7f386815eee0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3860009d90 tx=0x7f386000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.362+0000 7f386f06f700 1 -- 172.21.15.97:0/2329322539 shutdown_connections 2023-11-24T06:11:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.362+0000 7f386f06f700 1 --2- 172.21.15.97:0/2329322539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f386806c710 0x7f386815eee0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.362+0000 7f386f06f700 1 --2- 172.21.15.97:0/2329322539 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3868144660 0x7f38680b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.362+0000 7f386f06f700 1 -- 172.21.15.97:0/2329322539 >> 172.21.15.97:0/2329322539 conn(0x7f3868071ac0 msgr2=0x7f3868071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.362+0000 7f386f06f700 1 -- 172.21.15.97:0/2329322539 shutdown_connections 2023-11-24T06:11:39.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.362+0000 7f386f06f700 1 -- 172.21.15.97:0/2329322539 wait complete. 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.363+0000 7f386f06f700 1 Processor -- start 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.363+0000 7f386f06f700 1 -- start start 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f386f06f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f386806c710 0x7f3868140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f386f06f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3868144660 0x7f3868159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f386f06f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f386815ea60 0x7f3868139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f386f06f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3868140080 con 0x7f3868144660 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f386f06f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386813f400 con 0x7f386806c710 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f386f06f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386813f6e0 con 0x7f386815ea60 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f3867fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3868144660 0x7f3868159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f3867fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3868144660 0x7f3868159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50104/0 (socket says 172.21.15.97:50104) 2023-11-24T06:11:39.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f3867fff700 1 -- 172.21.15.97:0/570348705 learned_addr learned my addr 172.21.15.97:0/570348705 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f386d60c700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f386815ea60 0x7f3868139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f386ce0b700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f386806c710 0x7f3868140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f3867fff700 1 -- 172.21.15.97:0/570348705 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f386815ea60 msgr2=0x7f3868139eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f3867fff700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f386815ea60 0x7f3868139eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f3867fff700 1 -- 172.21.15.97:0/570348705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f386806c710 msgr2=0x7f3868140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f3867fff700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f386806c710 0x7f3868140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.364+0000 7f3867fff700 1 -- 172.21.15.97:0/570348705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860009a70 con 0x7f3868144660 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.365+0000 7f3867fff700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3868144660 0x7f3868159d80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f385400c550 tx=0x7f385400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.365+0000 7f38657fa700 1 -- 172.21.15.97:0/570348705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854004c80 con 0x7f3868144660 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.365+0000 7f38657fa700 1 -- 172.21.15.97:0/570348705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854004e00 con 0x7f3868144660 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.365+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386813f9c0 con 0x7f3868144660 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.365+0000 7f38657fa700 1 -- 172.21.15.97:0/570348705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854007310 con 0x7f3868144660 2023-11-24T06:11:39.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.365+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386815a2c0 con 0x7f3868144660 2023-11-24T06:11:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.366+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38680b5690 con 0x7f3868144660 2023-11-24T06:11:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.366+0000 7f38657fa700 1 -- 172.21.15.97:0/570348705 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3854007470 con 0x7f3868144660 2023-11-24T06:11:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.366+0000 7f38657fa700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f384c06b2c0 0x7f384c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.367+0000 7f38657fa700 1 -- 172.21.15.97:0/570348705 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f38540899a0 con 0x7f3868144660 2023-11-24T06:11:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.367+0000 7f386ce0b700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f384c06b2c0 0x7f384c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.367+0000 7f386ce0b700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f384c06b2c0 0x7f384c06d780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f385c009930 tx=0x7f385c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.374 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.371+0000 7f38657fa700 1 -- 172.21.15.97:0/570348705 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3854054560 con 0x7f3868144660 2023-11-24T06:11:39.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.605+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3868067f60 con 0x7f3868144660 2023-11-24T06:11:39.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.606+0000 7f38657fa700 1 -- 172.21.15.97:0/570348705 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f3854057b80 con 0x7f3868144660 2023-11-24T06:11:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:39.609 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5863,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2765643379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2765643379},{"type":"v1","addr":"172.21.15.183:6834","nonce":2765643379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1367625334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367625334},{"type":"v1","addr":"172.21.15.97:6836","nonce":1367625334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/807372372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":807372372},{"type":"v1","addr":"172.21.15.97:6837","nonce":807372372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:35.581019+0000","modified":"2023-11-24T06:11:38.607438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3452869588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3452869588},{"type":"v1","addr":"172.21.15.183:6835","nonce":3452869588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f384c06b2c0 msgr2=0x7f384c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f384c06b2c0 0x7f384c06d780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f385c009930 tx=0x7f385c008040 comp rx=0 tx=0).stop 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3868144660 msgr2=0x7f3868159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3868144660 0x7f3868159d80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f385400c550 tx=0x7f385400c910 comp rx=0 tx=0).stop 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 shutdown_connections 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f384c06b2c0 0x7f384c06d780 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f386815ea60 0x7f3868139eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f386806c710 0x7f3868140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 --2- 172.21.15.97:0/570348705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3868144660 0x7f3868159d80 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 >> 172.21.15.97:0/570348705 conn(0x7f3868071ac0 msgr2=0x7f38680ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:39.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 shutdown_connections 2023-11-24T06:11:39.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.608+0000 7f386f06f700 1 -- 172.21.15.97:0/570348705 wait complete. 2023-11-24T06:11:39.612 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 46 2023-11-24T06:11:39.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:39.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.740+0000 7f55397e3700 1 Processor -- start 2023-11-24T06:11:39.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.740+0000 7f55397e3700 1 -- start start 2023-11-24T06:11:39.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55397e3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340b2b60 0x7f55340ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55397e3700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55340cf530 con 0x7f55340cf0d0 2023-11-24T06:11:39.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55397e3700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553406bc00 con 0x7f553415e0f0 2023-11-24T06:11:39.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55397e3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553415e4d0 con 0x7f55340b2b60 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55337fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f55340cf0d0 0x7f55340d1f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42566/0 (socket says 172.21.15.97:42566) 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55337fe700 1 -- 172.21.15.97:0/1877712520 learned_addr learned my addr 172.21.15.97:0/1877712520 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55327fc700 1 --2- 172.21.15.97:0/1877712520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340b2b60 0x7f55340ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f5531ffb700 1 -- 172.21.15.97:0/1877712520 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732695119 0 0) 0x7f55340cf530 con 0x7f55340cf0d0 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f5531ffb700 1 -- 172.21.15.97:0/1877712520 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f551c0035b0 con 0x7f55340cf0d0 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f5531ffb700 1 -- 172.21.15.97:0/1877712520 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618909247 0 0) 0x7f553406bc00 con 0x7f553415e0f0 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f5531ffb700 1 -- 172.21.15.97:0/1877712520 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55340cf530 con 0x7f553415e0f0 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55327fc700 1 -- 172.21.15.97:0/1877712520 >> v1:172.21.15.183:6789/0 conn(0x7f553415e0f0 legacy=0x7f55340b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55327fc700 1 -- 172.21.15.97:0/1877712520 >> v1:172.21.15.97:6789/0 conn(0x7f55340cf0d0 legacy=0x7f55340d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.741+0000 7f55327fc700 1 -- 172.21.15.97:0/1877712520 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553406aff0 con 0x7f55340b2b60 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f55327fc700 1 --2- 172.21.15.97:0/1877712520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340b2b60 0x7f55340ceb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5528009870 tx=0x7f5528009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d8ef032d03d405e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f5531ffb700 1 -- 172.21.15.97:0/1877712520 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528011830 con 0x7f55340b2b60 2023-11-24T06:11:39.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f5531ffb700 1 -- 172.21.15.97:0/1877712520 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5528011990 con 0x7f55340b2b60 2023-11-24T06:11:39.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f5531ffb700 1 -- 172.21.15.97:0/1877712520 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5528011d20 con 0x7f55340b2b60 2023-11-24T06:11:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f55397e3700 1 -- 172.21.15.97:0/1877712520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340b2b60 msgr2=0x7f55340ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f55397e3700 1 --2- 172.21.15.97:0/1877712520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340b2b60 0x7f55340ceb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5528009870 tx=0x7f5528009b80 comp rx=0 tx=0).stop 2023-11-24T06:11:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f55397e3700 1 -- 172.21.15.97:0/1877712520 shutdown_connections 2023-11-24T06:11:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f55397e3700 1 --2- 172.21.15.97:0/1877712520 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340b2b60 0x7f55340ceb90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f55397e3700 1 -- 172.21.15.97:0/1877712520 >> 172.21.15.97:0/1877712520 conn(0x7f55340b5ce0 msgr2=0x7f55340b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f55397e3700 1 -- 172.21.15.97:0/1877712520 shutdown_connections 2023-11-24T06:11:39.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.742+0000 7f55397e3700 1 -- 172.21.15.97:0/1877712520 wait complete. 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55397e3700 1 Processor -- start 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55397e3700 1 -- start start 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55397e3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55340b2b60 0x7f55340b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55397e3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340cf0d0 0x7f55341f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55397e3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f553415e0f0 0x7f5534151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55397e3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553415fae0 con 0x7f553415e0f0 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55397e3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5534151660 con 0x7f55340b2b60 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55397e3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55340c8140 con 0x7f55340cf0d0 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f553415e0f0 0x7f5534151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f553415e0f0 0x7f5534151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50120/0 (socket says 172.21.15.97:50120) 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.744+0000 7f55337fe700 1 -- 172.21.15.97:0/2427979033 learned_addr learned my addr 172.21.15.97:0/2427979033 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55327fc700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340cf0d0 0x7f55341f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f5532ffd700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55340b2b60 0x7f55340b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55337fe700 1 -- 172.21.15.97:0/2427979033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340cf0d0 msgr2=0x7f55341f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55337fe700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340cf0d0 0x7f55341f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55337fe700 1 -- 172.21.15.97:0/2427979033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55340b2b60 msgr2=0x7f55340b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55337fe700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55340b2b60 0x7f55340b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55337fe700 1 -- 172.21.15.97:0/2427979033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280094a0 con 0x7f553415e0f0 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55337fe700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f553415e0f0 0x7f5534151120 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f552c00cba0 tx=0x7f552c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f551b7fe700 1 -- 172.21.15.97:0/2427979033 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c00ab20 con 0x7f553415e0f0 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f551b7fe700 1 -- 172.21.15.97:0/2427979033 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552c00ac80 con 0x7f553415e0f0 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55340c8420 con 0x7f553415e0f0 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f551b7fe700 1 -- 172.21.15.97:0/2427979033 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c012400 con 0x7f553415e0f0 2023-11-24T06:11:39.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.745+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55340c8700 con 0x7f553415e0f0 2023-11-24T06:11:39.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.746+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55340751e0 con 0x7f553415e0f0 2023-11-24T06:11:39.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.747+0000 7f551b7fe700 1 -- 172.21.15.97:0/2427979033 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f552c01b3b0 con 0x7f553415e0f0 2023-11-24T06:11:39.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.747+0000 7f551b7fe700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f552006b2c0 0x7f552006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.747+0000 7f551b7fe700 1 -- 172.21.15.97:0/2427979033 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f552c089a70 con 0x7f553415e0f0 2023-11-24T06:11:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.747+0000 7f5532ffd700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f552006b2c0 0x7f552006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:39.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.747+0000 7f5532ffd700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f552006b2c0 0x7f552006d780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5524007fd0 tx=0x7f5524009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:39.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.752+0000 7f551b7fe700 1 -- 172.21.15.97:0/2427979033 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f552c0545d0 con 0x7f553415e0f0 2023-11-24T06:11:39.945 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.943+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55340683f0 con 0x7f553415e0f0 2023-11-24T06:11:39.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.944+0000 7f551b7fe700 1 -- 172.21.15.97:0/2427979033 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16623 (secure 0 0 0) 0x7f552c017090 con 0x7f553415e0f0 2023-11-24T06:11:39.947 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:39.947 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":72,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:37.197270+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:33.973262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:34.541598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f552006b2c0 msgr2=0x7f552006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f552006b2c0 0x7f552006d780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5524007fd0 tx=0x7f5524009040 comp rx=0 tx=0).stop 2023-11-24T06:11:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f553415e0f0 msgr2=0x7f5534151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f553415e0f0 0x7f5534151120 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f552c00cba0 tx=0x7f552c00cf60 comp rx=0 tx=0).stop 2023-11-24T06:11:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 shutdown_connections 2023-11-24T06:11:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f553415e0f0 0x7f5534151120 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55340b2b60 0x7f55340b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f552006b2c0 0x7f552006d780 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 --2- 172.21.15.97:0/2427979033 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f55340cf0d0 0x7f55341f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 >> 172.21.15.97:0/2427979033 conn(0x7f55340b5ce0 msgr2=0x7f55340b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 shutdown_connections 2023-11-24T06:11:39.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:39.946+0000 7f55397e3700 1 -- 172.21.15.97:0/2427979033 wait complete. 2023-11-24T06:11:39.961 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:11:39.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:11:40.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.078+0000 7fcd36cbd700 1 Processor -- start 2023-11-24T06:11:40.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.078+0000 7fcd36cbd700 1 -- start start 2023-11-24T06:11:40.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd36cbd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd300adf00 0x7fcd3006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd36cbd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcd3006c540 0x7fcd300b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd36cbd700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300af950 con 0x7fcd301446a0 2023-11-24T06:11:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd36cbd700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd30144a80 con 0x7fcd3006c540 2023-11-24T06:11:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd36cbd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3006c9b0 con 0x7fcd300adf00 2023-11-24T06:11:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd3525a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcd3006c540 0x7fcd300b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd2ffff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd300adf00 0x7fcd3006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd34a59700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fcd301446a0 0x7fcd300b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51898/0 (socket says 172.21.15.97:51898) 2023-11-24T06:11:40.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd34a59700 1 -- 172.21.15.97:0/4229092045 learned_addr learned my addr 172.21.15.97:0/4229092045 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:40.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd2f7fe700 1 -- 172.21.15.97:0/4229092045 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726659919 0 0) 0x7fcd300af950 con 0x7fcd301446a0 2023-11-24T06:11:40.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd2f7fe700 1 -- 172.21.15.97:0/4229092045 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd200035b0 con 0x7fcd301446a0 2023-11-24T06:11:40.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd3525a700 1 -- 172.21.15.97:0/4229092045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd300adf00 msgr2=0x7fcd3006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd3525a700 1 --2- 172.21.15.97:0/4229092045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd300adf00 0x7fcd3006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd3525a700 1 -- 172.21.15.97:0/4229092045 >> v1:172.21.15.183:6789/0 conn(0x7fcd301446a0 legacy=0x7fcd300b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd3525a700 1 -- 172.21.15.97:0/4229092045 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300afc30 con 0x7fcd3006c540 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.080+0000 7fcd2ffff700 1 --2- 172.21.15.97:0/4229092045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd300adf00 0x7fcd3006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd3525a700 1 --2- 172.21.15.97:0/4229092045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd3006c540 0x7fcd300b3d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009dc0 tx=0x7fcd2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9275ae200e211706 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd2f7fe700 1 -- 172.21.15.97:0/4229092045 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800e070 con 0x7fcd3006c540 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd2f7fe700 1 -- 172.21.15.97:0/4229092045 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2800bae0 con 0x7fcd3006c540 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd2f7fe700 1 -- 172.21.15.97:0/4229092045 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2800be90 con 0x7fcd3006c540 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd36cbd700 1 -- 172.21.15.97:0/4229092045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd3006c540 msgr2=0x7fcd300b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/4229092045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd3006c540 0x7fcd300b3d40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009dc0 tx=0x7fcd2800b330 comp rx=0 tx=0).stop 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd36cbd700 1 -- 172.21.15.97:0/4229092045 shutdown_connections 2023-11-24T06:11:40.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/4229092045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd3006c540 0x7fcd300b3d40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/4229092045 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd300adf00 0x7fcd3006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd36cbd700 1 -- 172.21.15.97:0/4229092045 >> 172.21.15.97:0/4229092045 conn(0x7fcd30071b90 msgr2=0x7fcd30071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd36cbd700 1 -- 172.21.15.97:0/4229092045 shutdown_connections 2023-11-24T06:11:40.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.081+0000 7fcd36cbd700 1 -- 172.21.15.97:0/4229092045 wait complete. 2023-11-24T06:11:40.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.083+0000 7fcd36cbd700 1 Processor -- start 2023-11-24T06:11:40.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.083+0000 7fcd36cbd700 1 -- start start 2023-11-24T06:11:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd36cbd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd3006c540 0x7fcd3013a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd36cbd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd300adf00 0x7fcd300ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd36cbd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd301446a0 0x7fcd30159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd36cbd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300ce320 con 0x7fcd301446a0 2023-11-24T06:11:40.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd36cbd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300cec80 con 0x7fcd300adf00 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd36cbd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3015aa50 con 0x7fcd3006c540 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd3525a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd301446a0 0x7fcd30159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd3525a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd301446a0 0x7fcd30159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50134/0 (socket says 172.21.15.97:50134) 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd3525a700 1 -- 172.21.15.97:0/956408231 learned_addr learned my addr 172.21.15.97:0/956408231 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd34a59700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd3006c540 0x7fcd3013a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd2ffff700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd300adf00 0x7fcd300ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd3525a700 1 -- 172.21.15.97:0/956408231 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd3006c540 msgr2=0x7fcd3013a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd3525a700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd3006c540 0x7fcd3013a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd3525a700 1 -- 172.21.15.97:0/956408231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd300adf00 msgr2=0x7fcd300ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.084+0000 7fcd3525a700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd300adf00 0x7fcd300ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.085+0000 7fcd3525a700 1 -- 172.21.15.97:0/956408231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0094a0 con 0x7fcd301446a0 2023-11-24T06:11:40.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.085+0000 7fcd3525a700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd301446a0 0x7fcd30159960 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800b370 tx=0x7fcd28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.085+0000 7fcd2d7fa700 1 -- 172.21.15.97:0/956408231 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28004550 con 0x7fcd301446a0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.085+0000 7fcd2d7fa700 1 -- 172.21.15.97:0/956408231 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28020070 con 0x7fcd301446a0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.085+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd28009a70 con 0x7fcd301446a0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.085+0000 7fcd2d7fa700 1 -- 172.21.15.97:0/956408231 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280048f0 con 0x7fcd301446a0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.085+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd300cd980 con 0x7fcd301446a0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.086+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd300753f0 con 0x7fcd301446a0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.086+0000 7fcd2d7fa700 1 -- 172.21.15.97:0/956408231 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcd28012430 con 0x7fcd301446a0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.087+0000 7fcd2d7fa700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd1406b2c0 0x7fcd1406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.087+0000 7fcd2d7fa700 1 -- 172.21.15.97:0/956408231 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcd28022070 con 0x7fcd301446a0 2023-11-24T06:11:40.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.087+0000 7fcd34a59700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd1406b2c0 0x7fcd1406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.087+0000 7fcd34a59700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd1406b2c0 0x7fcd1406d780 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcd24007fd0 tx=0x7fcd24007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.091+0000 7fcd2d7fa700 1 -- 172.21.15.97:0/956408231 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd2805a2a0 con 0x7fcd301446a0 2023-11-24T06:11:40.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.278+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd300b04d0 con 0x7fcd1406b2c0 2023-11-24T06:11:40.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.286+0000 7fcd2d7fa700 1 -- 172.21.15.97:0/956408231 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277406 (secure 0 0 0) 0x7fcd240058e0 con 0x7fcd1406b2c0 2023-11-24T06:11:40.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:40.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd1406b2c0 msgr2=0x7fcd1406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd1406b2c0 0x7fcd1406d780 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcd24007fd0 tx=0x7fcd24007dc0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd301446a0 msgr2=0x7fcd30159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.294 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd301446a0 0x7fcd30159960 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcd2800b370 tx=0x7fcd28004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 shutdown_connections 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd301446a0 0x7fcd30159960 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcd1406b2c0 0x7fcd1406d780 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcd3006c540 0x7fcd3013a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 --2- 172.21.15.97:0/956408231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd300adf00 0x7fcd300ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 >> 172.21.15.97:0/956408231 conn(0x7fcd30071b90 msgr2=0x7fcd300cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 shutdown_connections 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.289+0000 7fcd36cbd700 1 -- 172.21.15.97:0/956408231 wait complete. 2023-11-24T06:11:40.295 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:11:40.304 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2023-11-24T06:11:38.356142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223832,"kb_used_data":7636,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750888,"statfs":{"total":773094113280,"available":772864909312,"internally_reserved":0,"allocated":7819264,"data_stored":3967645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.179996"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182602+0000","last_change":"2023-11-24T06:11:34.182602+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182602+0000","last_undegraded":"2023-11-24T06:11:34.182602+0000","last_fullsized":"2023-11-24T06:11:34.182602+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182910+0000","last_change":"2023-11-24T06:11:34.182910+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182910+0000","last_undegraded":"2023-11-24T06:11:34.182910+0000","last_fullsized":"2023-11-24T06:11:34.182910+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.179215+0000","last_change":"2023-11-24T06:11:34.179215+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.179215+0000","last_undegraded":"2023-11-24T06:11:34.179215+0000","last_fullsized":"2023-11-24T06:11:34.179215+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.181600+0000","last_change":"2023-11-24T06:11:34.181600+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.181600+0000","last_undegraded":"2023-11-24T06:11:34.181600+0000","last_fullsized":"2023-11-24T06:11:34.181600+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.183916+0000","last_change":"2023-11-24T06:11:34.183916+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.183916+0000","last_undegraded":"2023-11-24T06:11:34.183916+0000","last_fullsized":"2023-11-24T06:11:34.183916+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182784+0000","last_change":"2023-11-24T06:11:34.182784+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182784+0000","last_undegraded":"2023-11-24T06:11:34.182784+0000","last_fullsized":"2023-11-24T06:11:34.182784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182781+0000","last_change":"2023-11-24T06:11:34.182781+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182781+0000","last_undegraded":"2023-11-24T06:11:34.182781+0000","last_fullsized":"2023-11-24T06:11:34.182781+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.178772+0000","last_change":"2023-11-24T06:11:34.178772+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.178772+0000","last_undegraded":"2023-11-24T06:11:34.178772+0000","last_fullsized":"2023-11-24T06:11:34.178772+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.179464+0000","last_change":"2023-11-24T06:11:34.179464+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.179464+0000","last_undegraded":"2023-11-24T06:11:34.179464+0000","last_fullsized":"2023-11-24T06:11:34.179464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.181783+0000","last_change":"2023-11-24T06:11:34.181783+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.181783+0000","last_undegraded":"2023-11-24T06:11:34.181783+0000","last_fullsized":"2023-11-24T06:11:34.181783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.183927+0000","last_change":"2023-11-24T06:11:34.183927+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.183927+0000","last_undegraded":"2023-11-24T06:11:34.183927+0000","last_fullsized":"2023-11-24T06:11:34.183927+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.183397+0000","last_change":"2023-11-24T06:11:34.183397+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.183397+0000","last_undegraded":"2023-11-24T06:11:34.183397+0000","last_fullsized":"2023-11-24T06:11:34.183397+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182987+0000","last_change":"2023-11-24T06:11:34.182987+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182987+0000","last_undegraded":"2023-11-24T06:11:34.182987+0000","last_fullsized":"2023-11-24T06:11:34.182987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182349+0000","last_change":"2023-11-24T06:11:34.182349+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182349+0000","last_undegraded":"2023-11-24T06:11:34.182349+0000","last_fullsized":"2023-11-24T06:11:34.182349+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.180459+0000","last_change":"2023-11-24T06:11:34.180459+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.180459+0000","last_undegraded":"2023-11-24T06:11:34.180459+0000","last_fullsized":"2023-11-24T06:11:34.180459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.179196+0000","last_change":"2023-11-24T06:11:34.179196+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.179196+0000","last_undegraded":"2023-11-24T06:11:34.179196+0000","last_fullsized":"2023-11-24T06:11:34.179196+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182283+0000","last_change":"2023-11-24T06:11:34.182283+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182283+0000","last_undegraded":"2023-11-24T06:11:34.182283+0000","last_fullsized":"2023-11-24T06:11:34.182283+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.179432+0000","last_change":"2023-11-24T06:11:34.179432+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.179432+0000","last_undegraded":"2023-11-24T06:11:34.179432+0000","last_fullsized":"2023-11-24T06:11:34.179432+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.181608+0000","last_change":"2023-11-24T06:11:34.181608+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.181608+0000","last_undegraded":"2023-11-24T06:11:34.181608+0000","last_fullsized":"2023-11-24T06:11:34.181608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.183353+0000","last_change":"2023-11-24T06:11:34.183353+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.183353+0000","last_undegraded":"2023-11-24T06:11:34.183353+0000","last_fullsized":"2023-11-24T06:11:34.183353+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.181536+0000","last_change":"2023-11-24T06:11:34.181536+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.181536+0000","last_undegraded":"2023-11-24T06:11:34.181536+0000","last_fullsized":"2023-11-24T06:11:34.181536+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.179067+0000","last_change":"2023-11-24T06:11:34.179067+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.179067+0000","last_undegraded":"2023-11-24T06:11:34.179067+0000","last_fullsized":"2023-11-24T06:11:34.179067+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182856+0000","last_change":"2023-11-24T06:11:34.182856+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182856+0000","last_undegraded":"2023-11-24T06:11:34.182856+0000","last_fullsized":"2023-11-24T06:11:34.182856+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.181649+0000","last_change":"2023-11-24T06:11:34.181649+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.181649+0000","last_undegraded":"2023-11-24T06:11:34.181649+0000","last_fullsized":"2023-11-24T06:11:34.181649+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.179140+0000","last_change":"2023-11-24T06:11:34.179140+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.179140+0000","last_undegraded":"2023-11-24T06:11:34.179140+0000","last_fullsized":"2023-11-24T06:11:34.179140+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.178811+0000","last_change":"2023-11-24T06:11:34.178811+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.178811+0000","last_undegraded":"2023-11-24T06:11:34.178811+0000","last_fullsized":"2023-11-24T06:11:34.178811+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.183393+0000","last_change":"2023-11-24T06:11:34.183393+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.183393+0000","last_undegraded":"2023-11-24T06:11:34.183393+0000","last_fullsized":"2023-11-24T06:11:34.183393+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.178807+0000","last_change":"2023-11-24T06:11:34.178807+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.178807+0000","last_undegraded":"2023-11-24T06:11:34.178807+0000","last_fullsized":"2023-11-24T06:11:34.178807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.182777+0000","last_change":"2023-11-24T06:11:34.182777+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.182777+0000","last_undegraded":"2023-11-24T06:11:34.182777+0000","last_fullsized":"2023-11-24T06:11:34.182777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.179899+0000","last_change":"2023-11-24T06:11:34.179899+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.179899+0000","last_undegraded":"2023-11-24T06:11:34.179899+0000","last_fullsized":"2023-11-24T06:11:34.179899+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.181539+0000","last_change":"2023-11-24T06:11:34.181539+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.181539+0000","last_undegraded":"2023-11-24T06:11:34.181539+0000","last_fullsized":"2023-11-24T06:11:34.181539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T06:11:35.176246+0000","last_change":"2023-11-24T06:11:35.176246+0000","last_active":"2023-11-24T06:11:35.176246+0000","last_peered":"2023-11-24T06:11:35.176246+0000","last_clean":"2023-11-24T06:11:35.176246+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:35.176246+0000","last_undegraded":"2023-11-24T06:11:35.176246+0000","last_fullsized":"2023-11-24T06:11:35.176246+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2023-11-24T06:11:34.181668+0000","last_change":"2023-11-24T06:11:34.181668+0000","last_active":"2023-11-24T06:11:34.174123+0000","last_peered":"2023-11-24T06:11:34.174123+0000","last_clean":"2023-11-24T06:11:34.174123+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T06:11:34.181668+0000","last_undegraded":"2023-11-24T06:11:34.181668+0000","last_fullsized":"2023-11-24T06:11:34.181668+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":205,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-24T06:11:32.178425+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:11:32.178425+0000","last_peered":"2023-11-24T06:11:32.178425+0000","last_clean":"2023-11-24T06:11:32.178425+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:11:32.178425+0000","last_undegraded":"2023-11-24T06:11:32.178425+0000","last_fullsized":"2023-11-24T06:11:32.178425+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28408,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343432,"statfs":{"total":96636764160,"available":96607674368,"internally_reserved":0,"allocated":1421312,"data_stored":938001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.87,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623}]},{"osd":1,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68000000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.373,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":347633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.76800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34200000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.83099999999999996,"15min":0.88900000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.621,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":1.371,"15min":1.371},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66100000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.83099999999999996,"15min":0.875},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70299999999999996,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":5.2830000000000004,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70199999999999996,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71599999999999997,"5min":1.407,"15min":1.407},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78400000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.51200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":347633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.33600000000000002,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45100000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.25900000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.48599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.64600000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47399999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.375,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.52600000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.497,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71299999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.46800000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.78100000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81100000000000005,"5min":0.88,"15min":0.88},"last":0.877},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.75700000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.75900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28420,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343420,"statfs":{"total":96636764160,"available":96607662080,"internally_reserved":0,"allocated":1433600,"data_stored":943846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.72599999999999998,"15min":0.79200000000000004},"min":{"1min":0.224,"5min":0.216,"15min":0.216},"max":{"1min":0.46100000000000002,"5min":12.378,"15min":12.378},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58799999999999997,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.496}]},{"osd":2,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.501,"15min":0.52000000000000002},"min":{"1min":0.26200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.54300000000000004,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59899999999999998,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.0920000000000001,"15min":1.169},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97499999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83099999999999996,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93300000000000005,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.621,"5min":1.091,"15min":1.169},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0289999999999999,"5min":15.356,"15min":15.356},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.72899999999999998,"15min":0.75},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83999999999999997,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.752,"15min":0.77400000000000002},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":4.726,"15min":4.726},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.755,"15min":0.77600000000000002},"min":{"1min":0.45100000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":1.129,"15min":1.2090000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":15.388,"15min":15.388},"last":0.72599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":347633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33500000000000002,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.49299999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.30599999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.25900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.38},{"interface":"front","average":{"1min":0.373,"5min":0.38900000000000001,"15min":0.39200000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56299999999999994,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.28699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72899999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.42299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.438}]},{"osd":4,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0580000000000001,"5min":1.173,"15min":1.173},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99099999999999999,"5min":1.339,"15min":1.339},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":0.91100000000000003,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92500000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.018,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.079,"5min":1.405,"15min":1.405},"last":0.621},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95399999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.70899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":347633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.33600000000000002,"15min":0.34100000000000003},"min":{"1min":0.23799999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.42299999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.39400000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.28599999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.432,"5min":1.032,"15min":1.032},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.47899999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.372},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.52100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.35499999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90100000000000002,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.878,"5min":1.728,"15min":1.728},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88500000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.625}]},{"osd":6,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.161,"5min":1.222,"15min":1.222},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.499,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.64100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":347633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63300000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.39200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.495}]},{"osd":2,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38500000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.51000000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":827392,"data_stored":347633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89800000000000002,"5min":3.335,"15min":3.335},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81299999999999994,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77600000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72299999999999998,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76200000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59599999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71899999999999997,"15min":0.73999999999999999},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.80700000000000005,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79300000000000004,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76200000000000001,"5min":1.536,"15min":1.536},"last":0.629},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.59999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82899999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77300000000000002,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80900000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.64200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:11:40.309 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:11:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.461+0000 7feac6d5a700 1 Processor -- start 2023-11-24T06:11:40.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.461+0000 7feac6d5a700 1 -- start start 2023-11-24T06:11:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.462+0000 7feac6d5a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7feac015bdd0 0x7feac01604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.462+0000 7feac6d5a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.462+0000 7feac6d5a700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac00b2030 con 0x7feac00d1c70 2023-11-24T06:11:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.462+0000 7feac6d5a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac00b2f50 con 0x7feac015bdd0 2023-11-24T06:11:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.462+0000 7feac6d5a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac00b22d0 con 0x7feac00b3370 2023-11-24T06:11:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabffff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feac4af6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7feac015bdd0 0x7feac01604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feac52f7700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7feac00d1c70 0x7feac00ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51916/0 (socket says 172.21.15.97:51916) 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabffff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00b1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:37176/0 (socket says 172.21.15.97:37176) 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feac52f7700 1 -- 172.21.15.97:0/556313177 learned_addr learned my addr 172.21.15.97:0/556313177 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabffff700 1 -- 172.21.15.97:0/556313177 >> v2:172.21.15.183:3300/0 conn(0x7feac015bdd0 msgr2=0x7feac01604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabffff700 1 --2- 172.21.15.97:0/556313177 >> v2:172.21.15.183:3300/0 conn(0x7feac015bdd0 0x7feac01604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabffff700 1 -- 172.21.15.97:0/556313177 >> v1:172.21.15.183:6789/0 conn(0x7feac00d1c70 legacy=0x7feac00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabf7fe700 1 -- 172.21.15.97:0/556313177 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355155446 0 0) 0x7feac00b2030 con 0x7feac00d1c70 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabffff700 1 -- 172.21.15.97:0/556313177 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00b25b0 con 0x7feac00b3370 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feac4af6700 1 --2- 172.21.15.97:0/556313177 >> v2:172.21.15.183:3300/0 conn(0x7feac015bdd0 0x7feac01604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabffff700 1 --2- 172.21.15.97:0/556313177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00b1af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feaac009840 tx=0x7feaac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c55b8f92e84595 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabf7fe700 1 -- 172.21.15.97:0/556313177 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac011830 con 0x7feac00b3370 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabf7fe700 1 -- 172.21.15.97:0/556313177 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac011990 con 0x7feac00b3370 2023-11-24T06:11:40.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.463+0000 7feabf7fe700 1 -- 172.21.15.97:0/556313177 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac011d20 con 0x7feac00b3370 2023-11-24T06:11:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.464+0000 7feac6d5a700 1 -- 172.21.15.97:0/556313177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 msgr2=0x7feac00b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.464+0000 7feac6d5a700 1 --2- 172.21.15.97:0/556313177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00b1af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feaac009840 tx=0x7feaac009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.464+0000 7feac6d5a700 1 -- 172.21.15.97:0/556313177 shutdown_connections 2023-11-24T06:11:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.464+0000 7feac6d5a700 1 --2- 172.21.15.97:0/556313177 >> v2:172.21.15.183:3300/0 conn(0x7feac015bdd0 0x7feac01604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.464+0000 7feac6d5a700 1 --2- 172.21.15.97:0/556313177 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00b1af0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.464+0000 7feac6d5a700 1 -- 172.21.15.97:0/556313177 >> 172.21.15.97:0/556313177 conn(0x7feac00731a0 msgr2=0x7feac009aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.464+0000 7feac6d5a700 1 -- 172.21.15.97:0/556313177 shutdown_connections 2023-11-24T06:11:40.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.464+0000 7feac6d5a700 1 -- 172.21.15.97:0/556313177 wait complete. 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.465+0000 7feac6d5a700 1 Processor -- start 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.465+0000 7feac6d5a700 1 -- start start 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac6d5a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac6d5a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac00d1c70 0x7feac00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac6d5a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac015bdd0 0x7feac00c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac6d5a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00ceba0 con 0x7feac015bdd0 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac6d5a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac006bda0 con 0x7feac00d1c70 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac6d5a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac006b190 con 0x7feac00b3370 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac52f7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac015bdd0 0x7feac00c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac52f7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac015bdd0 0x7feac00c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50148/0 (socket says 172.21.15.97:50148) 2023-11-24T06:11:40.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.466+0000 7feac52f7700 1 -- 172.21.15.97:0/3324260997 learned_addr learned my addr 172.21.15.97:0/3324260997 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feabffff700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac00d1c70 0x7feac00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac4af6700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac52f7700 1 -- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 msgr2=0x7feac00cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac52f7700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac52f7700 1 -- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac00d1c70 msgr2=0x7feac00ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac52f7700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac00d1c70 0x7feac00ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac52f7700 1 -- 172.21.15.97:0/3324260997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40094a0 con 0x7feac015bdd0 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac4af6700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feabffff700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac00d1c70 0x7feac00ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac52f7700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac015bdd0 0x7feac00c74d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7feab8002230 tx=0x7feab800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feabd7fa700 1 -- 172.21.15.97:0/3324260997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab800ac20 con 0x7feac015bdd0 2023-11-24T06:11:40.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feabd7fa700 1 -- 172.21.15.97:0/3324260997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab800ada0 con 0x7feac015bdd0 2023-11-24T06:11:40.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac0094a0 con 0x7feac015bdd0 2023-11-24T06:11:40.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feabd7fa700 1 -- 172.21.15.97:0/3324260997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8007310 con 0x7feac015bdd0 2023-11-24T06:11:40.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.467+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac006b750 con 0x7feac015bdd0 2023-11-24T06:11:40.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.468+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac015d330 con 0x7feac015bdd0 2023-11-24T06:11:40.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.468+0000 7feabd7fa700 1 -- 172.21.15.97:0/3324260997 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7feab8007470 con 0x7feac015bdd0 2023-11-24T06:11:40.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.469+0000 7feabd7fa700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feaa406b2c0 0x7feaa406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.469+0000 7feabd7fa700 1 -- 172.21.15.97:0/3324260997 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7feab80898e0 con 0x7feac015bdd0 2023-11-24T06:11:40.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.469+0000 7feac4af6700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feaa406b2c0 0x7feaa406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.469+0000 7feac4af6700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feaa406b2c0 0x7feaa406d780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feab4009c50 tx=0x7feab4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.473+0000 7feabd7fa700 1 -- 172.21.15.97:0/3324260997 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feab8054360 con 0x7feac015bdd0 2023-11-24T06:11:40.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.657+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feac00b2b70 con 0x7feaa406b2c0 2023-11-24T06:11:40.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.665+0000 7feabd7fa700 1 -- 172.21.15.97:0/3324260997 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289289 (secure 0 0 0) 0x7feab4009d90 con 0x7feaa406b2c0 2023-11-24T06:11:40.671 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feaa406b2c0 msgr2=0x7feaa406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feaa406b2c0 0x7feaa406d780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feab4009c50 tx=0x7feab4008040 comp rx=0 tx=0).stop 2023-11-24T06:11:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac015bdd0 msgr2=0x7feac00c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac015bdd0 0x7feac00c74d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7feab8002230 tx=0x7feab800a060 comp rx=0 tx=0).stop 2023-11-24T06:11:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 shutdown_connections 2023-11-24T06:11:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac015bdd0 0x7feac00c74d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7feaa406b2c0 0x7feaa406d780 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7feac00b3370 0x7feac00cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 --2- 172.21.15.97:0/3324260997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feac00d1c70 0x7feac00ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 >> 172.21.15.97:0/3324260997 conn(0x7feac00731a0 msgr2=0x7feac0159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 shutdown_connections 2023-11-24T06:11:40.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.668+0000 7feac6d5a700 1 -- 172.21.15.97:0/3324260997 wait complete. 2023-11-24T06:11:40.674 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:11:40.683 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2023-11-24T06:11:40.356405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224236,"kb_used_data":8072,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750484,"statfs":{"total":773094113280,"available":772864495616,"internally_reserved":0,"allocated":8265728,"data_stored":4168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000858"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443646+0000","last_change":"2023-11-24T06:11:36.214188+0000","last_active":"2023-11-24T06:11:37.443646+0000","last_peered":"2023-11-24T06:11:37.443646+0000","last_clean":"2023-11-24T06:11:37.443646+0000","last_became_active":"2023-11-24T06:11:36.213883+0000","last_became_peered":"2023-11-24T06:11:36.213883+0000","last_unstale":"2023-11-24T06:11:37.443646+0000","last_undegraded":"2023-11-24T06:11:37.443646+0000","last_fullsized":"2023-11-24T06:11:37.443646+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:45.497615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435515+0000","last_change":"2023-11-24T06:11:36.210474+0000","last_active":"2023-11-24T06:11:37.435515+0000","last_peered":"2023-11-24T06:11:37.435515+0000","last_clean":"2023-11-24T06:11:37.435515+0000","last_became_active":"2023-11-24T06:11:36.210202+0000","last_became_peered":"2023-11-24T06:11:36.210202+0000","last_unstale":"2023-11-24T06:11:37.435515+0000","last_undegraded":"2023-11-24T06:11:37.435515+0000","last_fullsized":"2023-11-24T06:11:37.435515+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:44.117998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554247+0000","last_change":"2023-11-24T06:11:36.206934+0000","last_active":"2023-11-24T06:11:37.554247+0000","last_peered":"2023-11-24T06:11:37.554247+0000","last_clean":"2023-11-24T06:11:37.554247+0000","last_became_active":"2023-11-24T06:11:36.206644+0000","last_became_peered":"2023-11-24T06:11:36.206644+0000","last_unstale":"2023-11-24T06:11:37.554247+0000","last_undegraded":"2023-11-24T06:11:37.554247+0000","last_fullsized":"2023-11-24T06:11:37.554247+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:48:41.098737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.203846+0000","last_change":"2023-11-24T06:11:36.204740+0000","last_active":"2023-11-24T06:11:37.203846+0000","last_peered":"2023-11-24T06:11:37.203846+0000","last_clean":"2023-11-24T06:11:37.203846+0000","last_became_active":"2023-11-24T06:11:36.204462+0000","last_became_peered":"2023-11-24T06:11:36.204462+0000","last_unstale":"2023-11-24T06:11:37.203846+0000","last_undegraded":"2023-11-24T06:11:37.203846+0000","last_fullsized":"2023-11-24T06:11:37.203846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:38:12.828027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554160+0000","last_change":"2023-11-24T06:11:36.202974+0000","last_active":"2023-11-24T06:11:37.554160+0000","last_peered":"2023-11-24T06:11:37.554160+0000","last_clean":"2023-11-24T06:11:37.554160+0000","last_became_active":"2023-11-24T06:11:36.202678+0000","last_became_peered":"2023-11-24T06:11:36.202678+0000","last_unstale":"2023-11-24T06:11:37.554160+0000","last_undegraded":"2023-11-24T06:11:37.554160+0000","last_fullsized":"2023-11-24T06:11:37.554160+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:37:06.301475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435474+0000","last_change":"2023-11-24T06:11:36.211402+0000","last_active":"2023-11-24T06:11:37.435474+0000","last_peered":"2023-11-24T06:11:37.435474+0000","last_clean":"2023-11-24T06:11:37.435474+0000","last_became_active":"2023-11-24T06:11:36.211167+0000","last_became_peered":"2023-11-24T06:11:36.211167+0000","last_unstale":"2023-11-24T06:11:37.435474+0000","last_undegraded":"2023-11-24T06:11:37.435474+0000","last_fullsized":"2023-11-24T06:11:37.435474+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:06.194381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210021+0000","last_change":"2023-11-24T06:11:36.200073+0000","last_active":"2023-11-24T06:11:37.210021+0000","last_peered":"2023-11-24T06:11:37.210021+0000","last_clean":"2023-11-24T06:11:37.210021+0000","last_became_active":"2023-11-24T06:11:36.199789+0000","last_became_peered":"2023-11-24T06:11:36.199789+0000","last_unstale":"2023-11-24T06:11:37.210021+0000","last_undegraded":"2023-11-24T06:11:37.210021+0000","last_fullsized":"2023-11-24T06:11:37.210021+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:33:52.178491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435068+0000","last_change":"2023-11-24T06:11:36.202484+0000","last_active":"2023-11-24T06:11:37.435068+0000","last_peered":"2023-11-24T06:11:37.435068+0000","last_clean":"2023-11-24T06:11:37.435068+0000","last_became_active":"2023-11-24T06:11:36.201681+0000","last_became_peered":"2023-11-24T06:11:36.201681+0000","last_unstale":"2023-11-24T06:11:37.435068+0000","last_undegraded":"2023-11-24T06:11:37.435068+0000","last_fullsized":"2023-11-24T06:11:37.435068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:01.435811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554286+0000","last_change":"2023-11-24T06:11:36.209195+0000","last_active":"2023-11-24T06:11:37.554286+0000","last_peered":"2023-11-24T06:11:37.554286+0000","last_clean":"2023-11-24T06:11:37.554286+0000","last_became_active":"2023-11-24T06:11:36.208880+0000","last_became_peered":"2023-11-24T06:11:36.208880+0000","last_unstale":"2023-11-24T06:11:37.554286+0000","last_undegraded":"2023-11-24T06:11:37.554286+0000","last_fullsized":"2023-11-24T06:11:37.554286+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:05.639188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210195+0000","last_change":"2023-11-24T06:11:36.204397+0000","last_active":"2023-11-24T06:11:37.210195+0000","last_peered":"2023-11-24T06:11:37.210195+0000","last_clean":"2023-11-24T06:11:37.210195+0000","last_became_active":"2023-11-24T06:11:36.198826+0000","last_became_peered":"2023-11-24T06:11:36.198826+0000","last_unstale":"2023-11-24T06:11:37.210195+0000","last_undegraded":"2023-11-24T06:11:37.210195+0000","last_fullsized":"2023-11-24T06:11:37.210195+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:06.616828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443492+0000","last_change":"2023-11-24T06:11:36.212824+0000","last_active":"2023-11-24T06:11:37.443492+0000","last_peered":"2023-11-24T06:11:37.443492+0000","last_clean":"2023-11-24T06:11:37.443492+0000","last_became_active":"2023-11-24T06:11:36.212520+0000","last_became_peered":"2023-11-24T06:11:36.212520+0000","last_unstale":"2023-11-24T06:11:37.443492+0000","last_undegraded":"2023-11-24T06:11:37.443492+0000","last_fullsized":"2023-11-24T06:11:37.443492+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:24:24.262110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.203744+0000","last_change":"2023-11-24T06:11:36.198148+0000","last_active":"2023-11-24T06:11:37.203744+0000","last_peered":"2023-11-24T06:11:37.203744+0000","last_clean":"2023-11-24T06:11:37.203744+0000","last_became_active":"2023-11-24T06:11:36.197864+0000","last_became_peered":"2023-11-24T06:11:36.197864+0000","last_unstale":"2023-11-24T06:11:37.203744+0000","last_undegraded":"2023-11-24T06:11:37.203744+0000","last_fullsized":"2023-11-24T06:11:37.203744+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:50:54.353290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.509032+0000","last_change":"2023-11-24T06:11:36.207361+0000","last_active":"2023-11-24T06:11:37.509032+0000","last_peered":"2023-11-24T06:11:37.509032+0000","last_clean":"2023-11-24T06:11:37.509032+0000","last_became_active":"2023-11-24T06:11:36.207029+0000","last_became_peered":"2023-11-24T06:11:36.207029+0000","last_unstale":"2023-11-24T06:11:37.509032+0000","last_undegraded":"2023-11-24T06:11:37.509032+0000","last_fullsized":"2023-11-24T06:11:37.509032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:27.651351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554133+0000","last_change":"2023-11-24T06:11:36.207972+0000","last_active":"2023-11-24T06:11:37.554133+0000","last_peered":"2023-11-24T06:11:37.554133+0000","last_clean":"2023-11-24T06:11:37.554133+0000","last_became_active":"2023-11-24T06:11:36.207510+0000","last_became_peered":"2023-11-24T06:11:36.207510+0000","last_unstale":"2023-11-24T06:11:37.554133+0000","last_undegraded":"2023-11-24T06:11:37.554133+0000","last_fullsized":"2023-11-24T06:11:37.554133+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:31:05.526287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.508980+0000","last_change":"2023-11-24T06:11:36.206803+0000","last_active":"2023-11-24T06:11:37.508980+0000","last_peered":"2023-11-24T06:11:37.508980+0000","last_clean":"2023-11-24T06:11:37.508980+0000","last_became_active":"2023-11-24T06:11:36.206526+0000","last_became_peered":"2023-11-24T06:11:36.206526+0000","last_unstale":"2023-11-24T06:11:37.508980+0000","last_undegraded":"2023-11-24T06:11:37.508980+0000","last_fullsized":"2023-11-24T06:11:37.508980+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:50.189327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.209497+0000","last_change":"2023-11-24T06:11:36.205986+0000","last_active":"2023-11-24T06:11:37.209497+0000","last_peered":"2023-11-24T06:11:37.209497+0000","last_clean":"2023-11-24T06:11:37.209497+0000","last_became_active":"2023-11-24T06:11:36.205691+0000","last_became_peered":"2023-11-24T06:11:36.205691+0000","last_unstale":"2023-11-24T06:11:37.209497+0000","last_undegraded":"2023-11-24T06:11:37.209497+0000","last_fullsized":"2023-11-24T06:11:37.209497+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:55.998773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554382+0000","last_change":"2023-11-24T06:11:36.209180+0000","last_active":"2023-11-24T06:11:37.554382+0000","last_peered":"2023-11-24T06:11:37.554382+0000","last_clean":"2023-11-24T06:11:37.554382+0000","last_became_active":"2023-11-24T06:11:36.208441+0000","last_became_peered":"2023-11-24T06:11:36.208441+0000","last_unstale":"2023-11-24T06:11:37.554382+0000","last_undegraded":"2023-11-24T06:11:37.554382+0000","last_fullsized":"2023-11-24T06:11:37.554382+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:17.891202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.203782+0000","last_change":"2023-11-24T06:11:36.205669+0000","last_active":"2023-11-24T06:11:37.203782+0000","last_peered":"2023-11-24T06:11:37.203782+0000","last_clean":"2023-11-24T06:11:37.203782+0000","last_became_active":"2023-11-24T06:11:36.205407+0000","last_became_peered":"2023-11-24T06:11:36.205407+0000","last_unstale":"2023-11-24T06:11:37.203782+0000","last_undegraded":"2023-11-24T06:11:37.203782+0000","last_fullsized":"2023-11-24T06:11:37.203782+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:51:36.230703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435244+0000","last_change":"2023-11-24T06:11:36.208933+0000","last_active":"2023-11-24T06:11:37.435244+0000","last_peered":"2023-11-24T06:11:37.435244+0000","last_clean":"2023-11-24T06:11:37.435244+0000","last_became_active":"2023-11-24T06:11:36.208676+0000","last_became_peered":"2023-11-24T06:11:36.208676+0000","last_unstale":"2023-11-24T06:11:37.435244+0000","last_undegraded":"2023-11-24T06:11:37.435244+0000","last_fullsized":"2023-11-24T06:11:37.435244+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:24.147739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.509042+0000","last_change":"2023-11-24T06:11:36.209231+0000","last_active":"2023-11-24T06:11:37.509042+0000","last_peered":"2023-11-24T06:11:37.509042+0000","last_clean":"2023-11-24T06:11:37.509042+0000","last_became_active":"2023-11-24T06:11:36.208932+0000","last_became_peered":"2023-11-24T06:11:36.208932+0000","last_unstale":"2023-11-24T06:11:37.509042+0000","last_undegraded":"2023-11-24T06:11:37.509042+0000","last_fullsized":"2023-11-24T06:11:37.509042+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:46.098851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.209712+0000","last_change":"2023-11-24T06:11:36.208949+0000","last_active":"2023-11-24T06:11:37.209712+0000","last_peered":"2023-11-24T06:11:37.209712+0000","last_clean":"2023-11-24T06:11:37.209712+0000","last_became_active":"2023-11-24T06:11:36.208654+0000","last_became_peered":"2023-11-24T06:11:36.208654+0000","last_unstale":"2023-11-24T06:11:37.209712+0000","last_undegraded":"2023-11-24T06:11:37.209712+0000","last_fullsized":"2023-11-24T06:11:37.209712+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:28:44.117998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210587+0000","last_change":"2023-11-24T06:11:36.209404+0000","last_active":"2023-11-24T06:11:37.210587+0000","last_peered":"2023-11-24T06:11:37.210587+0000","last_clean":"2023-11-24T06:11:37.210587+0000","last_became_active":"2023-11-24T06:11:36.209087+0000","last_became_peered":"2023-11-24T06:11:36.209087+0000","last_unstale":"2023-11-24T06:11:37.210587+0000","last_undegraded":"2023-11-24T06:11:37.210587+0000","last_fullsized":"2023-11-24T06:11:37.210587+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:39.836604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443269+0000","last_change":"2023-11-24T06:11:36.212817+0000","last_active":"2023-11-24T06:11:37.443269+0000","last_peered":"2023-11-24T06:11:37.443269+0000","last_clean":"2023-11-24T06:11:37.443269+0000","last_became_active":"2023-11-24T06:11:36.212100+0000","last_became_peered":"2023-11-24T06:11:36.212100+0000","last_unstale":"2023-11-24T06:11:37.443269+0000","last_undegraded":"2023-11-24T06:11:37.443269+0000","last_fullsized":"2023-11-24T06:11:37.443269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:57.703941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443431+0000","last_change":"2023-11-24T06:11:36.208215+0000","last_active":"2023-11-24T06:11:37.443431+0000","last_peered":"2023-11-24T06:11:37.443431+0000","last_clean":"2023-11-24T06:11:37.443431+0000","last_became_active":"2023-11-24T06:11:36.207964+0000","last_became_peered":"2023-11-24T06:11:36.207964+0000","last_unstale":"2023-11-24T06:11:37.443431+0000","last_undegraded":"2023-11-24T06:11:37.443431+0000","last_fullsized":"2023-11-24T06:11:37.443431+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:30.073223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210067+0000","last_change":"2023-11-24T06:11:36.208110+0000","last_active":"2023-11-24T06:11:37.210067+0000","last_peered":"2023-11-24T06:11:37.210067+0000","last_clean":"2023-11-24T06:11:37.210067+0000","last_became_active":"2023-11-24T06:11:36.207777+0000","last_became_peered":"2023-11-24T06:11:36.207777+0000","last_unstale":"2023-11-24T06:11:37.210067+0000","last_undegraded":"2023-11-24T06:11:37.210067+0000","last_fullsized":"2023-11-24T06:11:37.210067+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:52:27.211532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210425+0000","last_change":"2023-11-24T06:11:36.205020+0000","last_active":"2023-11-24T06:11:37.210425+0000","last_peered":"2023-11-24T06:11:37.210425+0000","last_clean":"2023-11-24T06:11:37.210425+0000","last_became_active":"2023-11-24T06:11:36.204742+0000","last_became_peered":"2023-11-24T06:11:36.204742+0000","last_unstale":"2023-11-24T06:11:37.210425+0000","last_undegraded":"2023-11-24T06:11:37.210425+0000","last_fullsized":"2023-11-24T06:11:37.210425+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:45:59.013648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554255+0000","last_change":"2023-11-24T06:11:36.205313+0000","last_active":"2023-11-24T06:11:37.554255+0000","last_peered":"2023-11-24T06:11:37.554255+0000","last_clean":"2023-11-24T06:11:37.554255+0000","last_became_active":"2023-11-24T06:11:36.204992+0000","last_became_peered":"2023-11-24T06:11:36.204992+0000","last_unstale":"2023-11-24T06:11:37.554255+0000","last_undegraded":"2023-11-24T06:11:37.554255+0000","last_fullsized":"2023-11-24T06:11:37.554255+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:47.199596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210528+0000","last_change":"2023-11-24T06:11:36.208323+0000","last_active":"2023-11-24T06:11:37.210528+0000","last_peered":"2023-11-24T06:11:37.210528+0000","last_clean":"2023-11-24T06:11:37.210528+0000","last_became_active":"2023-11-24T06:11:36.207989+0000","last_became_peered":"2023-11-24T06:11:36.207989+0000","last_unstale":"2023-11-24T06:11:37.210528+0000","last_undegraded":"2023-11-24T06:11:37.210528+0000","last_fullsized":"2023-11-24T06:11:37.210528+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:35:53.234505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.553975+0000","last_change":"2023-11-24T06:11:36.202366+0000","last_active":"2023-11-24T06:11:37.553975+0000","last_peered":"2023-11-24T06:11:37.553975+0000","last_clean":"2023-11-24T06:11:37.553975+0000","last_became_active":"2023-11-24T06:11:36.201935+0000","last_became_peered":"2023-11-24T06:11:36.201935+0000","last_unstale":"2023-11-24T06:11:37.553975+0000","last_undegraded":"2023-11-24T06:11:37.553975+0000","last_fullsized":"2023-11-24T06:11:37.553975+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:23:46.141922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.209817+0000","last_change":"2023-11-24T06:11:36.203094+0000","last_active":"2023-11-24T06:11:37.209817+0000","last_peered":"2023-11-24T06:11:37.209817+0000","last_clean":"2023-11-24T06:11:37.209817+0000","last_became_active":"2023-11-24T06:11:36.202788+0000","last_became_peered":"2023-11-24T06:11:36.202788+0000","last_unstale":"2023-11-24T06:11:37.209817+0000","last_undegraded":"2023-11-24T06:11:37.209817+0000","last_fullsized":"2023-11-24T06:11:37.209817+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:39.836604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.209829+0000","last_change":"2023-11-24T06:11:36.197579+0000","last_active":"2023-11-24T06:11:37.209829+0000","last_peered":"2023-11-24T06:11:37.209829+0000","last_clean":"2023-11-24T06:11:37.209829+0000","last_became_active":"2023-11-24T06:11:36.197247+0000","last_became_peered":"2023-11-24T06:11:36.197247+0000","last_unstale":"2023-11-24T06:11:37.209829+0000","last_undegraded":"2023-11-24T06:11:37.209829+0000","last_fullsized":"2023-11-24T06:11:37.209829+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:53:51.124014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435004+0000","last_change":"2023-11-24T06:11:36.205568+0000","last_active":"2023-11-24T06:11:37.435004+0000","last_peered":"2023-11-24T06:11:37.435004+0000","last_clean":"2023-11-24T06:11:37.435004+0000","last_became_active":"2023-11-24T06:11:36.205315+0000","last_became_peered":"2023-11-24T06:11:36.205315+0000","last_unstale":"2023-11-24T06:11:37.435004+0000","last_undegraded":"2023-11-24T06:11:37.435004+0000","last_fullsized":"2023-11-24T06:11:37.435004+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:44:33.992815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443572+0000","last_change":"2023-11-24T06:11:36.210925+0000","last_active":"2023-11-24T06:11:37.443572+0000","last_peered":"2023-11-24T06:11:37.443572+0000","last_clean":"2023-11-24T06:11:37.443572+0000","last_became_active":"2023-11-24T06:11:36.210371+0000","last_became_peered":"2023-11-24T06:11:36.210371+0000","last_unstale":"2023-11-24T06:11:37.443572+0000","last_undegraded":"2023-11-24T06:11:37.443572+0000","last_fullsized":"2023-11-24T06:11:37.443572+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:27.651351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.212973+0000","last_change":"2023-11-24T06:11:35.200318+0000","last_active":"2023-11-24T06:11:36.212973+0000","last_peered":"2023-11-24T06:11:36.212973+0000","last_clean":"2023-11-24T06:11:36.212973+0000","last_became_active":"2023-11-24T06:11:35.200107+0000","last_became_peered":"2023-11-24T06:11:35.200107+0000","last_unstale":"2023-11-24T06:11:36.212973+0000","last_undegraded":"2023-11-24T06:11:36.212973+0000","last_fullsized":"2023-11-24T06:11:36.212973+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:29:41.560611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554400+0000","last_change":"2023-11-24T06:11:36.209023+0000","last_active":"2023-11-24T06:11:37.554400+0000","last_peered":"2023-11-24T06:11:37.554400+0000","last_clean":"2023-11-24T06:11:37.554400+0000","last_became_active":"2023-11-24T06:11:36.208084+0000","last_became_peered":"2023-11-24T06:11:36.208084+0000","last_unstale":"2023-11-24T06:11:37.554400+0000","last_undegraded":"2023-11-24T06:11:37.554400+0000","last_fullsized":"2023-11-24T06:11:37.554400+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:45.794301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.205487+0000","last_change":"2023-11-24T06:11:35.199681+0000","last_active":"2023-11-24T06:11:36.205487+0000","last_peered":"2023-11-24T06:11:36.205487+0000","last_clean":"2023-11-24T06:11:36.205487+0000","last_became_active":"2023-11-24T06:11:35.199417+0000","last_became_peered":"2023-11-24T06:11:35.199417+0000","last_unstale":"2023-11-24T06:11:36.205487+0000","last_undegraded":"2023-11-24T06:11:36.205487+0000","last_fullsized":"2023-11-24T06:11:36.205487+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:56.701819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435204+0000","last_change":"2023-11-24T06:11:36.202212+0000","last_active":"2023-11-24T06:11:37.435204+0000","last_peered":"2023-11-24T06:11:37.435204+0000","last_clean":"2023-11-24T06:11:37.435204+0000","last_became_active":"2023-11-24T06:11:36.201926+0000","last_became_peered":"2023-11-24T06:11:36.201926+0000","last_unstale":"2023-11-24T06:11:37.435204+0000","last_undegraded":"2023-11-24T06:11:37.435204+0000","last_fullsized":"2023-11-24T06:11:37.435204+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.535831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.208235+0000","last_change":"2023-11-24T06:11:35.197380+0000","last_active":"2023-11-24T06:11:36.208235+0000","last_peered":"2023-11-24T06:11:36.208235+0000","last_clean":"2023-11-24T06:11:36.208235+0000","last_became_active":"2023-11-24T06:11:35.197112+0000","last_became_peered":"2023-11-24T06:11:35.197112+0000","last_unstale":"2023-11-24T06:11:36.208235+0000","last_undegraded":"2023-11-24T06:11:36.208235+0000","last_fullsized":"2023-11-24T06:11:36.208235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:14:26.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443654+0000","last_change":"2023-11-24T06:11:36.206490+0000","last_active":"2023-11-24T06:11:37.443654+0000","last_peered":"2023-11-24T06:11:37.443654+0000","last_clean":"2023-11-24T06:11:37.443654+0000","last_became_active":"2023-11-24T06:11:36.206207+0000","last_became_peered":"2023-11-24T06:11:36.206207+0000","last_unstale":"2023-11-24T06:11:37.443654+0000","last_undegraded":"2023-11-24T06:11:37.443654+0000","last_fullsized":"2023-11-24T06:11:37.443654+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:46.098851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.212571+0000","last_change":"2023-11-24T06:11:35.206053+0000","last_active":"2023-11-24T06:11:36.212571+0000","last_peered":"2023-11-24T06:11:36.212571+0000","last_clean":"2023-11-24T06:11:36.212571+0000","last_became_active":"2023-11-24T06:11:35.205752+0000","last_became_peered":"2023-11-24T06:11:35.205752+0000","last_unstale":"2023-11-24T06:11:36.212571+0000","last_undegraded":"2023-11-24T06:11:36.212571+0000","last_fullsized":"2023-11-24T06:11:36.212571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:47:28.876567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210332+0000","last_change":"2023-11-24T06:11:36.193874+0000","last_active":"2023-11-24T06:11:37.210332+0000","last_peered":"2023-11-24T06:11:37.210332+0000","last_clean":"2023-11-24T06:11:37.210332+0000","last_became_active":"2023-11-24T06:11:36.193553+0000","last_became_peered":"2023-11-24T06:11:36.193553+0000","last_unstale":"2023-11-24T06:11:37.210332+0000","last_undegraded":"2023-11-24T06:11:37.210332+0000","last_fullsized":"2023-11-24T06:11:37.210332+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:36.665408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.212722+0000","last_change":"2023-11-24T06:11:35.198555+0000","last_active":"2023-11-24T06:11:36.212722+0000","last_peered":"2023-11-24T06:11:36.212722+0000","last_clean":"2023-11-24T06:11:36.212722+0000","last_became_active":"2023-11-24T06:11:35.198353+0000","last_became_peered":"2023-11-24T06:11:35.198353+0000","last_unstale":"2023-11-24T06:11:36.212722+0000","last_undegraded":"2023-11-24T06:11:36.212722+0000","last_fullsized":"2023-11-24T06:11:36.212722+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:21:42.164402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.209725+0000","last_change":"2023-11-24T06:11:36.206279+0000","last_active":"2023-11-24T06:11:37.209725+0000","last_peered":"2023-11-24T06:11:37.209725+0000","last_clean":"2023-11-24T06:11:37.209725+0000","last_became_active":"2023-11-24T06:11:36.206010+0000","last_became_peered":"2023-11-24T06:11:36.206010+0000","last_unstale":"2023-11-24T06:11:37.209725+0000","last_undegraded":"2023-11-24T06:11:37.209725+0000","last_fullsized":"2023-11-24T06:11:37.209725+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:48.909317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.205910+0000","last_change":"2023-11-24T06:11:35.199877+0000","last_active":"2023-11-24T06:11:36.205910+0000","last_peered":"2023-11-24T06:11:36.205910+0000","last_clean":"2023-11-24T06:11:36.205910+0000","last_became_active":"2023-11-24T06:11:35.199626+0000","last_became_peered":"2023-11-24T06:11:35.199626+0000","last_unstale":"2023-11-24T06:11:36.205910+0000","last_undegraded":"2023-11-24T06:11:36.205910+0000","last_fullsized":"2023-11-24T06:11:36.205910+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:07:24.558366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.553867+0000","last_change":"2023-11-24T06:11:36.211271+0000","last_active":"2023-11-24T06:11:37.553867+0000","last_peered":"2023-11-24T06:11:37.553867+0000","last_clean":"2023-11-24T06:11:37.553867+0000","last_became_active":"2023-11-24T06:11:36.210977+0000","last_became_peered":"2023-11-24T06:11:36.210977+0000","last_unstale":"2023-11-24T06:11:37.553867+0000","last_undegraded":"2023-11-24T06:11:37.553867+0000","last_fullsized":"2023-11-24T06:11:37.553867+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:51:06.537582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.185301+0000","last_change":"2023-11-24T06:11:35.209654+0000","last_active":"2023-11-24T06:11:36.185301+0000","last_peered":"2023-11-24T06:11:36.185301+0000","last_clean":"2023-11-24T06:11:36.185301+0000","last_became_active":"2023-11-24T06:11:35.209437+0000","last_became_peered":"2023-11-24T06:11:35.209437+0000","last_unstale":"2023-11-24T06:11:36.185301+0000","last_undegraded":"2023-11-24T06:11:36.185301+0000","last_fullsized":"2023-11-24T06:11:36.185301+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:58:44.792178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.508979+0000","last_change":"2023-11-24T06:11:36.206344+0000","last_active":"2023-11-24T06:11:37.508979+0000","last_peered":"2023-11-24T06:11:37.508979+0000","last_clean":"2023-11-24T06:11:37.508979+0000","last_became_active":"2023-11-24T06:11:36.206087+0000","last_became_peered":"2023-11-24T06:11:36.206087+0000","last_unstale":"2023-11-24T06:11:37.508979+0000","last_undegraded":"2023-11-24T06:11:37.508979+0000","last_fullsized":"2023-11-24T06:11:37.508979+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:30.073223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.193100+0000","last_change":"2023-11-24T06:11:35.202662+0000","last_active":"2023-11-24T06:11:36.193100+0000","last_peered":"2023-11-24T06:11:36.193100+0000","last_clean":"2023-11-24T06:11:36.193100+0000","last_became_active":"2023-11-24T06:11:35.202431+0000","last_became_peered":"2023-11-24T06:11:35.202431+0000","last_unstale":"2023-11-24T06:11:36.193100+0000","last_undegraded":"2023-11-24T06:11:36.193100+0000","last_fullsized":"2023-11-24T06:11:36.193100+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:31.957064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210425+0000","last_change":"2023-11-24T06:11:36.202327+0000","last_active":"2023-11-24T06:11:37.210425+0000","last_peered":"2023-11-24T06:11:37.210425+0000","last_clean":"2023-11-24T06:11:37.210425+0000","last_became_active":"2023-11-24T06:11:36.202025+0000","last_became_peered":"2023-11-24T06:11:36.202025+0000","last_unstale":"2023-11-24T06:11:37.210425+0000","last_undegraded":"2023-11-24T06:11:37.210425+0000","last_fullsized":"2023-11-24T06:11:37.210425+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:54.583198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.188705+0000","last_change":"2023-11-24T06:11:35.203386+0000","last_active":"2023-11-24T06:11:36.188705+0000","last_peered":"2023-11-24T06:11:36.188705+0000","last_clean":"2023-11-24T06:11:36.188705+0000","last_became_active":"2023-11-24T06:11:35.203010+0000","last_became_peered":"2023-11-24T06:11:35.203010+0000","last_unstale":"2023-11-24T06:11:36.188705+0000","last_undegraded":"2023-11-24T06:11:36.188705+0000","last_fullsized":"2023-11-24T06:11:36.188705+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:00.433689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443629+0000","last_change":"2023-11-24T06:11:36.207120+0000","last_active":"2023-11-24T06:11:37.443629+0000","last_peered":"2023-11-24T06:11:37.443629+0000","last_clean":"2023-11-24T06:11:37.443629+0000","last_became_active":"2023-11-24T06:11:36.206771+0000","last_became_peered":"2023-11-24T06:11:36.206771+0000","last_unstale":"2023-11-24T06:11:37.443629+0000","last_undegraded":"2023-11-24T06:11:37.443629+0000","last_fullsized":"2023-11-24T06:11:37.443629+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:23.078660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.201457+0000","last_change":"2023-11-24T06:11:35.197529+0000","last_active":"2023-11-24T06:11:36.201457+0000","last_peered":"2023-11-24T06:11:36.201457+0000","last_clean":"2023-11-24T06:11:36.201457+0000","last_became_active":"2023-11-24T06:11:35.196915+0000","last_became_peered":"2023-11-24T06:11:35.196915+0000","last_unstale":"2023-11-24T06:11:36.201457+0000","last_undegraded":"2023-11-24T06:11:36.201457+0000","last_fullsized":"2023-11-24T06:11:36.201457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:05:01.629357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435536+0000","last_change":"2023-11-24T06:11:36.210378+0000","last_active":"2023-11-24T06:11:37.435536+0000","last_peered":"2023-11-24T06:11:37.435536+0000","last_clean":"2023-11-24T06:11:37.435536+0000","last_became_active":"2023-11-24T06:11:36.210097+0000","last_became_peered":"2023-11-24T06:11:36.210097+0000","last_unstale":"2023-11-24T06:11:37.435536+0000","last_undegraded":"2023-11-24T06:11:37.435536+0000","last_fullsized":"2023-11-24T06:11:37.435536+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:48.909317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.212525+0000","last_change":"2023-11-24T06:11:35.199414+0000","last_active":"2023-11-24T06:11:36.212525+0000","last_peered":"2023-11-24T06:11:36.212525+0000","last_clean":"2023-11-24T06:11:36.212525+0000","last_became_active":"2023-11-24T06:11:35.199137+0000","last_became_peered":"2023-11-24T06:11:35.199137+0000","last_unstale":"2023-11-24T06:11:36.212525+0000","last_undegraded":"2023-11-24T06:11:36.212525+0000","last_fullsized":"2023-11-24T06:11:36.212525+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:02.941743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554191+0000","last_change":"2023-11-24T06:11:36.201624+0000","last_active":"2023-11-24T06:11:37.554191+0000","last_peered":"2023-11-24T06:11:37.554191+0000","last_clean":"2023-11-24T06:11:37.554191+0000","last_became_active":"2023-11-24T06:11:36.201338+0000","last_became_peered":"2023-11-24T06:11:36.201338+0000","last_unstale":"2023-11-24T06:11:37.554191+0000","last_undegraded":"2023-11-24T06:11:37.554191+0000","last_fullsized":"2023-11-24T06:11:37.554191+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:39.934816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.216087+0000","last_change":"2023-11-24T06:11:35.202411+0000","last_active":"2023-11-24T06:11:36.216087+0000","last_peered":"2023-11-24T06:11:36.216087+0000","last_clean":"2023-11-24T06:11:36.216087+0000","last_became_active":"2023-11-24T06:11:35.202161+0000","last_became_peered":"2023-11-24T06:11:35.202161+0000","last_unstale":"2023-11-24T06:11:36.216087+0000","last_undegraded":"2023-11-24T06:11:36.216087+0000","last_fullsized":"2023-11-24T06:11:36.216087+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:46.799215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443325+0000","last_change":"2023-11-24T06:11:36.214327+0000","last_active":"2023-11-24T06:11:37.443325+0000","last_peered":"2023-11-24T06:11:37.443325+0000","last_clean":"2023-11-24T06:11:37.443325+0000","last_became_active":"2023-11-24T06:11:36.214002+0000","last_became_peered":"2023-11-24T06:11:36.214002+0000","last_unstale":"2023-11-24T06:11:37.443325+0000","last_undegraded":"2023-11-24T06:11:37.443325+0000","last_fullsized":"2023-11-24T06:11:37.443325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:30:56.806877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.210316+0000","last_change":"2023-11-24T06:11:35.199163+0000","last_active":"2023-11-24T06:11:36.210316+0000","last_peered":"2023-11-24T06:11:36.210316+0000","last_clean":"2023-11-24T06:11:36.210316+0000","last_became_active":"2023-11-24T06:11:35.198885+0000","last_became_peered":"2023-11-24T06:11:35.198885+0000","last_unstale":"2023-11-24T06:11:36.210316+0000","last_undegraded":"2023-11-24T06:11:36.210316+0000","last_fullsized":"2023-11-24T06:11:36.210316+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:38.932693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210087+0000","last_change":"2023-11-24T06:11:36.209351+0000","last_active":"2023-11-24T06:11:37.210087+0000","last_peered":"2023-11-24T06:11:37.210087+0000","last_clean":"2023-11-24T06:11:37.210087+0000","last_became_active":"2023-11-24T06:11:36.209040+0000","last_became_peered":"2023-11-24T06:11:36.209040+0000","last_unstale":"2023-11-24T06:11:37.210087+0000","last_undegraded":"2023-11-24T06:11:37.210087+0000","last_fullsized":"2023-11-24T06:11:37.210087+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:14.970225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.195924+0000","last_change":"2023-11-24T06:11:35.203769+0000","last_active":"2023-11-24T06:11:36.195924+0000","last_peered":"2023-11-24T06:11:36.195924+0000","last_clean":"2023-11-24T06:11:36.195924+0000","last_became_active":"2023-11-24T06:11:35.203458+0000","last_became_peered":"2023-11-24T06:11:35.203458+0000","last_unstale":"2023-11-24T06:11:36.195924+0000","last_undegraded":"2023-11-24T06:11:36.195924+0000","last_fullsized":"2023-11-24T06:11:36.195924+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:28.013426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.509009+0000","last_change":"2023-11-24T06:11:36.207235+0000","last_active":"2023-11-24T06:11:37.509009+0000","last_peered":"2023-11-24T06:11:37.509009+0000","last_clean":"2023-11-24T06:11:37.509009+0000","last_became_active":"2023-11-24T06:11:36.206003+0000","last_became_peered":"2023-11-24T06:11:36.206003+0000","last_unstale":"2023-11-24T06:11:37.509009+0000","last_undegraded":"2023-11-24T06:11:37.509009+0000","last_fullsized":"2023-11-24T06:11:37.509009+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:24:24.262110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.187803+0000","last_change":"2023-11-24T06:11:35.192301+0000","last_active":"2023-11-24T06:11:36.187803+0000","last_peered":"2023-11-24T06:11:36.187803+0000","last_clean":"2023-11-24T06:11:36.187803+0000","last_became_active":"2023-11-24T06:11:35.192014+0000","last_became_peered":"2023-11-24T06:11:35.192014+0000","last_unstale":"2023-11-24T06:11:36.187803+0000","last_undegraded":"2023-11-24T06:11:36.187803+0000","last_fullsized":"2023-11-24T06:11:36.187803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:47.907194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554040+0000","last_change":"2023-11-24T06:11:36.201230+0000","last_active":"2023-11-24T06:11:37.554040+0000","last_peered":"2023-11-24T06:11:37.554040+0000","last_clean":"2023-11-24T06:11:37.554040+0000","last_became_active":"2023-11-24T06:11:36.200938+0000","last_became_peered":"2023-11-24T06:11:36.200938+0000","last_unstale":"2023-11-24T06:11:37.554040+0000","last_undegraded":"2023-11-24T06:11:37.554040+0000","last_fullsized":"2023-11-24T06:11:37.554040+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:55.444402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.215064+0000","last_change":"2023-11-24T06:11:35.198642+0000","last_active":"2023-11-24T06:11:36.215064+0000","last_peered":"2023-11-24T06:11:36.215064+0000","last_clean":"2023-11-24T06:11:36.215064+0000","last_became_active":"2023-11-24T06:11:35.198370+0000","last_became_peered":"2023-11-24T06:11:35.198370+0000","last_unstale":"2023-11-24T06:11:36.215064+0000","last_undegraded":"2023-11-24T06:11:36.215064+0000","last_fullsized":"2023-11-24T06:11:36.215064+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:51:31.752252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.210162+0000","last_change":"2023-11-24T06:11:36.205320+0000","last_active":"2023-11-24T06:11:37.210162+0000","last_peered":"2023-11-24T06:11:37.210162+0000","last_clean":"2023-11-24T06:11:37.210162+0000","last_became_active":"2023-11-24T06:11:36.204986+0000","last_became_peered":"2023-11-24T06:11:36.204986+0000","last_unstale":"2023-11-24T06:11:37.210162+0000","last_undegraded":"2023-11-24T06:11:37.210162+0000","last_fullsized":"2023-11-24T06:11:37.210162+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:43.872104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.214547+0000","last_change":"2023-11-24T06:11:35.203932+0000","last_active":"2023-11-24T06:11:36.214547+0000","last_peered":"2023-11-24T06:11:36.214547+0000","last_clean":"2023-11-24T06:11:36.214547+0000","last_became_active":"2023-11-24T06:11:35.203630+0000","last_became_peered":"2023-11-24T06:11:35.203630+0000","last_unstale":"2023-11-24T06:11:36.214547+0000","last_undegraded":"2023-11-24T06:11:36.214547+0000","last_fullsized":"2023-11-24T06:11:36.214547+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:20.139044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443724+0000","last_change":"2023-11-24T06:11:36.211168+0000","last_active":"2023-11-24T06:11:37.443724+0000","last_peered":"2023-11-24T06:11:37.443724+0000","last_clean":"2023-11-24T06:11:37.443724+0000","last_became_active":"2023-11-24T06:11:36.210529+0000","last_became_peered":"2023-11-24T06:11:36.210529+0000","last_unstale":"2023-11-24T06:11:37.443724+0000","last_undegraded":"2023-11-24T06:11:37.443724+0000","last_fullsized":"2023-11-24T06:11:37.443724+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:13.438265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.208037+0000","last_change":"2023-11-24T06:11:35.198432+0000","last_active":"2023-11-24T06:11:36.208037+0000","last_peered":"2023-11-24T06:11:36.208037+0000","last_clean":"2023-11-24T06:11:36.208037+0000","last_became_active":"2023-11-24T06:11:35.198125+0000","last_became_peered":"2023-11-24T06:11:35.198125+0000","last_unstale":"2023-11-24T06:11:36.208037+0000","last_undegraded":"2023-11-24T06:11:36.208037+0000","last_fullsized":"2023-11-24T06:11:36.208037+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:44:32.990693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.203823+0000","last_change":"2023-11-24T06:11:36.198213+0000","last_active":"2023-11-24T06:11:37.203823+0000","last_peered":"2023-11-24T06:11:37.203823+0000","last_clean":"2023-11-24T06:11:37.203823+0000","last_became_active":"2023-11-24T06:11:36.197912+0000","last_became_peered":"2023-11-24T06:11:36.197912+0000","last_unstale":"2023-11-24T06:11:37.203823+0000","last_undegraded":"2023-11-24T06:11:37.203823+0000","last_fullsized":"2023-11-24T06:11:37.203823+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:55.563253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.210019+0000","last_change":"2023-11-24T06:11:35.197602+0000","last_active":"2023-11-24T06:11:36.210019+0000","last_peered":"2023-11-24T06:11:36.210019+0000","last_clean":"2023-11-24T06:11:36.210019+0000","last_became_active":"2023-11-24T06:11:35.196947+0000","last_became_peered":"2023-11-24T06:11:35.196947+0000","last_unstale":"2023-11-24T06:11:36.210019+0000","last_undegraded":"2023-11-24T06:11:36.210019+0000","last_fullsized":"2023-11-24T06:11:36.210019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:14:26.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554350+0000","last_change":"2023-11-24T06:11:36.209147+0000","last_active":"2023-11-24T06:11:37.554350+0000","last_peered":"2023-11-24T06:11:37.554350+0000","last_clean":"2023-11-24T06:11:37.554350+0000","last_became_active":"2023-11-24T06:11:36.208825+0000","last_became_peered":"2023-11-24T06:11:36.208825+0000","last_unstale":"2023-11-24T06:11:37.554350+0000","last_undegraded":"2023-11-24T06:11:37.554350+0000","last_fullsized":"2023-11-24T06:11:37.554350+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:20:55.146506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.198486+0000","last_change":"2023-11-24T06:11:35.198311+0000","last_active":"2023-11-24T06:11:36.198486+0000","last_peered":"2023-11-24T06:11:36.198486+0000","last_clean":"2023-11-24T06:11:36.198486+0000","last_became_active":"2023-11-24T06:11:35.198037+0000","last_became_peered":"2023-11-24T06:11:35.198037+0000","last_unstale":"2023-11-24T06:11:36.198486+0000","last_undegraded":"2023-11-24T06:11:36.198486+0000","last_fullsized":"2023-11-24T06:11:36.198486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:38.834481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554110+0000","last_change":"2023-11-24T06:11:36.201592+0000","last_active":"2023-11-24T06:11:37.554110+0000","last_peered":"2023-11-24T06:11:37.554110+0000","last_clean":"2023-11-24T06:11:37.554110+0000","last_became_active":"2023-11-24T06:11:36.200795+0000","last_became_peered":"2023-11-24T06:11:36.200795+0000","last_unstale":"2023-11-24T06:11:37.554110+0000","last_undegraded":"2023-11-24T06:11:37.554110+0000","last_fullsized":"2023-11-24T06:11:37.554110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:50.785734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.195569+0000","last_change":"2023-11-24T06:11:35.199868+0000","last_active":"2023-11-24T06:11:36.195569+0000","last_peered":"2023-11-24T06:11:36.195569+0000","last_clean":"2023-11-24T06:11:36.195569+0000","last_became_active":"2023-11-24T06:11:35.199596+0000","last_became_peered":"2023-11-24T06:11:35.199596+0000","last_unstale":"2023-11-24T06:11:36.195569+0000","last_undegraded":"2023-11-24T06:11:36.195569+0000","last_fullsized":"2023-11-24T06:11:36.195569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:08.350691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.553795+0000","last_change":"2023-11-24T06:11:36.206608+0000","last_active":"2023-11-24T06:11:37.553795+0000","last_peered":"2023-11-24T06:11:37.553795+0000","last_clean":"2023-11-24T06:11:37.553795+0000","last_became_active":"2023-11-24T06:11:36.206332+0000","last_became_peered":"2023-11-24T06:11:36.206332+0000","last_unstale":"2023-11-24T06:11:37.553795+0000","last_undegraded":"2023-11-24T06:11:37.553795+0000","last_fullsized":"2023-11-24T06:11:37.553795+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:05:38.755434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.187941+0000","last_change":"2023-11-24T06:11:35.204843+0000","last_active":"2023-11-24T06:11:36.187941+0000","last_peered":"2023-11-24T06:11:36.187941+0000","last_clean":"2023-11-24T06:11:36.187941+0000","last_became_active":"2023-11-24T06:11:35.204625+0000","last_became_peered":"2023-11-24T06:11:35.204625+0000","last_unstale":"2023-11-24T06:11:36.187941+0000","last_undegraded":"2023-11-24T06:11:36.187941+0000","last_fullsized":"2023-11-24T06:11:36.187941+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:05.614705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.553843+0000","last_change":"2023-11-24T06:11:36.205962+0000","last_active":"2023-11-24T06:11:37.553843+0000","last_peered":"2023-11-24T06:11:37.553843+0000","last_clean":"2023-11-24T06:11:37.553843+0000","last_became_active":"2023-11-24T06:11:36.205669+0000","last_became_peered":"2023-11-24T06:11:36.205669+0000","last_unstale":"2023-11-24T06:11:37.553843+0000","last_undegraded":"2023-11-24T06:11:37.553843+0000","last_fullsized":"2023-11-24T06:11:37.553843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:22.960483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.192779+0000","last_change":"2023-11-24T06:11:35.203494+0000","last_active":"2023-11-24T06:11:36.192779+0000","last_peered":"2023-11-24T06:11:36.192779+0000","last_clean":"2023-11-24T06:11:36.192779+0000","last_became_active":"2023-11-24T06:11:35.203200+0000","last_became_peered":"2023-11-24T06:11:35.203200+0000","last_unstale":"2023-11-24T06:11:36.192779+0000","last_undegraded":"2023-11-24T06:11:36.192779+0000","last_fullsized":"2023-11-24T06:11:36.192779+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:59:09.839853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.509295+0000","last_change":"2023-11-24T06:11:36.202540+0000","last_active":"2023-11-24T06:11:37.509295+0000","last_peered":"2023-11-24T06:11:37.509295+0000","last_clean":"2023-11-24T06:11:37.509295+0000","last_became_active":"2023-11-24T06:11:36.202244+0000","last_became_peered":"2023-11-24T06:11:36.202244+0000","last_unstale":"2023-11-24T06:11:37.509295+0000","last_undegraded":"2023-11-24T06:11:37.509295+0000","last_fullsized":"2023-11-24T06:11:37.509295+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:57:23.078660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.214750+0000","last_change":"2023-11-24T06:11:35.203940+0000","last_active":"2023-11-24T06:11:36.214750+0000","last_peered":"2023-11-24T06:11:36.214750+0000","last_clean":"2023-11-24T06:11:36.214750+0000","last_became_active":"2023-11-24T06:11:35.203671+0000","last_became_peered":"2023-11-24T06:11:35.203671+0000","last_unstale":"2023-11-24T06:11:36.214750+0000","last_undegraded":"2023-11-24T06:11:36.214750+0000","last_fullsized":"2023-11-24T06:11:36.214750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:34:27.845309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.209763+0000","last_change":"2023-11-24T06:11:36.199697+0000","last_active":"2023-11-24T06:11:37.209763+0000","last_peered":"2023-11-24T06:11:37.209763+0000","last_clean":"2023-11-24T06:11:37.209763+0000","last_became_active":"2023-11-24T06:11:36.199440+0000","last_became_peered":"2023-11-24T06:11:36.199440+0000","last_unstale":"2023-11-24T06:11:37.209763+0000","last_undegraded":"2023-11-24T06:11:37.209763+0000","last_fullsized":"2023-11-24T06:11:37.209763+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:35:53.234505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.188137+0000","last_change":"2023-11-24T06:11:35.192603+0000","last_active":"2023-11-24T06:11:36.188137+0000","last_peered":"2023-11-24T06:11:36.188137+0000","last_clean":"2023-11-24T06:11:36.188137+0000","last_became_active":"2023-11-24T06:11:35.191847+0000","last_became_peered":"2023-11-24T06:11:35.191847+0000","last_unstale":"2023-11-24T06:11:36.188137+0000","last_undegraded":"2023-11-24T06:11:36.188137+0000","last_fullsized":"2023-11-24T06:11:36.188137+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:52:26.209409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443490+0000","last_change":"2023-11-24T06:11:36.207555+0000","last_active":"2023-11-24T06:11:37.443490+0000","last_peered":"2023-11-24T06:11:37.443490+0000","last_clean":"2023-11-24T06:11:37.443490+0000","last_became_active":"2023-11-24T06:11:36.207360+0000","last_became_peered":"2023-11-24T06:11:36.207360+0000","last_unstale":"2023-11-24T06:11:37.443490+0000","last_undegraded":"2023-11-24T06:11:37.443490+0000","last_fullsized":"2023-11-24T06:11:37.443490+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:42:50.189327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.190148+0000","last_change":"2023-11-24T06:11:35.203951+0000","last_active":"2023-11-24T06:11:36.190148+0000","last_peered":"2023-11-24T06:11:36.190148+0000","last_clean":"2023-11-24T06:11:36.190148+0000","last_became_active":"2023-11-24T06:11:35.203621+0000","last_became_peered":"2023-11-24T06:11:35.203621+0000","last_unstale":"2023-11-24T06:11:36.190148+0000","last_undegraded":"2023-11-24T06:11:36.190148+0000","last_fullsized":"2023-11-24T06:11:36.190148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:30.698930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.443652+0000","last_change":"2023-11-24T06:11:36.212537+0000","last_active":"2023-11-24T06:11:37.443652+0000","last_peered":"2023-11-24T06:11:37.443652+0000","last_clean":"2023-11-24T06:11:37.443652+0000","last_became_active":"2023-11-24T06:11:36.212178+0000","last_became_peered":"2023-11-24T06:11:36.212178+0000","last_unstale":"2023-11-24T06:11:37.443652+0000","last_undegraded":"2023-11-24T06:11:37.443652+0000","last_fullsized":"2023-11-24T06:11:37.443652+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:16:12.358396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.194003+0000","last_change":"2023-11-24T06:11:35.195579+0000","last_active":"2023-11-24T06:11:36.194003+0000","last_peered":"2023-11-24T06:11:36.194003+0000","last_clean":"2023-11-24T06:11:36.194003+0000","last_became_active":"2023-11-24T06:11:35.195306+0000","last_became_peered":"2023-11-24T06:11:35.195306+0000","last_unstale":"2023-11-24T06:11:36.194003+0000","last_undegraded":"2023-11-24T06:11:36.194003+0000","last_fullsized":"2023-11-24T06:11:36.194003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:52:35.231797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554181+0000","last_change":"2023-11-24T06:11:36.197267+0000","last_active":"2023-11-24T06:11:37.554181+0000","last_peered":"2023-11-24T06:11:37.554181+0000","last_clean":"2023-11-24T06:11:37.554181+0000","last_became_active":"2023-11-24T06:11:36.196964+0000","last_became_peered":"2023-11-24T06:11:36.196964+0000","last_unstale":"2023-11-24T06:11:37.554181+0000","last_undegraded":"2023-11-24T06:11:37.554181+0000","last_fullsized":"2023-11-24T06:11:37.554181+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:33:26.719282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.190051+0000","last_change":"2023-11-24T06:11:35.201419+0000","last_active":"2023-11-24T06:11:36.190051+0000","last_peered":"2023-11-24T06:11:36.190051+0000","last_clean":"2023-11-24T06:11:36.190051+0000","last_became_active":"2023-11-24T06:11:35.201141+0000","last_became_peered":"2023-11-24T06:11:35.201141+0000","last_unstale":"2023-11-24T06:11:36.190051+0000","last_undegraded":"2023-11-24T06:11:36.190051+0000","last_fullsized":"2023-11-24T06:11:36.190051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:01.818389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435483+0000","last_change":"2023-11-24T06:11:36.202148+0000","last_active":"2023-11-24T06:11:37.435483+0000","last_peered":"2023-11-24T06:11:37.435483+0000","last_clean":"2023-11-24T06:11:37.435483+0000","last_became_active":"2023-11-24T06:11:36.201874+0000","last_became_peered":"2023-11-24T06:11:36.201874+0000","last_unstale":"2023-11-24T06:11:37.435483+0000","last_undegraded":"2023-11-24T06:11:37.435483+0000","last_fullsized":"2023-11-24T06:11:37.435483+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:01:55.998773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.185429+0000","last_change":"2023-11-24T06:11:35.209739+0000","last_active":"2023-11-24T06:11:36.185429+0000","last_peered":"2023-11-24T06:11:36.185429+0000","last_clean":"2023-11-24T06:11:36.185429+0000","last_became_active":"2023-11-24T06:11:35.209452+0000","last_became_peered":"2023-11-24T06:11:35.209452+0000","last_unstale":"2023-11-24T06:11:36.185429+0000","last_undegraded":"2023-11-24T06:11:36.185429+0000","last_fullsized":"2023-11-24T06:11:36.185429+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:04.637065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.203565+0000","last_change":"2023-11-24T06:11:36.200634+0000","last_active":"2023-11-24T06:11:37.203565+0000","last_peered":"2023-11-24T06:11:37.203565+0000","last_clean":"2023-11-24T06:11:37.203565+0000","last_became_active":"2023-11-24T06:11:36.200338+0000","last_became_peered":"2023-11-24T06:11:36.200338+0000","last_unstale":"2023-11-24T06:11:37.203565+0000","last_undegraded":"2023-11-24T06:11:37.203565+0000","last_fullsized":"2023-11-24T06:11:37.203565+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:36.524959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.187470+0000","last_change":"2023-11-24T06:11:35.196161+0000","last_active":"2023-11-24T06:11:36.187470+0000","last_peered":"2023-11-24T06:11:36.187470+0000","last_clean":"2023-11-24T06:11:36.187470+0000","last_became_active":"2023-11-24T06:11:35.195875+0000","last_became_peered":"2023-11-24T06:11:35.195875+0000","last_unstale":"2023-11-24T06:11:36.187470+0000","last_undegraded":"2023-11-24T06:11:36.187470+0000","last_fullsized":"2023-11-24T06:11:36.187470+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:21:28.013426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.435007+0000","last_change":"2023-11-24T06:11:36.200713+0000","last_active":"2023-11-24T06:11:37.435007+0000","last_peered":"2023-11-24T06:11:37.435007+0000","last_clean":"2023-11-24T06:11:37.435007+0000","last_became_active":"2023-11-24T06:11:36.200483+0000","last_became_peered":"2023-11-24T06:11:36.200483+0000","last_unstale":"2023-11-24T06:11:37.435007+0000","last_undegraded":"2023-11-24T06:11:37.435007+0000","last_fullsized":"2023-11-24T06:11:37.435007+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:53:53.022510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.195367+0000","last_change":"2023-11-24T06:11:35.205909+0000","last_active":"2023-11-24T06:11:36.195367+0000","last_peered":"2023-11-24T06:11:36.195367+0000","last_clean":"2023-11-24T06:11:36.195367+0000","last_became_active":"2023-11-24T06:11:35.205606+0000","last_became_peered":"2023-11-24T06:11:35.205606+0000","last_unstale":"2023-11-24T06:11:36.195367+0000","last_undegraded":"2023-11-24T06:11:36.195367+0000","last_fullsized":"2023-11-24T06:11:36.195367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:38:11.825904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T06:11:37.554326+0000","last_change":"2023-11-24T06:11:36.206749+0000","last_active":"2023-11-24T06:11:37.554326+0000","last_peered":"2023-11-24T06:11:37.554326+0000","last_clean":"2023-11-24T06:11:37.554326+0000","last_became_active":"2023-11-24T06:11:36.205783+0000","last_became_peered":"2023-11-24T06:11:36.205783+0000","last_unstale":"2023-11-24T06:11:37.554326+0000","last_undegraded":"2023-11-24T06:11:37.554326+0000","last_fullsized":"2023-11-24T06:11:37.554326+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:35.176246+0000","last_clean_scrub_stamp":"2023-11-24T06:11:35.176246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:05.433008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.209823+0000","last_change":"2023-11-24T06:11:35.205795+0000","last_active":"2023-11-24T06:11:36.209823+0000","last_peered":"2023-11-24T06:11:36.209823+0000","last_clean":"2023-11-24T06:11:36.209823+0000","last_became_active":"2023-11-24T06:11:35.205494+0000","last_became_peered":"2023-11-24T06:11:35.205494+0000","last_unstale":"2023-11-24T06:11:36.209823+0000","last_undegraded":"2023-11-24T06:11:36.209823+0000","last_fullsized":"2023-11-24T06:11:36.209823+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:11:34.174123+0000","last_clean_scrub_stamp":"2023-11-24T06:11:34.174123+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:22.205416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":208,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-24T06:11:36.198312+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:11:36.198312+0000","last_peered":"2023-11-24T06:11:36.198312+0000","last_clean":"2023-11-24T06:11:36.198312+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:11:36.198312+0000","last_undegraded":"2023-11-24T06:11:36.198312+0000","last_fullsized":"2023-11-24T06:11:36.198312+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28468,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343372,"statfs":{"total":96636764160,"available":96607612928,"internally_reserved":0,"allocated":1478656,"data_stored":963896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.87,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53100000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68000000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.628},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59299999999999997,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82599999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.495},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.373,"5min":0.371,"15min":0.371},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.41199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64600000000000002},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":373004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.76800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34200000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.83099999999999996,"15min":0.88900000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.621,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68600000000000005,"5min":1.371,"15min":1.371},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.57899999999999996,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66100000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.83099999999999996,"15min":0.875},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70299999999999996,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.64000000000000001,"15min":0.62},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":5.2830000000000004,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70199999999999996,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71599999999999997,"5min":1.407,"15min":1.407},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54700000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78400000000000003,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61099999999999999,"15min":0.61299999999999999},"min":{"1min":0.42299999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.76600000000000001,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.38,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.44,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":375140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.28999999999999998,"5min":0.33600000000000002,"15min":0.34399999999999997},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.45100000000000001,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.25900000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.48599999999999999,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.27500000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.33400000000000002,"15min":0.33600000000000002},"min":{"1min":0.24399999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.47399999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34999999999999998,"5min":0.375,"15min":0.379},"min":{"1min":0.28499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.52600000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.39900000000000002,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.497,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35199999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46000000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.628,"15min":0.63700000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.71299999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38900000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78900000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.497},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.46800000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.78100000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.435}]},{"osd":7,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81100000000000005,"5min":0.88,"15min":0.88},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.75700000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.48799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28456,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343384,"statfs":{"total":96636764160,"available":96607625216,"internally_reserved":0,"allocated":1470464,"data_stored":963280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.72599999999999998,"15min":0.79200000000000004},"min":{"1min":0.224,"5min":0.216,"15min":0.216},"max":{"1min":0.46100000000000002,"5min":12.378,"15min":12.378},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.58799999999999997,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.501,"15min":0.52000000000000002},"min":{"1min":0.26200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.54300000000000004,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59899999999999998,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.498,"15min":0.51800000000000002},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60199999999999998,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.58099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.38700000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.0920000000000001,"15min":1.169},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.97499999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83099999999999996,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93300000000000005,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.621,"5min":1.091,"15min":1.169},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0289999999999999,"5min":15.356,"15min":15.356},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.72899999999999998,"15min":0.75},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.83999999999999997,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.752,"15min":0.77400000000000002},"min":{"1min":0.44400000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":4.726,"15min":4.726},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.755,"15min":0.77600000000000002},"min":{"1min":0.45100000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.82799999999999996,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":1.129,"15min":1.2090000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":15.388,"15min":15.388},"last":0.65200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":374524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.309,"5min":0.32400000000000001,"15min":0.32600000000000001},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.49399999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33500000000000002,"15min":0.33700000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.49299999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.30599999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.25900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.38},{"interface":"front","average":{"1min":0.373,"5min":0.38900000000000001,"15min":0.39200000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.56299999999999994,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.28699999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.72899999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41899999999999998,"15min":0.42299999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.76300000000000001,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.438}]},{"osd":4,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.44,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0580000000000001,"5min":1.173,"15min":1.173},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99099999999999999,"5min":1.339,"15min":1.339},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.443,"5min":0.375,"15min":0.375},"max":{"1min":0.91100000000000003,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.92500000000000004,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.45600000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.018,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.45700000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.079,"5min":1.405,"15min":1.405},"last":0.621},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95399999999999996,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.70899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":373024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.33600000000000002,"15min":0.34100000000000003},"min":{"1min":0.23799999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.42299999999999999,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.32600000000000001,"5min":0.36599999999999999,"15min":0.372},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.39400000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.26400000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.432,"5min":1.032,"15min":1.032},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.379,"15min":0.38800000000000001},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.46899999999999997,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.32200000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.26700000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.47899999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.52100000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.46500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.90100000000000002,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.438,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.878,"5min":1.728,"15min":1.728},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61599999999999999,"15min":0.61399999999999999},"min":{"1min":0.48099999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88500000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.47799999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83299999999999996,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.37}]},{"osd":6,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.45800000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.161,"5min":1.222,"15min":1.222},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.499,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.41299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":372982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.621},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.504,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63300000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.39200000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72399999999999998,"5min":0.747,"15min":0.747},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.40300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54500000000000004,"15min":0.53700000000000003},"min":{"1min":0.438,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.52600000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73899999999999999,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.70299999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.53800000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.49399999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.40500000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40999999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.63700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":880640,"data_stored":373002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.42699999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.89800000000000002,"5min":3.335,"15min":3.335},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73199999999999998,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.71499999999999997,"15min":0.745},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81299999999999994,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77600000000000002,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.39100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72299999999999998,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76200000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.626}]},{"osd":3,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.71899999999999997,"15min":0.73999999999999999},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.80700000000000005,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79300000000000004,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.58499999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76200000000000001,"5min":1.536,"15min":1.536},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.59999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82899999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.63900000000000001,"15min":0.65000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77300000000000002,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:10:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64700000000000002,"15min":0.65500000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80900000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84399999999999997,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.75}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:11:40.688 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:11:40.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:11:40.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.848+0000 7f5346f59700 1 Processor -- start 2023-11-24T06:11:40.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.848+0000 7f5346f59700 1 -- start start 2023-11-24T06:11:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f5346f59700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f53400cf4e0 0x7f53400cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f5346f59700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f53400cfe80 0x7f534006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f5346f59700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534006de00 con 0x7f5340140540 2023-11-24T06:11:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f5346f59700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53400d0bb0 con 0x7f53400cf4e0 2023-11-24T06:11:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f5346f59700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53400d0310 con 0x7f53400cfe80 2023-11-24T06:11:40.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f533ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f53400cf4e0 0x7f53400cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f533ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f53400cf4e0 0x7f53400cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50152/0 (socket says 172.21.15.97:50152) 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f533ffff700 1 -- 172.21.15.97:0/323076609 learned_addr learned my addr 172.21.15.97:0/323076609 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f53454f6700 1 --2- 172.21.15.97:0/323076609 >> v2:172.21.15.183:3300/0 conn(0x7f53400cfe80 0x7f534006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f533ffff700 1 -- 172.21.15.97:0/323076609 >> v2:172.21.15.183:3300/0 conn(0x7f53400cfe80 msgr2=0x7f534006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f533ffff700 1 --2- 172.21.15.97:0/323076609 >> v2:172.21.15.183:3300/0 conn(0x7f53400cfe80 0x7f534006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f533ffff700 1 -- 172.21.15.97:0/323076609 >> v1:172.21.15.183:6789/0 conn(0x7f5340140540 legacy=0x7f5340140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.849+0000 7f533ffff700 1 -- 172.21.15.97:0/323076609 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53400d05f0 con 0x7f53400cf4e0 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f533ffff700 1 --2- 172.21.15.97:0/323076609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53400cf4e0 0x7f53400cf940 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f532c00bb90 tx=0x7f532c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6546b079f2bf2165 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f533f7fe700 1 -- 172.21.15.97:0/323076609 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00dd10 con 0x7f53400cf4e0 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f533f7fe700 1 -- 172.21.15.97:0/323076609 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c00de70 con 0x7f53400cf4e0 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f533f7fe700 1 -- 172.21.15.97:0/323076609 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c010670 con 0x7f53400cf4e0 2023-11-24T06:11:40.854 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f5346f59700 1 -- 172.21.15.97:0/323076609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53400cf4e0 msgr2=0x7f53400cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f5346f59700 1 --2- 172.21.15.97:0/323076609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53400cf4e0 0x7f53400cf940 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f532c00bb90 tx=0x7f532c00bea0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f5346f59700 1 -- 172.21.15.97:0/323076609 shutdown_connections 2023-11-24T06:11:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f5346f59700 1 --2- 172.21.15.97:0/323076609 >> v2:172.21.15.183:3300/0 conn(0x7f53400cfe80 0x7f534006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f5346f59700 1 --2- 172.21.15.97:0/323076609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53400cf4e0 0x7f53400cf940 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f5346f59700 1 -- 172.21.15.97:0/323076609 >> 172.21.15.97:0/323076609 conn(0x7f5340072960 msgr2=0x7f534006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f5346f59700 1 -- 172.21.15.97:0/323076609 shutdown_connections 2023-11-24T06:11:40.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.850+0000 7f5346f59700 1 -- 172.21.15.97:0/323076609 wait complete. 2023-11-24T06:11:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.851+0000 7f5346f59700 1 Processor -- start 2023-11-24T06:11:40.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.852+0000 7f5346f59700 1 -- start start 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.852+0000 7f5346f59700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53400cf4e0 0x7f5340139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.852+0000 7f5346f59700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5340140540 0x7f534013ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.852+0000 7f5346f59700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f534013f100 0x7f53400ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.852+0000 7f5346f59700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534013f5b0 con 0x7f5340140540 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.852+0000 7f5346f59700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53400b06a0 con 0x7f53400cf4e0 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.852+0000 7f5346f59700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534013b430 con 0x7f534013f100 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.852+0000 7f533ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5340140540 0x7f534013ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5340140540 0x7f534013ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50160/0 (socket says 172.21.15.97:50160) 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533ffff700 1 -- 172.21.15.97:0/3037088604 learned_addr learned my addr 172.21.15.97:0/3037088604 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f53454f6700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f534013f100 0x7f53400ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f5344cf5700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53400cf4e0 0x7f5340139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533ffff700 1 -- 172.21.15.97:0/3037088604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f534013f100 msgr2=0x7f53400ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533ffff700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f534013f100 0x7f53400ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533ffff700 1 -- 172.21.15.97:0/3037088604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53400cf4e0 msgr2=0x7f5340139650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533ffff700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53400cf4e0 0x7f5340139650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533ffff700 1 -- 172.21.15.97:0/3037088604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c00b7f0 con 0x7f5340140540 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533ffff700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5340140540 0x7f534013ebc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5340071490 tx=0x7f532c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533d7fa700 1 -- 172.21.15.97:0/3037088604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c01d070 con 0x7f5340140540 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533d7fa700 1 -- 172.21.15.97:0/3037088604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c010820 con 0x7f5340140540 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53400ae450 con 0x7f5340140540 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f533d7fa700 1 -- 172.21.15.97:0/3037088604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c010bd0 con 0x7f5340140540 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.853+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53400aea10 con 0x7f5340140540 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.854+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5340068fb0 con 0x7f5340140540 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.854+0000 7f533d7fa700 1 -- 172.21.15.97:0/3037088604 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f532c0036a0 con 0x7f5340140540 2023-11-24T06:11:40.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.855+0000 7f533d7fa700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f532406b2c0 0x7f532406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.855+0000 7f533d7fa700 1 -- 172.21.15.97:0/3037088604 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f532c08b400 con 0x7f5340140540 2023-11-24T06:11:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.855+0000 7f5344cf5700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f532406b2c0 0x7f532406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:40.860 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.855+0000 7f5344cf5700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f532406b2c0 0x7f532406d780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f53400b1c50 tx=0x7f5334009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:40.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:40.859+0000 7f533d7fa700 1 -- 172.21.15.97:0/3037088604 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f532c055e30 con 0x7f5340140540 2023-11-24T06:11:41.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.127+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f534013d390 con 0x7f5340140540 2023-11-24T06:11:41.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.129+0000 7f533d7fa700 1 -- 172.21.15.97:0/3037088604 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f532c0270b0 con 0x7f5340140540 2023-11-24T06:11:41.132 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-11-24T06:11:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA== 2023-11-24T06:11:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-11-24T06:11:41.132 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-11-24T06:11:41.133 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f532406b2c0 msgr2=0x7f532406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f532406b2c0 0x7f532406d780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f53400b1c50 tx=0x7f5334009040 comp rx=0 tx=0).stop 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5340140540 msgr2=0x7f534013ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5340140540 0x7f534013ebc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5340071490 tx=0x7f532c004750 comp rx=0 tx=0).stop 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 shutdown_connections 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f532406b2c0 0x7f532406d780 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f534013f100 0x7f53400ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53400cf4e0 0x7f5340139650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 --2- 172.21.15.97:0/3037088604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5340140540 0x7f534013ebc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 >> 172.21.15.97:0/3037088604 conn(0x7f5340072960 msgr2=0x7f534006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:41.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 shutdown_connections 2023-11-24T06:11:41.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.132+0000 7f5346f59700 1 -- 172.21.15.97:0/3037088604 wait complete. 2023-11-24T06:11:41.136 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-11-24T06:11:41.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.266+0000 7f07d2b4c700 1 Processor -- start 2023-11-24T06:11:41.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.266+0000 7f07d2b4c700 1 -- start start 2023-11-24T06:11:41.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.267+0000 7f07d2b4c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc0b3370 0x7f07cc1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.267+0000 7f07d2b4c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f07cc0d1c70 0x7f07cc0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.267+0000 7f07d2b4c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc0b22d0 con 0x7f07cc15bdd0 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.267+0000 7f07d2b4c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc0b2f50 con 0x7f07cc0d1c70 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.267+0000 7f07d2b4c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0d2640 con 0x7f07cc0b3370 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.267+0000 7f07cbfff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc0b3370 0x7f07cc1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.267+0000 7f07d10e9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f07cc0d1c70 0x7f07cc0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07d08e8700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f07cc15bdd0 0x7f07cc0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51940/0 (socket says 172.21.15.97:51940) 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07d10e9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f07cc0d1c70 0x7f07cc0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:44330/0 (socket says 172.21.15.97:44330) 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cbfff700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc0b3370 0x7f07cc1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:37202/0 (socket says 172.21.15.97:37202) 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07d08e8700 1 -- 172.21.15.97:0/1686280656 learned_addr learned my addr 172.21.15.97:0/1686280656 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cbfff700 1 -- 172.21.15.97:0/1686280656 >> v2:172.21.15.183:3300/0 conn(0x7f07cc0d1c70 msgr2=0x7f07cc0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:41.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cbfff700 1 --2- 172.21.15.97:0/1686280656 >> v2:172.21.15.183:3300/0 conn(0x7f07cc0d1c70 0x7f07cc0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cbfff700 1 -- 172.21.15.97:0/1686280656 >> v1:172.21.15.183:6789/0 conn(0x7f07cc15bdd0 legacy=0x7f07cc0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cb7fe700 1 -- 172.21.15.97:0/1686280656 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638620959 0 0) 0x7f07cc0b22d0 con 0x7f07cc15bdd0 2023-11-24T06:11:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cbfff700 1 -- 172.21.15.97:0/1686280656 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0b25b0 con 0x7f07cc0b3370 2023-11-24T06:11:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07d10e9700 1 --2- 172.21.15.97:0/1686280656 >> v2:172.21.15.183:3300/0 conn(0x7f07cc0d1c70 0x7f07cc0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:41.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cbfff700 1 --2- 172.21.15.97:0/1686280656 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc0b3370 0x7f07cc1604e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009840 tx=0x7f07b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e94a15365848850a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cb7fe700 1 -- 172.21.15.97:0/1686280656 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8011830 con 0x7f07cc0b3370 2023-11-24T06:11:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cb7fe700 1 -- 172.21.15.97:0/1686280656 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b8011990 con 0x7f07cc0b3370 2023-11-24T06:11:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.268+0000 7f07cb7fe700 1 -- 172.21.15.97:0/1686280656 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8011d20 con 0x7f07cc0b3370 2023-11-24T06:11:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.269+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1686280656 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc0b3370 msgr2=0x7f07cc1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.269+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1686280656 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc0b3370 0x7f07cc1604e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009840 tx=0x7f07b8009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.269+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1686280656 shutdown_connections 2023-11-24T06:11:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.269+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1686280656 >> v2:172.21.15.183:3300/0 conn(0x7f07cc0d1c70 0x7f07cc0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.269+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1686280656 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc0b3370 0x7f07cc1604e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.269+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1686280656 >> 172.21.15.97:0/1686280656 conn(0x7f07cc0731a0 msgr2=0x7f07cc09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.269+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1686280656 shutdown_connections 2023-11-24T06:11:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.269+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1686280656 wait complete. 2023-11-24T06:11:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.270+0000 7f07d2b4c700 1 Processor -- start 2023-11-24T06:11:41.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.270+0000 7f07d2b4c700 1 -- start start 2023-11-24T06:11:41.276 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07d2b4c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07cc0b3370 0x7f07cc150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07d2b4c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc0d1c70 0x7f07cc0c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07d2b4c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc15bdd0 0x7f07cc0c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07d2b4c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc15f1e0 con 0x7f07cc0d1c70 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07d2b4c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c5cf0 con 0x7f07cc0b3370 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07d2b4c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc15e8b0 con 0x7f07cc15bdd0 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07cbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc0d1c70 0x7f07cc0c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07cbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc0d1c70 0x7f07cc0c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50174/0 (socket says 172.21.15.97:50174) 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07cbfff700 1 -- 172.21.15.97:0/1585403099 learned_addr learned my addr 172.21.15.97:0/1585403099 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07d08e8700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07cc0b3370 0x7f07cc150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07d10e9700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc15bdd0 0x7f07cc0c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07cbfff700 1 -- 172.21.15.97:0/1585403099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc15bdd0 msgr2=0x7f07cc0c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07cbfff700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc15bdd0 0x7f07cc0c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07cbfff700 1 -- 172.21.15.97:0/1585403099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07cc0b3370 msgr2=0x7f07cc150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07cbfff700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07cc0b3370 0x7f07cc150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.271+0000 7f07cbfff700 1 -- 172.21.15.97:0/1585403099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c4009a70 con 0x7f07cc0d1c70 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.272+0000 7f07cbfff700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc0d1c70 0x7f07cc0c6040 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009fd0 tx=0x7f07b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.272+0000 7f07c97fa700 1 -- 172.21.15.97:0/1585403099 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b80045d0 con 0x7f07cc0d1c70 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.272+0000 7f07c97fa700 1 -- 172.21.15.97:0/1585403099 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b801f070 con 0x7f07cc0d1c70 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.272+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b80094a0 con 0x7f07cc0d1c70 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.272+0000 7f07c97fa700 1 -- 172.21.15.97:0/1585403099 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b80038a0 con 0x7f07cc0d1c70 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.272+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc15ee70 con 0x7f07cc0d1c70 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.273+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc0cda80 con 0x7f07cc0d1c70 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.273+0000 7f07c97fa700 1 -- 172.21.15.97:0/1585403099 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f07b8003a00 con 0x7f07cc0d1c70 2023-11-24T06:11:41.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.274+0000 7f07c97fa700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07b006b2c0 0x7f07b006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:41.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.274+0000 7f07d08e8700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07b006b2c0 0x7f07b006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:41.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.274+0000 7f07c97fa700 1 -- 172.21.15.97:0/1585403099 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f07b8021070 con 0x7f07cc0d1c70 2023-11-24T06:11:41.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.274+0000 7f07d08e8700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07b006b2c0 0x7f07b006d780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f07c0000f60 tx=0x7f07c0007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:41.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.278+0000 7f07c97fa700 1 -- 172.21.15.97:0/1585403099 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07b80563f0 con 0x7f07cc0d1c70 2023-11-24T06:11:41.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.509+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07cc067f60 con 0x7f07cc0d1c70 2023-11-24T06:11:41.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.509+0000 7f07c97fa700 1 -- 172.21.15.97:0/1585403099 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f07b8003cb0 con 0x7f07cc0d1c70 2023-11-24T06:11:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:41.512 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5863,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2765643379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2765643379},{"type":"v1","addr":"172.21.15.183:6834","nonce":2765643379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1367625334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367625334},{"type":"v1","addr":"172.21.15.97:6836","nonce":1367625334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/807372372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":807372372},{"type":"v1","addr":"172.21.15.97:6837","nonce":807372372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:35.581019+0000","modified":"2023-11-24T06:11:38.607438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3452869588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3452869588},{"type":"v1","addr":"172.21.15.183:6835","nonce":3452869588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:11:41.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07b006b2c0 msgr2=0x7f07b006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07b006b2c0 0x7f07b006d780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f07c0000f60 tx=0x7f07c0007e70 comp rx=0 tx=0).stop 2023-11-24T06:11:41.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc0d1c70 msgr2=0x7f07cc0c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc0d1c70 0x7f07cc0c6040 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009fd0 tx=0x7f07b8004750 comp rx=0 tx=0).stop 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 shutdown_connections 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f07b006b2c0 0x7f07b006d780 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07cc15bdd0 0x7f07cc0c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07cc0b3370 0x7f07cc150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 --2- 172.21.15.97:0/1585403099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07cc0d1c70 0x7f07cc0c6040 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 >> 172.21.15.97:0/1585403099 conn(0x7f07cc0731a0 msgr2=0x7f07cc0cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 shutdown_connections 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:41.511+0000 7f07d2b4c700 1 -- 172.21.15.97:0/1585403099 wait complete. 2023-11-24T06:11:41.515 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 46 2023-11-24T06:11:41.929 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:11:35.581019+0000', 'modified': '2023-11-24T06:11:38.607438+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6474}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6474': {'gid': 6474, 'name': 'b', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6835/3452869588', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6833', 'nonce': 3452869588}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 3452869588}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:11:41.929 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:11:41.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-24T06:11:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.049+0000 7f0520eab700 1 Processor -- start 2023-11-24T06:11:42.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.049+0000 7f0520eab700 1 -- start start 2023-11-24T06:11:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.050+0000 7f0520eab700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f051c0b04b0 0x7f051c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.050+0000 7f0520eab700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0af1e0 0x7f051c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.050+0000 7f0520eab700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051c15c540 con 0x7f051c15c160 2023-11-24T06:11:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.050+0000 7f0520eab700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f051c0b2ee0 con 0x7f051c0b04b0 2023-11-24T06:11:42.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.050+0000 7f0520eab700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0add40 con 0x7f051c0af1e0 2023-11-24T06:11:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f0519d9b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f051c0b04b0 0x7f051c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051ad9d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0af1e0 0x7f051c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f0519d9b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f051c0b04b0 0x7f051c0622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:44340/0 (socket says 172.21.15.97:44340) 2023-11-24T06:11:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051a59c700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f051c15c160 0x7f051c0afda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:51954/0 (socket says 172.21.15.97:51954) 2023-11-24T06:11:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051ad9d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0af1e0 0x7f051c0b2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:37216/0 (socket says 172.21.15.97:37216) 2023-11-24T06:11:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f0519d9b700 1 -- 172.21.15.97:0/2504936524 learned_addr learned my addr 172.21.15.97:0/2504936524 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:42.055 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051ad9d700 1 -- 172.21.15.97:0/2504936524 >> v2:172.21.15.183:3300/0 conn(0x7f051c0b04b0 msgr2=0x7f051c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051ad9d700 1 --2- 172.21.15.97:0/2504936524 >> v2:172.21.15.183:3300/0 conn(0x7f051c0b04b0 0x7f051c0622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051ad9d700 1 -- 172.21.15.97:0/2504936524 >> v1:172.21.15.183:6789/0 conn(0x7f051c15c160 legacy=0x7f051c0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051ad9d700 1 -- 172.21.15.97:0/2504936524 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c0ad410 con 0x7f051c0af1e0 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f0519d9b700 1 --2- 172.21.15.97:0/2504936524 >> v2:172.21.15.183:3300/0 conn(0x7f051c0b04b0 0x7f051c0622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051ad9d700 1 --2- 172.21.15.97:0/2504936524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0af1e0 0x7f051c0b2870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0514009d90 tx=0x7f051400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7da880a78e834f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051959a700 1 -- 172.21.15.97:0/2504936524 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400e070 con 0x7f051c0af1e0 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051959a700 1 -- 172.21.15.97:0/2504936524 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051400bb20 con 0x7f051c0af1e0 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.051+0000 7f051959a700 1 -- 172.21.15.97:0/2504936524 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514012430 con 0x7f051c0af1e0 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.052+0000 7f0520eab700 1 -- 172.21.15.97:0/2504936524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0af1e0 msgr2=0x7f051c0b2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.052+0000 7f0520eab700 1 --2- 172.21.15.97:0/2504936524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0af1e0 0x7f051c0b2870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0514009d90 tx=0x7f051400b330 comp rx=0 tx=0).stop 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.052+0000 7f0520eab700 1 -- 172.21.15.97:0/2504936524 shutdown_connections 2023-11-24T06:11:42.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.052+0000 7f0520eab700 1 --2- 172.21.15.97:0/2504936524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0af1e0 0x7f051c0b2870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.052+0000 7f0520eab700 1 --2- 172.21.15.97:0/2504936524 >> v2:172.21.15.183:3300/0 conn(0x7f051c0b04b0 0x7f051c0622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.052+0000 7f0520eab700 1 -- 172.21.15.97:0/2504936524 >> 172.21.15.97:0/2504936524 conn(0x7f051c0b57b0 msgr2=0x7f051c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.052+0000 7f0520eab700 1 -- 172.21.15.97:0/2504936524 shutdown_connections 2023-11-24T06:11:42.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.052+0000 7f0520eab700 1 -- 172.21.15.97:0/2504936524 wait complete. 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.053+0000 7f0520eab700 1 Processor -- start 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.053+0000 7f0520eab700 1 -- start start 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f0520eab700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0af1e0 0x7f051c150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f0520eab700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0b04b0 0x7f051c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f0520eab700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c15c160 0x7f051c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f0520eab700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0cdab0 con 0x7f051c15c160 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f0520eab700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0cc0c0 con 0x7f051c0af1e0 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f0520eab700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c0cb6f0 con 0x7f051c0b04b0 2023-11-24T06:11:42.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f051ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c15c160 0x7f051c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f051ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c15c160 0x7f051c0cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50188/0 (socket says 172.21.15.97:50188) 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f051ad9d700 1 -- 172.21.15.97:0/60796590 learned_addr learned my addr 172.21.15.97:0/60796590 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f051a59c700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0af1e0 0x7f051c150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f0519d9b700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0b04b0 0x7f051c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.054+0000 7f051ad9d700 1 -- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0b04b0 msgr2=0x7f051c0cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f051ad9d700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0b04b0 0x7f051c0cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f051ad9d700 1 -- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0af1e0 msgr2=0x7f051c150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f051ad9d700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0af1e0 0x7f051c150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f051ad9d700 1 -- 172.21.15.97:0/60796590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c009550 con 0x7f051c15c160 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f0519d9b700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0b04b0 0x7f051c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f051ad9d700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c15c160 0x7f051c0cbb80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0514014040 tx=0x7f0514004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f050affd700 1 -- 172.21.15.97:0/60796590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514012430 con 0x7f051c15c160 2023-11-24T06:11:42.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f050affd700 1 -- 172.21.15.97:0/60796590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0514020070 con 0x7f051c15c160 2023-11-24T06:11:42.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f0520eab700 1 -- 172.21.15.97:0/60796590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0514009a70 con 0x7f051c15c160 2023-11-24T06:11:42.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f050affd700 1 -- 172.21.15.97:0/60796590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514003ea0 con 0x7f051c15c160 2023-11-24T06:11:42.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.055+0000 7f0520eab700 1 -- 172.21.15.97:0/60796590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051c0cb0a0 con 0x7f051c15c160 2023-11-24T06:11:42.061 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.056+0000 7f050affd700 1 -- 172.21.15.97:0/60796590 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f0514004280 con 0x7f051c15c160 2023-11-24T06:11:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.057+0000 7f050affd700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f04fc06b2c0 0x7f04fc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.057+0000 7f050affd700 1 -- 172.21.15.97:0/60796590 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0514022070 con 0x7f051c15c160 2023-11-24T06:11:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.057+0000 7f051a59c700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f04fc06b2c0 0x7f04fc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.057+0000 7f051a59c700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f04fc06b2c0 0x7f04fc06d780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0510000f60 tx=0x7f0510007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.059+0000 7f052d0a4b80 1 Processor -- start 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.059+0000 7f052d0a4b80 1 -- start start 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.059+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc188080 0x55d8cc1ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc185490 0x55d8cc1b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc0fb770 con 0x55d8cc19ded0 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc0e2750 con 0x55d8cc188080 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc0e0070 con 0x55d8cc185490 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50196/0 (socket says 172.21.15.97:50196) 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 -- 172.21.15.97:0/2869778572 learned_addr learned my addr 172.21.15.97:0/2869778572 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f051a59c700 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc188080 0x55d8cc1ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f051ad9d700 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc185490 0x55d8cc1b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 -- 172.21.15.97:0/2869778572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc185490 msgr2=0x55d8cc1b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc185490 0x55d8cc1b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 -- 172.21.15.97:0/2869778572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc188080 msgr2=0x55d8cc1ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc188080 0x55d8cc1ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 -- 172.21.15.97:0/2869778572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cbf585e0 con 0x55d8cc19ded0 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f0519d9b700 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1ced80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f050c009dc0 tx=0x7f050c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e08ed4aa0030dad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f04fbfff700 1 -- 172.21.15.97:0/2869778572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c011ce0 con 0x55d8cc19ded0 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f04fbfff700 1 -- 172.21.15.97:0/2869778572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c011e40 con 0x55d8cc19ded0 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.060+0000 7f04fbfff700 1 -- 172.21.15.97:0/2869778572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c015660 con 0x55d8cc19ded0 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2869778572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 msgr2=0x55d8cc1ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1ced80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f050c009dc0 tx=0x7f050c0092c0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2869778572 shutdown_connections 2023-11-24T06:11:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc185490 0x55d8cc1b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc188080 0x55d8cc1ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2869778572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1ced80 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2869778572 >> 172.21.15.97:0/2869778572 conn(0x55d8cc0b6310 msgr2=0x55d8cc1cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2869778572 shutdown_connections 2023-11-24T06:11:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.061+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2869778572 wait complete. 2023-11-24T06:11:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.062+0000 7f052d0a4b80 1 -- start start 2023-11-24T06:11:42.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.062+0000 7f052d0a4b80 1 Processor -- start 2023-11-24T06:11:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc19d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc0db960 con 0x55d8cc185490 2023-11-24T06:11:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc095510 con 0x55d8cc19ded0 2023-11-24T06:11:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc0951b0 con 0x55d8cc188080 2023-11-24T06:11:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 10 client.0 _ll_register_callbacks cb 0x55d8cbfc0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:11:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50200/0 (socket says 172.21.15.97:50200) 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 -- 172.21.15.97:0/2700775079 learned_addr learned my addr 172.21.15.97:0/2700775079 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051ad9d700 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f0519d9b700 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 -- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 msgr2=0x55d8cc19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 -- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 msgr2=0x55d8cc1a2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1a2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 -- 172.21.15.97:0/2700775079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050c0099f0 con 0x55d8cc185490 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f051a59c700 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1a03d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0510001e70 tx=0x7f0510000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ba4d997191bded server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f04f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:11:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f04f97fa700 1 -- 172.21.15.97:0/2700775079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510013b40 con 0x55d8cc185490 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f04f97fa700 1 -- 172.21.15.97:0/2700775079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f051001b070 con 0x55d8cc185490 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2700775079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8cc17aa40 con 0x55d8cc185490 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f04f97fa700 1 -- 172.21.15.97:0/2700775079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0510013ca0 con 0x55d8cc185490 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 10 client.6591 fetch_fsmap learned FSMap version 46 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.063+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2700775079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55d8cbf04d90 con 0x55d8cc185490 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f04f97fa700 1 -- 172.21.15.97:0/2700775079 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d8cc17aa40 con 0x55d8cc185490 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f04f97fa700 1 -- 172.21.15.97:0/2700775079 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f051002c3b0 con 0x55d8cc185490 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f052d0a4b80 10 client.6591 fetch_fsmap finished waiting for FSMap version 46 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f052d0a4b80 10 client.6591 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f052d0a4b80 20 client.6591 populate_metadata read hostname 'smithi097' 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] conn(0x55d8cc17c690 0x55d8cc1b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f052d0a4b80 4 client.6591 mds_command: new command op to 6474 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:11:42.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2700775079 --> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8cc156840 con 0x55d8cc17c690 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.064+0000 7f0519d9b700 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] conn(0x55d8cc17c690 0x55d8cc1b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.065+0000 7f04f97fa700 10 client.6591 ms_handle_connect on v2:172.21.15.183:6833/3452869588 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.065+0000 7f0519d9b700 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] conn(0x55d8cc17c690 0x55d8cc1b5330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a6b7eeaa540073 server_cookie=9e172cb30e953418 in_seq=0 out_seq=0 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.065+0000 7f04f97fa700 1 -- 172.21.15.97:0/2700775079 <== mds.0 v2:172.21.15.183:6833/3452869588 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8cc156840 con 0x55d8cc17c690 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.065+0000 7f04f97fa700 10 client.6591 handle_command_reply: tid=0 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.065+0000 7f052d0a4b80 1 client.6591 shutdown 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.065+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2700775079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 msgr2=0x55d8cc1a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1a03d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0510001e70 tx=0x7f0510000ed0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2700775079 shutdown_connections 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] conn(0x55d8cc17c690 0x55d8cc1b5330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1a2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1a03d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2700775079 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f04f97fa700 0 client.6591 ms_handle_reset on v2:172.21.15.183:6833/3452869588 2023-11-24T06:11:42.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2700775079 >> 172.21.15.97:0/2700775079 conn(0x55d8cc0b6310 msgr2=0x55d8cc1d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2700775079 shutdown_connections 2023-11-24T06:11:42.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2700775079 wait complete. 2023-11-24T06:11:42.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.066+0000 7f052d0a4b80 20 client.6591 trim_cache size 0 max 16384 2023-11-24T06:11:42.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.084+0000 7f052d0a4b80 1 Processor -- start 2023-11-24T06:11:42.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.084+0000 7f052d0a4b80 1 -- start start 2023-11-24T06:11:42.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc1c2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cbf5a8d0 con 0x55d8cc185490 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc183be0 con 0x55d8cc19ded0 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc164350 con 0x55d8cc188080 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50208/0 (socket says 172.21.15.97:50208) 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 -- 172.21.15.97:0/1543943087 learned_addr learned my addr 172.21.15.97:0/1543943087 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051ad9d700 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f0519d9b700 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc1c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 -- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 msgr2=0x55d8cc1c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc1c2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 -- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 msgr2=0x55d8cc1c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 -- 172.21.15.97:0/1543943087 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cbeedc10 con 0x55d8cc185490 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051ad9d700 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.085+0000 7f051a59c700 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1cf890 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f05100013e0 tx=0x7f0510020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b82a438edcd5368 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f0508ff9700 1 -- 172.21.15.97:0/1543943087 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510007b30 con 0x55d8cc185490 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f0508ff9700 1 -- 172.21.15.97:0/1543943087 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051000b040 con 0x55d8cc185490 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f0508ff9700 1 -- 172.21.15.97:0/1543943087 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051002c3b0 con 0x55d8cc185490 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 -- 172.21.15.97:0/1543943087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 msgr2=0x55d8cc1cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1cf890 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f05100013e0 tx=0x7f0510020000 comp rx=0 tx=0).stop 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 -- 172.21.15.97:0/1543943087 shutdown_connections 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc19ded0 0x55d8cc1c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc185490 0x55d8cc1cf890 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/1543943087 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc1c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 -- 172.21.15.97:0/1543943087 >> 172.21.15.97:0/1543943087 conn(0x55d8cc0b6310 msgr2=0x55d8cc1ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 -- 172.21.15.97:0/1543943087 shutdown_connections 2023-11-24T06:11:42.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.086+0000 7f052d0a4b80 1 -- 172.21.15.97:0/1543943087 wait complete. 2023-11-24T06:11:42.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.087+0000 7f052d0a4b80 1 -- start start 2023-11-24T06:11:42.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.087+0000 7f052d0a4b80 1 Processor -- start 2023-11-24T06:11:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc185490 0x55d8cc1cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc1b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f052d0a4b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1e55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc1a2d50 con 0x55d8cc19ded0 2023-11-24T06:11:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc182b90 con 0x55d8cc185490 2023-11-24T06:11:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f052d0a4b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d8cc1cddd0 con 0x55d8cc188080 2023-11-24T06:11:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f052d0a4b80 10 client.0 _ll_register_callbacks cb 0x55d8cbfc0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:11:42.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1e55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1e55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50218/0 (socket says 172.21.15.97:50218) 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 -- 172.21.15.97:0/2463971197 learned_addr learned my addr 172.21.15.97:0/2463971197 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051a59c700 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc185490 0x55d8cc1cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f0519d9b700 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc1b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 -- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 msgr2=0x55d8cc1b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc1b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 -- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc185490 msgr2=0x55d8cc1cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc185490 0x55d8cc1cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 -- 172.21.15.97:0/2463971197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8cbf04d90 con 0x55d8cc19ded0 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f051ad9d700 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1e55c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0514056890 tx=0x7f051405e400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91224e5c02061b02 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f04f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:11:42.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f04f97fa700 1 -- 172.21.15.97:0/2463971197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051406f070 con 0x55d8cc19ded0 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f04f97fa700 1 -- 172.21.15.97:0/2463971197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f051405e530 con 0x55d8cc19ded0 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.088+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2463971197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f050c011e40 con 0x55d8cc19ded0 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f04f97fa700 1 -- 172.21.15.97:0/2463971197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05140626e0 con 0x55d8cc19ded0 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f052d0a4b80 10 client.6597 fetch_fsmap learned FSMap version 46 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2463971197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x55d8cbf2b200 con 0x55d8cc19ded0 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f04f97fa700 1 -- 172.21.15.97:0/2463971197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c011e40 con 0x55d8cc19ded0 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f04f97fa700 1 -- 172.21.15.97:0/2463971197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 46) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0514062250 con 0x55d8cc19ded0 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f052d0a4b80 10 client.6597 fetch_fsmap finished waiting for FSMap version 46 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f052d0a4b80 10 client.6597 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f052d0a4b80 20 client.6597 populate_metadata read hostname 'smithi097' 2023-11-24T06:11:42.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] conn(0x55d8cc1c9400 0x55d8cc1c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f052d0a4b80 4 client.6597 mds_command: new command op to 6474 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2463971197 --> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d8cc0a1f80 con 0x55d8cc1c9400 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.089+0000 7f051a59c700 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] conn(0x55d8cc1c9400 0x55d8cc1c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f04f97fa700 10 client.6597 ms_handle_connect on v2:172.21.15.183:6833/3452869588 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f051a59c700 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] conn(0x55d8cc1c9400 0x55d8cc1c97e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d09d0224a5af2b server_cookie=97fc3d30023ba751 in_seq=0 out_seq=0 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f04f97fa700 1 -- 172.21.15.97:0/2463971197 <== mds.0 v2:172.21.15.183:6833/3452869588 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d8cc0a1f80 con 0x55d8cc1c9400 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f04f97fa700 10 client.6597 handle_command_reply: tid=0 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 client.6597 shutdown 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2463971197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 msgr2=0x55d8cc1e55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1e55c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0514056890 tx=0x7f051405e400 comp rx=0 tx=0).stop 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2463971197 shutdown_connections 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:6833/3452869588,v1:172.21.15.183:6835/3452869588] conn(0x55d8cc1c9400 0x55d8cc1c97e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8cc19ded0 0x55d8cc1e55c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8cc185490 0x55d8cc1cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 --2- 172.21.15.97:0/2463971197 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55d8cc188080 0x55d8cc1b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f04f97fa700 0 client.6597 ms_handle_reset on v2:172.21.15.183:6833/3452869588 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2463971197 >> 172.21.15.97:0/2463971197 conn(0x55d8cc0b6310 msgr2=0x55d8cc1ccc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2463971197 shutdown_connections 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 1 -- 172.21.15.97:0/2463971197 wait complete. 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.090+0000 7f052d0a4b80 20 client.6597 trim_cache size 0 max 16384 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:11:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9", 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6474, 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.9079332290000002, 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 46, 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 72, 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 72, 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 10.838028114 2023-11-24T06:11:42.101 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:11:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 -- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f04fc06b2c0 msgr2=0x7f04fc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f04fc06b2c0 0x7f04fc06d780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0510000f60 tx=0x7f0510007ed0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 -- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c15c160 msgr2=0x7f051c0cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c15c160 0x7f051c0cbb80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0514014040 tx=0x7f0514004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 -- 172.21.15.97:0/60796590 shutdown_connections 2023-11-24T06:11:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f051c15c160 0x7f051c0cbb80 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f051c0af1e0 0x7f051c150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f04fc06b2c0 0x7f04fc06d780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 --2- 172.21.15.97:0/60796590 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f051c0b04b0 0x7f051c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 -- 172.21.15.97:0/60796590 >> 172.21.15.97:0/60796590 conn(0x7f051c0b57b0 msgr2=0x7f051c06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 -- 172.21.15.97:0/60796590 shutdown_connections 2023-11-24T06:11:42.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.091+0000 7f0520eab700 1 -- 172.21.15.97:0/60796590 wait complete. 2023-11-24T06:11:42.105 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:11:42.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.222+0000 7f412bf81700 1 Processor -- start 2023-11-24T06:11:42.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.223+0000 7f412bf81700 1 -- start start 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.223+0000 7f412bf81700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f412406c710 0x7f41240b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.223+0000 7f412bf81700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41240ae430 con 0x7f41240af4d0 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.223+0000 7f412bf81700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41240ae710 con 0x7f412415e080 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.223+0000 7f412bf81700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41240af0b0 con 0x7f412406c710 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f412951c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f412406c710 0x7f41240b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f412a51e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f41240af4d0 0x7f412406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42678/0 (socket says 172.21.15.97:42678) 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f412951c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f412406c710 0x7f41240b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50224/0 (socket says 172.21.15.97:50224) 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f412a51e700 1 -- 172.21.15.97:0/277247100 learned_addr learned my addr 172.21.15.97:0/277247100 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:42.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f412951c700 1 -- 172.21.15.97:0/277247100 >> v1:172.21.15.183:6789/0 conn(0x7f412415e080 legacy=0x7f412406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:42.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f412951c700 1 -- 172.21.15.97:0/277247100 >> v1:172.21.15.97:6789/0 conn(0x7f41240af4d0 legacy=0x7f412406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f412951c700 1 -- 172.21.15.97:0/277247100 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41240ae9f0 con 0x7f412406c710 2023-11-24T06:11:42.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f4128d1b700 1 -- 172.21.15.97:0/277247100 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542219502 0 0) 0x7f41240ae430 con 0x7f41240af4d0 2023-11-24T06:11:42.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f412951c700 1 --2- 172.21.15.97:0/277247100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412406c710 0x7f41240b5050 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4120009840 tx=0x7f4120009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6731be5548c835aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f4128d1b700 1 -- 172.21.15.97:0/277247100 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120004230 con 0x7f412406c710 2023-11-24T06:11:42.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f4128d1b700 1 -- 172.21.15.97:0/277247100 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120004390 con 0x7f412406c710 2023-11-24T06:11:42.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.224+0000 7f4128d1b700 1 -- 172.21.15.97:0/277247100 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41200036a0 con 0x7f412406c710 2023-11-24T06:11:42.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.225+0000 7f412bf81700 1 -- 172.21.15.97:0/277247100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412406c710 msgr2=0x7f41240b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.225+0000 7f412bf81700 1 --2- 172.21.15.97:0/277247100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412406c710 0x7f41240b5050 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4120009840 tx=0x7f4120009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:42.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.225+0000 7f412bf81700 1 -- 172.21.15.97:0/277247100 shutdown_connections 2023-11-24T06:11:42.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.225+0000 7f412bf81700 1 --2- 172.21.15.97:0/277247100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f412406c710 0x7f41240b5050 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.225+0000 7f412bf81700 1 -- 172.21.15.97:0/277247100 >> 172.21.15.97:0/277247100 conn(0x7f4124071b90 msgr2=0x7f4124071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.229 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.225+0000 7f412bf81700 1 -- 172.21.15.97:0/277247100 shutdown_connections 2023-11-24T06:11:42.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.225+0000 7f412bf81700 1 -- 172.21.15.97:0/277247100 wait complete. 2023-11-24T06:11:42.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.226+0000 7f412bf81700 1 Processor -- start 2023-11-24T06:11:42.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.226+0000 7f412bf81700 1 -- start start 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412bf81700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f412406c710 0x7f412415e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412bf81700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41240af4d0 0x7f4124155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412bf81700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f412415e080 0x7f4124155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412bf81700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4124155dd0 con 0x7f41240af4d0 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412bf81700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41240cfb50 con 0x7f412406c710 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412bf81700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412415eaf0 con 0x7f412415e080 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412951c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41240af4d0 0x7f4124155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412951c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41240af4d0 0x7f4124155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50230/0 (socket says 172.21.15.97:50230) 2023-11-24T06:11:42.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412951c700 1 -- 172.21.15.97:0/1218768433 learned_addr learned my addr 172.21.15.97:0/1218768433 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f4129d1d700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f412406c710 0x7f412415e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412a51e700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f412415e080 0x7f4124155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412951c700 1 -- 172.21.15.97:0/1218768433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f412415e080 msgr2=0x7f4124155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412951c700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f412415e080 0x7f4124155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412951c700 1 -- 172.21.15.97:0/1218768433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f412406c710 msgr2=0x7f412415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412951c700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f412406c710 0x7f412415e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.227+0000 7f412951c700 1 -- 172.21.15.97:0/1218768433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41200094a0 con 0x7f41240af4d0 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.228+0000 7f412951c700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41240af4d0 0x7f4124155350 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f41200090b0 tx=0x7f4120004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.229+0000 7f411a7fc700 1 -- 172.21.15.97:0/1218768433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120003ea0 con 0x7f41240af4d0 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.229+0000 7f411a7fc700 1 -- 172.21.15.97:0/1218768433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412001e070 con 0x7f41240af4d0 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.229+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41240cffe0 con 0x7f41240af4d0 2023-11-24T06:11:42.233 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.229+0000 7f411a7fc700 1 -- 172.21.15.97:0/1218768433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120011850 con 0x7f41240af4d0 2023-11-24T06:11:42.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.229+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41240cf500 con 0x7f41240af4d0 2023-11-24T06:11:42.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.230+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41240753f0 con 0x7f41240af4d0 2023-11-24T06:11:42.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.230+0000 7f411a7fc700 1 -- 172.21.15.97:0/1218768433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f41200119b0 con 0x7f41240af4d0 2023-11-24T06:11:42.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.231+0000 7f411a7fc700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410806b2c0 0x7f410806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:42.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.231+0000 7f411a7fc700 1 -- 172.21.15.97:0/1218768433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f412008ae10 con 0x7f41240af4d0 2023-11-24T06:11:42.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.231+0000 7f4129d1d700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410806b2c0 0x7f410806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:42.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.231+0000 7f4129d1d700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410806b2c0 0x7f410806d780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4114009960 tx=0x7f41140090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:42.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.235+0000 7f411a7fc700 1 -- 172.21.15.97:0/1218768433 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f412008e030 con 0x7f41240af4d0 2023-11-24T06:11:42.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.470+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4124067f60 con 0x7f41240af4d0 2023-11-24T06:11:42.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.470+0000 7f411a7fc700 1 -- 172.21.15.97:0/1218768433 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4120020020 con 0x7f41240af4d0 2023-11-24T06:11:42.473 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:11:42.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.472+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410806b2c0 msgr2=0x7f410806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.472+0000 7f412bf81700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410806b2c0 0x7f410806d780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4114009960 tx=0x7f41140090e0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.472+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41240af4d0 msgr2=0x7f4124155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.472+0000 7f412bf81700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41240af4d0 0x7f4124155350 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f41200090b0 tx=0x7f4120004750 comp rx=0 tx=0).stop 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.473+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 shutdown_connections 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.473+0000 7f412bf81700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f410806b2c0 0x7f410806d780 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.473+0000 7f412bf81700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f412415e080 0x7f4124155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.473+0000 7f412bf81700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f412406c710 0x7f412415e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.473+0000 7f412bf81700 1 --2- 172.21.15.97:0/1218768433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41240af4d0 0x7f4124155350 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.473+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 >> 172.21.15.97:0/1218768433 conn(0x7f4124071b90 msgr2=0x7f41240b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.473+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 shutdown_connections 2023-11-24T06:11:42.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:42.473+0000 7f412bf81700 1 -- 172.21.15.97:0/1218768433 wait complete. 2023-11-24T06:11:42.487 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:11:42.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:11:42.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-24T06:11:42.487 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:11:42.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:42.488 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:11:42.488 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:11:42.488 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:11:42.488 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:42.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:42.488 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-24T06:11:42.505 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:11:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:11:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42034sec preferred_lft 42034sec 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c9d4:801c:12b3:b866/64 scope link noprefixroute 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::7c25:b1ff:fe16:36f1/64 scope link 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether a2:28:0d:2d:82:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:11:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a028:dff:fe2d:822b/64 scope link 2023-11-24T06:11:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:42.508 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:11:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::1875:dff:fec0:4af7/64 scope link 2023-11-24T06:11:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:42.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:42.508 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:11:42.570 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-24T06:11:42.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:42.572 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-24T06:11:42.629 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-24T06:11:42.629 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:11:42.629 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:11:42.629 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:11:42.629 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:11:42.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:42.630 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:11:42.630 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:11:42.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:11:42.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:11:42.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:11:42.630 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:11:42.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:42.733 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:11:42.749 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:11:42.749 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:11:42.750 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:42.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:42.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-24T06:11:42.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:42.816 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:42.872 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:11:42.873 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:42.874 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:42.929 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:11:42.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:42.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:43.014 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:43.014 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:43.037 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:43.038 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:43.038 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:11:43.038 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:11:43.038 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:11:43.038 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:11:43.038 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:11:43.038 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:11:43.072 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:43.072 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:43.072 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:11:43.072 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:11:43.072 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:11:43.073 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:11:43.073 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:11:43.073 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:11:43.073 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:11:43.073 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:11:43.073 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:11:43.075 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:11:43.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:43.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:43.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:43.110 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:11:43.198 INFO:teuthology.orchestra.run.smithi097.stdout:client6609 2023-11-24T06:11:43.198 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:43.198 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:43.215 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:11:43.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.373+0000 7f26bf11b700 1 Processor -- start 2023-11-24T06:11:43.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.374+0000 7f26bf11b700 1 -- start start 2023-11-24T06:11:43.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.374+0000 7f26bf11b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b80b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.374+0000 7f26bf11b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26b80af4d0 0x7f26b80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.374+0000 7f26bf11b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b80ae430 con 0x7f26b815e080 2023-11-24T06:11:43.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.374+0000 7f26bf11b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b80af0b0 con 0x7f26b80af4d0 2023-11-24T06:11:43.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.374+0000 7f26bf11b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80b05a0 con 0x7f26b806c710 2023-11-24T06:11:43.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.374+0000 7f26bd6b8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26b80af4d0 0x7f26b80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26bd6b8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f26b80af4d0 0x7f26b80afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50244/0 (socket says 172.21.15.97:50244) 2023-11-24T06:11:43.379 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26bd6b8700 1 -- 172.21.15.97:0/2492308217 learned_addr learned my addr 172.21.15.97:0/2492308217 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26b7fff700 1 --2- 172.21.15.97:0/2492308217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b80b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26bd6b8700 1 -- 172.21.15.97:0/2492308217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 msgr2=0x7f26b80b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26bd6b8700 1 --2- 172.21.15.97:0/2492308217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b80b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26bd6b8700 1 -- 172.21.15.97:0/2492308217 >> v1:172.21.15.183:6789/0 conn(0x7f26b815e080 legacy=0x7f26b806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26bd6b8700 1 -- 172.21.15.97:0/2492308217 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80ae710 con 0x7f26b80af4d0 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26b7fff700 1 --2- 172.21.15.97:0/2492308217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26bd6b8700 1 --2- 172.21.15.97:0/2492308217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b80afb90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009e80 tx=0x7f26b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe1e28f8e3cf90c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26b77fe700 1 -- 172.21.15.97:0/2492308217 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b000e070 con 0x7f26b80af4d0 2023-11-24T06:11:43.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26b77fe700 1 -- 172.21.15.97:0/2492308217 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b0003a20 con 0x7f26b80af4d0 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.375+0000 7f26b77fe700 1 -- 172.21.15.97:0/2492308217 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0003dd0 con 0x7f26b80af4d0 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.376+0000 7f26bf11b700 1 -- 172.21.15.97:0/2492308217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 msgr2=0x7f26b80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.376+0000 7f26bf11b700 1 --2- 172.21.15.97:0/2492308217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b80afb90 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009e80 tx=0x7f26b000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.376+0000 7f26bf11b700 1 -- 172.21.15.97:0/2492308217 shutdown_connections 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.376+0000 7f26bf11b700 1 --2- 172.21.15.97:0/2492308217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b80afb90 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.376+0000 7f26bf11b700 1 --2- 172.21.15.97:0/2492308217 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b80b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.376+0000 7f26bf11b700 1 -- 172.21.15.97:0/2492308217 >> 172.21.15.97:0/2492308217 conn(0x7f26b8071b90 msgr2=0x7f26b8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.376+0000 7f26bf11b700 1 -- 172.21.15.97:0/2492308217 shutdown_connections 2023-11-24T06:11:43.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.376+0000 7f26bf11b700 1 -- 172.21.15.97:0/2492308217 wait complete. 2023-11-24T06:11:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.377+0000 7f26bf11b700 1 Processor -- start 2023-11-24T06:11:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.377+0000 7f26bf11b700 1 -- start start 2023-11-24T06:11:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26bf11b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b8158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26bf11b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b8160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26bf11b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b815e080 0x7f26b81f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26bf11b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8145240 con 0x7f26b80af4d0 2023-11-24T06:11:43.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26bf11b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80d0870 con 0x7f26b815e080 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26bf11b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8161390 con 0x7f26b806c710 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b8160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b8160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50248/0 (socket says 172.21.15.97:50248) 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 -- 172.21.15.97:0/8493825 learned_addr learned my addr 172.21.15.97:0/8493825 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26bceb7700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b8158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26bd6b8700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b815e080 0x7f26b81f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 -- 172.21.15.97:0/8493825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 msgr2=0x7f26b8158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b8158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 -- 172.21.15.97:0/8493825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b815e080 msgr2=0x7f26b81f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b815e080 0x7f26b81f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 -- 172.21.15.97:0/8493825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b0009b30 con 0x7f26b80af4d0 2023-11-24T06:11:43.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.378+0000 7f26b7fff700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b8160e50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f26a400c750 tx=0x7f26a400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.379+0000 7f26b57fa700 1 -- 172.21.15.97:0/8493825 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4011800 con 0x7f26b80af4d0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.379+0000 7f26b57fa700 1 -- 172.21.15.97:0/8493825 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a4011960 con 0x7f26b80af4d0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.379+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b80d0590 con 0x7f26b80af4d0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.379+0000 7f26b57fa700 1 -- 172.21.15.97:0/8493825 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4011cd0 con 0x7f26b80af4d0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.379+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b8158310 con 0x7f26b80af4d0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.379+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b80753f0 con 0x7f26b80af4d0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.380+0000 7f26b57fa700 1 -- 172.21.15.97:0/8493825 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f26a4021350 con 0x7f26b80af4d0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.380+0000 7f26b57fa700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f269c06b2c0 0x7f269c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.380+0000 7f26b57fa700 1 -- 172.21.15.97:0/8493825 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f26a4089dd0 con 0x7f26b80af4d0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.380+0000 7f26bceb7700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f269c06b2c0 0x7f269c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:43.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.381+0000 7f26bceb7700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f269c06b2c0 0x7f269c06d780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009870 tx=0x7f26ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:43.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.385+0000 7f26b57fa700 1 -- 172.21.15.97:0/8493825 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26a4054800 con 0x7f26b80af4d0 2023-11-24T06:11:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.620+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f26b8067f60 con 0x7f26b80af4d0 2023-11-24T06:11:43.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.621+0000 7f26b57fa700 1 -- 172.21.15.97:0/8493825 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f26a401e070 con 0x7f26b80af4d0 2023-11-24T06:11:43.623 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f269c06b2c0 msgr2=0x7f269c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f269c06b2c0 0x7f269c06d780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009870 tx=0x7f26ac008040 comp rx=0 tx=0).stop 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 msgr2=0x7f26b8160e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b8160e50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f26a400c750 tx=0x7f26a400cb10 comp rx=0 tx=0).stop 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 shutdown_connections 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26b815e080 0x7f26b81f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f269c06b2c0 0x7f269c06d780 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f26b806c710 0x7f26b8158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 --2- 172.21.15.97:0/8493825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b80af4d0 0x7f26b8160e50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:43.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 >> 172.21.15.97:0/8493825 conn(0x7f26b8071b90 msgr2=0x7f26b806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:43.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 shutdown_connections 2023-11-24T06:11:43.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:43.623+0000 7f26bf11b700 1 -- 172.21.15.97:0/8493825 wait complete. 2023-11-24T06:11:43.637 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:11:43.638 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:11:43.638 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-11-24T06:11:43.638 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:11:43.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:43.638 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:11:43.638 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:11:43.638 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:11:43.638 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:43.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:43.639 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-11-24T06:11:43.660 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:11:43.660 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:11:43.660 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:11:43.660 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:43.660 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:11:43.661 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:43.661 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:43.661 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:43.661 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:43.661 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:43.661 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:11:43.661 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:43.661 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:11:43.662 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:43.662 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:11:43.662 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 42025sec preferred_lft 42025sec 2023-11-24T06:11:43.662 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::5382:2bb9:1085:5950/64 scope link noprefixroute 2023-11-24T06:11:43.662 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:43.662 INFO:teuthology.orchestra.run.smithi183.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:11:43.662 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:11:43.662 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:11:43.663 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:43.663 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::b8cd:8aff:fe67:b24d/64 scope link 2023-11-24T06:11:43.663 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:43.663 INFO:teuthology.orchestra.run.smithi183.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:43.663 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether fe:4f:a7:e6:1c:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:11:43.663 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::fc4f:a7ff:fee6:1c7f/64 scope link 2023-11-24T06:11:43.663 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:43.663 INFO:teuthology.orchestra.run.smithi183.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:11:43.664 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:11:43.664 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::4034:29ff:fe35:b2a5/64 scope link 2023-11-24T06:11:43.664 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:11:43.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:43.664 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:11:43.729 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-11-24T06:11:43.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:43.731 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-11-24T06:11:43.791 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-11-24T06:11:43.791 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:11:43.791 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:11:43.792 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:11:43.792 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:11:43.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:43.792 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:11:43.792 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:11:43.792 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:11:43.793 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:11:43.793 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:11:43.793 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:11:43.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:43.902 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:11:43.919 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:11:43.919 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:11:43.919 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:43.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:43.920 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2023-11-24T06:11:43.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:43.987 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:44.043 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:11:44.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:11:44.043 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:44.099 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:11:44.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:44.100 DEBUG:teuthology.orchestra.run.smithi183:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:44.184 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:44.185 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:44.213 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:44.213 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:44.213 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:11:44.213 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:11:44.213 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:11:44.213 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:11:44.214 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:11:44.214 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:11:44.289 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:11:44.289 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:11:44.289 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:11:44.289 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:11:44.289 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:11:44.289 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:11:44.290 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:11:44.290 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:11:44.290 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:11:44.290 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:11:44.290 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:11:44.291 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:11:44.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:44.291 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:44.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:44.318 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:11:44.389 INFO:teuthology.orchestra.run.smithi183.stdout:client6618 2023-11-24T06:11:44.389 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:44.389 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:44.446 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:11:44.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.581+0000 7f11ad0ab700 1 Processor -- start 2023-11-24T06:11:44.584 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.581+0000 7f11ad0ab700 1 -- start start 2023-11-24T06:11:44.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.582+0000 7f11ad0ab700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f11a815e3c0 0x7f11a80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.582+0000 7f11ad0ab700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a80b5e70 0x7f11a80b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.582+0000 7f11ad0ab700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a80b62a0 con 0x7f11a815dfe0 2023-11-24T06:11:44.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.582+0000 7f11ad0ab700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a80d2ae0 con 0x7f11a815e3c0 2023-11-24T06:11:44.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.582+0000 7f11ad0ab700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80d1e60 con 0x7f11a80b5e70 2023-11-24T06:11:44.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.582+0000 7f11a6d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f11a815dfe0 0x7f11a80d3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42706/0 (socket says 172.21.15.97:42706) 2023-11-24T06:11:44.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.582+0000 7f11a6d9d700 1 -- 172.21.15.97:0/3225694518 learned_addr learned my addr 172.21.15.97:0/3225694518 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a659c700 1 --2- 172.21.15.97:0/3225694518 >> v2:172.21.15.183:3300/0 conn(0x7f11a815e3c0 0x7f11a80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a759e700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a80b5e70 0x7f11a80b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559064726 0 0) 0x7f11a80b62a0 con 0x7f11a815dfe0 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11900035b0 con 0x7f11a815dfe0 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4092787025 0 0) 0x7f11900035b0 con 0x7f11a815dfe0 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a80b5e70 msgr2=0x7f11a80b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a80b5e70 0x7f11a80b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 >> v2:172.21.15.183:3300/0 conn(0x7f11a815e3c0 msgr2=0x7f11a80b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 --2- 172.21.15.97:0/3225694518 >> v2:172.21.15.183:3300/0 conn(0x7f11a815e3c0 0x7f11a80b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80d2140 con 0x7f11a815dfe0 2023-11-24T06:11:44.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f119c001670 con 0x7f11a815dfe0 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 >> v1:172.21.15.97:6789/0 conn(0x7f11a815dfe0 legacy=0x7f11a80d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190003a00 0x7f1190003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a80b5e70 0x7f1190004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190004d90 0x7f1190005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80d2ae0 con 0x7f1190003a00 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80d1e60 con 0x7f1190004d90 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119000ba10 con 0x7f11a80b5e70 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a6d9d700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190003a00 0x7f1190003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a659c700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a80b5e70 0x7f1190004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a759e700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190004d90 0x7f1190005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a6d9d700 1 -- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a80b5e70 msgr2=0x7f1190004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:44.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a6d9d700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f11a80b5e70 0x7f1190004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a6d9d700 1 -- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190004d90 msgr2=0x7f1190005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a6d9d700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1190004d90 0x7f1190005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.583+0000 7f11a6d9d700 1 -- 172.21.15.97:0/3225694518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11900035b0 con 0x7f1190003a00 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11a6d9d700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190003a00 0x7f1190003e20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f119c00c370 tx=0x7f1190005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8239bdff31af5d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c007c60 con 0x7f1190003a00 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c005960 con 0x7f1190003a00 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11a5d9b700 1 -- 172.21.15.97:0/3225694518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c005d10 con 0x7f1190003a00 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11ad0ab700 1 -- 172.21.15.97:0/3225694518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190003a00 msgr2=0x7f1190003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11ad0ab700 1 --2- 172.21.15.97:0/3225694518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1190003a00 0x7f1190003e20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f119c00c370 tx=0x7f1190005880 comp rx=0 tx=0).stop 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11a759e700 1 -- 172.21.15.97:0/3225694518 reap_dead start 2023-11-24T06:11:44.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11ad0ab700 1 -- 172.21.15.97:0/3225694518 shutdown_connections 2023-11-24T06:11:44.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11ad0ab700 1 -- 172.21.15.97:0/3225694518 >> 172.21.15.97:0/3225694518 conn(0x7f11a80d0b10 msgr2=0x7f11a80d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:44.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11ad0ab700 1 -- 172.21.15.97:0/3225694518 shutdown_connections 2023-11-24T06:11:44.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.584+0000 7f11ad0ab700 1 -- 172.21.15.97:0/3225694518 wait complete. 2023-11-24T06:11:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.586+0000 7f11ad0ab700 1 Processor -- start 2023-11-24T06:11:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.586+0000 7f11ad0ab700 1 -- start start 2023-11-24T06:11:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11ad0ab700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1190003a00 0x7f11a8156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11ad0ab700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a81f4b20 0x7f11a815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11ad0ab700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11a81575b0 0x7f11a81f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11ad0ab700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a8159f60 con 0x7f11a81575b0 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11ad0ab700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a815a8c0 con 0x7f11a81f4b20 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11ad0ab700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a81592e0 con 0x7f1190003a00 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11a81575b0 0x7f11a81f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11a81575b0 0x7f11a81f7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50270/0 (socket says 172.21.15.97:50270) 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 -- 172.21.15.97:0/229761493 learned_addr learned my addr 172.21.15.97:0/229761493 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a6d9d700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1190003a00 0x7f11a8156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a659c700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a81f4b20 0x7f11a815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 -- 172.21.15.97:0/229761493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1190003a00 msgr2=0x7f11a8156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1190003a00 0x7f11a8156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 -- 172.21.15.97:0/229761493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a81f4b20 msgr2=0x7f11a815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a81f4b20 0x7f11a815a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 -- 172.21.15.97:0/229761493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a81595c0 con 0x7f11a81575b0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11a759e700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11a81575b0 0x7f11a81f7470 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f11a80d4f50 tx=0x7f11a0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f118f7fe700 1 -- 172.21.15.97:0/229761493 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a000e040 con 0x7f11a81575b0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f118f7fe700 1 -- 172.21.15.97:0/229761493 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a000f040 con 0x7f11a81575b0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a81598a0 con 0x7f11a81575b0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f118f7fe700 1 -- 172.21.15.97:0/229761493 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a000fc40 con 0x7f11a81575b0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.587+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a8157a10 con 0x7f11a81575b0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.588+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a8157c30 con 0x7f11a81575b0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.589+0000 7f118f7fe700 1 -- 172.21.15.97:0/229761493 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f11a0015430 con 0x7f11a81575b0 2023-11-24T06:11:44.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.589+0000 7f118f7fe700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f119406b2c0 0x7f119406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:44.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.589+0000 7f118f7fe700 1 -- 172.21.15.97:0/229761493 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f11a008a540 con 0x7f11a81575b0 2023-11-24T06:11:44.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.589+0000 7f11a6d9d700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f119406b2c0 0x7f119406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:44.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.590+0000 7f11a6d9d700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f119406b2c0 0x7f119406d780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f119c015040 tx=0x7f119c00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:44.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.594+0000 7f118f7fe700 1 -- 172.21.15.97:0/229761493 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11a0055020 con 0x7f11a81575b0 2023-11-24T06:11:44.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:44.827+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f11a80751e0 con 0x7f11a81575b0 2023-11-24T06:11:45.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.238+0000 7f118f7fe700 1 -- 172.21.15.97:0/229761493 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 136+0+0 (secure 0 0 0) 0x7f11a0058640 con 0x7f11a81575b0 2023-11-24T06:11:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.240+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f119406b2c0 msgr2=0x7f119406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.240+0000 7f11ad0ab700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f119406b2c0 0x7f119406d780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f119c015040 tx=0x7f119c00a000 comp rx=0 tx=0).stop 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.240+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11a81575b0 msgr2=0x7f11a81f7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.240+0000 7f11ad0ab700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11a81575b0 0x7f11a81f7470 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f11a80d4f50 tx=0x7f11a0011ed0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.241+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 shutdown_connections 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.241+0000 7f11ad0ab700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11a81575b0 0x7f11a81f7470 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.241+0000 7f11ad0ab700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f119406b2c0 0x7f119406d780 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.241+0000 7f11ad0ab700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1190003a00 0x7f11a8156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.241+0000 7f11ad0ab700 1 --2- 172.21.15.97:0/229761493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11a81f4b20 0x7f11a815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.241+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 >> 172.21.15.97:0/229761493 conn(0x7f11a80d0b10 msgr2=0x7f11a80b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.241+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 shutdown_connections 2023-11-24T06:11:45.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.241+0000 7f11ad0ab700 1 -- 172.21.15.97:0/229761493 wait complete. 2023-11-24T06:11:45.257 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:11:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.377+0000 7faf32d76700 1 Processor -- start 2023-11-24T06:11:45.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.377+0000 7faf32d76700 1 -- start start 2023-11-24T06:11:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf32d76700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf2c1446a0 0x7faf2c0b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf32d76700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c0af950 con 0x7faf2c0adf00 2023-11-24T06:11:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf32d76700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c0afc30 con 0x7faf2c06c710 2023-11-24T06:11:45.382 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf32d76700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2c144a80 con 0x7faf2c1446a0 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf30b12700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faf2c1446a0 0x7faf2c0b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf2bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faf2c0adf00 0x7faf2c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42726/0 (socket says 172.21.15.97:42726) 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf2bfff700 1 -- 172.21.15.97:0/3320316708 learned_addr learned my addr 172.21.15.97:0/3320316708 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf30b12700 1 -- 172.21.15.97:0/3320316708 >> v1:172.21.15.183:6789/0 conn(0x7faf2c06c710 legacy=0x7faf2c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf30b12700 1 -- 172.21.15.97:0/3320316708 >> v1:172.21.15.97:6789/0 conn(0x7faf2c0adf00 legacy=0x7faf2c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf30b12700 1 -- 172.21.15.97:0/3320316708 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0aff10 con 0x7faf2c1446a0 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf2b7fe700 1 -- 172.21.15.97:0/3320316708 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172579495 0 0) 0x7faf2c0af950 con 0x7faf2c0adf00 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.378+0000 7faf30b12700 1 --2- 172.21.15.97:0/3320316708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c1446a0 0x7faf2c0b5ae0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7faf20009840 tx=0x7faf20009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84bac397f03ff5a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf2b7fe700 1 -- 172.21.15.97:0/3320316708 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20011830 con 0x7faf2c1446a0 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf2b7fe700 1 -- 172.21.15.97:0/3320316708 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf20011990 con 0x7faf2c1446a0 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf2b7fe700 1 -- 172.21.15.97:0/3320316708 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20011d20 con 0x7faf2c1446a0 2023-11-24T06:11:45.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf32d76700 1 -- 172.21.15.97:0/3320316708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c1446a0 msgr2=0x7faf2c0b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf32d76700 1 --2- 172.21.15.97:0/3320316708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c1446a0 0x7faf2c0b5ae0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7faf20009840 tx=0x7faf20009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:45.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf32d76700 1 -- 172.21.15.97:0/3320316708 shutdown_connections 2023-11-24T06:11:45.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf32d76700 1 --2- 172.21.15.97:0/3320316708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c1446a0 0x7faf2c0b5ae0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf32d76700 1 -- 172.21.15.97:0/3320316708 >> 172.21.15.97:0/3320316708 conn(0x7faf2c071b90 msgr2=0x7faf2c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:45.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf32d76700 1 -- 172.21.15.97:0/3320316708 shutdown_connections 2023-11-24T06:11:45.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.379+0000 7faf32d76700 1 -- 172.21.15.97:0/3320316708 wait complete. 2023-11-24T06:11:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.380+0000 7faf32d76700 1 Processor -- start 2023-11-24T06:11:45.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.380+0000 7faf32d76700 1 -- start start 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf32d76700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf2c06c710 0x7faf2c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf32d76700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c0adf00 0x7faf2c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf32d76700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf2c1446a0 0x7faf2c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf32d76700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c15a8a0 con 0x7faf2c0adf00 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf32d76700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c15ab00 con 0x7faf2c06c710 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf32d76700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0cd6a0 con 0x7faf2c1446a0 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c0adf00 0x7faf2c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c0adf00 0x7faf2c15a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50282/0 (socket says 172.21.15.97:50282) 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf2bfff700 1 -- 172.21.15.97:0/1296676895 learned_addr learned my addr 172.21.15.97:0/1296676895 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf30b12700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf2c06c710 0x7faf2c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.381+0000 7faf31313700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf2c1446a0 0x7faf2c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.387 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf2bfff700 1 -- 172.21.15.97:0/1296676895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf2c1446a0 msgr2=0x7faf2c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf2bfff700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf2c1446a0 0x7faf2c09aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf2bfff700 1 -- 172.21.15.97:0/1296676895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf2c06c710 msgr2=0x7faf2c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf2bfff700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf2c06c710 0x7faf2c0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf2bfff700 1 -- 172.21.15.97:0/1296676895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf200094a0 con 0x7faf2c0adf00 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf2bfff700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c0adf00 0x7faf2c15a360 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7faf1800c4a0 tx=0x7faf1800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf297fa700 1 -- 172.21.15.97:0/1296676895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18004c80 con 0x7faf2c0adf00 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf297fa700 1 -- 172.21.15.97:0/1296676895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf18004de0 con 0x7faf2c0adf00 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c0cd980 con 0x7faf2c0adf00 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf297fa700 1 -- 172.21.15.97:0/1296676895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18007310 con 0x7faf2c0adf00 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.382+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c0cdf40 con 0x7faf2c0adf00 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.383+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c0753f0 con 0x7faf2c0adf00 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.383+0000 7faf297fa700 1 -- 172.21.15.97:0/1296676895 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7faf18007470 con 0x7faf2c0adf00 2023-11-24T06:11:45.388 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.384+0000 7faf297fa700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faf1006b2c0 0x7faf1006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.384+0000 7faf297fa700 1 -- 172.21.15.97:0/1296676895 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7faf180898e0 con 0x7faf2c0adf00 2023-11-24T06:11:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.384+0000 7faf30b12700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faf1006b2c0 0x7faf1006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.389 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.384+0000 7faf30b12700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faf1006b2c0 0x7faf1006d780 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faf20006ec0 tx=0x7faf2001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.391 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.388+0000 7faf297fa700 1 -- 172.21.15.97:0/1296676895 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf18054530 con 0x7faf2c0adf00 2023-11-24T06:11:45.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.605+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faf2c0cd810 con 0x7faf1006b2c0 2023-11-24T06:11:45.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.606+0000 7faf297fa700 1 -- 172.21.15.97:0/1296676895 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7faf2c0cd810 con 0x7faf1006b2c0 2023-11-24T06:11:45.609 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:11:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:11:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephfs" 2023-11-24T06:11:45.609 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:11:45.609 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:11:45.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faf1006b2c0 msgr2=0x7faf1006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faf1006b2c0 0x7faf1006d780 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7faf20006ec0 tx=0x7faf2001c040 comp rx=0 tx=0).stop 2023-11-24T06:11:45.611 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c0adf00 msgr2=0x7faf2c15a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c0adf00 0x7faf2c15a360 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7faf1800c4a0 tx=0x7faf1800c860 comp rx=0 tx=0).stop 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 shutdown_connections 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faf1006b2c0 0x7faf1006d780 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf2c1446a0 0x7faf2c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf2c06c710 0x7faf2c0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 --2- 172.21.15.97:0/1296676895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf2c0adf00 0x7faf2c15a360 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 >> 172.21.15.97:0/1296676895 conn(0x7faf2c071b90 msgr2=0x7faf2c071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 shutdown_connections 2023-11-24T06:11:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.608+0000 7faf32d76700 1 -- 172.21.15.97:0/1296676895 wait complete. 2023-11-24T06:11:45.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:11:45.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.743+0000 7fa3bd960700 1 Processor -- start 2023-11-24T06:11:45.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.743+0000 7fa3bd960700 1 -- start start 2023-11-24T06:11:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3bd960700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa3b81587f0 0x7fa3b806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3bd960700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3bd960700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b806b530 con 0x7fa3b80b3c60 2023-11-24T06:11:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3bd960700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b80ae5e0 con 0x7fa3b81587f0 2023-11-24T06:11:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3bd960700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b815be30 con 0x7fa3b815b370 2023-11-24T06:11:45.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b6ffd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa3b81587f0 0x7fa3b806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b67fc700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b77fe700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7fa3b80b3c60 0x7fa3b80af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:52066/0 (socket says 172.21.15.97:52066) 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b6ffd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa3b81587f0 0x7fa3b806aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:44448/0 (socket says 172.21.15.97:44448) 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b77fe700 1 -- 172.21.15.97:0/3912457435 learned_addr learned my addr 172.21.15.97:0/3912457435 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b67fc700 1 -- 172.21.15.97:0/3912457435 >> v2:172.21.15.183:3300/0 conn(0x7fa3b81587f0 msgr2=0x7fa3b806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b67fc700 1 --2- 172.21.15.97:0/3912457435 >> v2:172.21.15.183:3300/0 conn(0x7fa3b81587f0 0x7fa3b806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b67fc700 1 -- 172.21.15.97:0/3912457435 >> v1:172.21.15.183:6789/0 conn(0x7fa3b80b3c60 legacy=0x7fa3b80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b5ffb700 1 -- 172.21.15.97:0/3912457435 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769617175 0 0) 0x7fa3b806b530 con 0x7fa3b80b3c60 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b67fc700 1 -- 172.21.15.97:0/3912457435 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b815c110 con 0x7fa3b815b370 2023-11-24T06:11:45.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.744+0000 7fa3b6ffd700 1 --2- 172.21.15.97:0/3912457435 >> v2:172.21.15.183:3300/0 conn(0x7fa3b81587f0 0x7fa3b806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3b67fc700 1 --2- 172.21.15.97:0/3912457435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b815b7d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009840 tx=0x7fa3ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9f4fb935d90a332 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3b5ffb700 1 -- 172.21.15.97:0/3912457435 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac011830 con 0x7fa3b815b370 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3b5ffb700 1 -- 172.21.15.97:0/3912457435 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac011990 con 0x7fa3b815b370 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3b5ffb700 1 -- 172.21.15.97:0/3912457435 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac011d20 con 0x7fa3b815b370 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3bd960700 1 -- 172.21.15.97:0/3912457435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 msgr2=0x7fa3b815b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3bd960700 1 --2- 172.21.15.97:0/3912457435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b815b7d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009840 tx=0x7fa3ac009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3bd960700 1 -- 172.21.15.97:0/3912457435 shutdown_connections 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3bd960700 1 --2- 172.21.15.97:0/3912457435 >> v2:172.21.15.183:3300/0 conn(0x7fa3b81587f0 0x7fa3b806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3bd960700 1 --2- 172.21.15.97:0/3912457435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b815b7d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3bd960700 1 -- 172.21.15.97:0/3912457435 >> 172.21.15.97:0/3912457435 conn(0x7fa3b80d0bf0 msgr2=0x7fa3b80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3bd960700 1 -- 172.21.15.97:0/3912457435 shutdown_connections 2023-11-24T06:11:45.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.745+0000 7fa3bd960700 1 -- 172.21.15.97:0/3912457435 wait complete. 2023-11-24T06:11:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.746+0000 7fa3bd960700 1 Processor -- start 2023-11-24T06:11:45.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.747+0000 7fa3bd960700 1 -- start start 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.747+0000 7fa3bd960700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b80b3c60 0x7fa3b809ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.747+0000 7fa3bd960700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b81587f0 0x7fa3b80d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.747+0000 7fa3bd960700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.747+0000 7fa3bd960700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8148170 con 0x7fa3b81587f0 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.747+0000 7fa3bd960700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80b1970 con 0x7fa3b80b3c60 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.747+0000 7fa3bd960700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80b1c50 con 0x7fa3b815b370 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b81587f0 0x7fa3b80d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b81587f0 0x7fa3b80d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50296/0 (socket says 172.21.15.97:50296) 2023-11-24T06:11:45.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 -- 172.21.15.97:0/363040189 learned_addr learned my addr 172.21.15.97:0/363040189 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b6ffd700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b80b3c60 0x7fa3b809ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b77fe700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 -- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 msgr2=0x7fa3b806e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b806e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 -- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b80b3c60 msgr2=0x7fa3b809ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b80b3c60 0x7fa3b809ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 -- 172.21.15.97:0/363040189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a80094a0 con 0x7fa3b81587f0 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b77fe700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b6ffd700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b80b3c60 0x7fa3b809ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3b67fc700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b81587f0 0x7fa3b80d0110 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009b90 tx=0x7fa3ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa39ffff700 1 -- 172.21.15.97:0/363040189 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0038a0 con 0x7fa3b81587f0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa39ffff700 1 -- 172.21.15.97:0/363040189 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac01e070 con 0x7fa3b81587f0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac0094a0 con 0x7fa3b81587f0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa39ffff700 1 -- 172.21.15.97:0/363040189 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac003c40 con 0x7fa3b81587f0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.748+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b80b0bd0 con 0x7fa3b81587f0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.749+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b8068fb0 con 0x7fa3b81587f0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.750+0000 7fa39ffff700 1 -- 172.21.15.97:0/363040189 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa3ac02d450 con 0x7fa3b81587f0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.750+0000 7fa39ffff700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa3a406b2c0 0x7fa3a406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.750+0000 7fa39ffff700 1 -- 172.21.15.97:0/363040189 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa3ac08abd0 con 0x7fa3b81587f0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.750+0000 7fa3b6ffd700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa3a406b2c0 0x7fa3a406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:45.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.751+0000 7fa3b6ffd700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa3a406b2c0 0x7fa3a406d780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa3b815def0 tx=0x7fa3a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.755+0000 7fa39ffff700 1 -- 172.21.15.97:0/363040189 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3ac055820 con 0x7fa3b81587f0 2023-11-24T06:11:45.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.944+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa3b80d0650 con 0x7fa3b81587f0 2023-11-24T06:11:45.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.944+0000 7fa39ffff700 1 -- 172.21.15.97:0/363040189 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa3ac0150c0 con 0x7fa3b81587f0 2023-11-24T06:11:45.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa3a406b2c0 msgr2=0x7fa3a406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa3a406b2c0 0x7fa3a406d780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa3b815def0 tx=0x7fa3a8008040 comp rx=0 tx=0).stop 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b81587f0 msgr2=0x7fa3b80d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b81587f0 0x7fa3b80d0110 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009b90 tx=0x7fa3ac004750 comp rx=0 tx=0).stop 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 shutdown_connections 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa3a406b2c0 0x7fa3a406d780 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa3b815b370 0x7fa3b806e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b80b3c60 0x7fa3b809ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 --2- 172.21.15.97:0/363040189 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3b81587f0 0x7fa3b80d0110 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 >> 172.21.15.97:0/363040189 conn(0x7fa3b80d0bf0 msgr2=0x7fa3b8074700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:45.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.946+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 shutdown_connections 2023-11-24T06:11:45.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:45.947+0000 7fa3bd960700 1 -- 172.21.15.97:0/363040189 wait complete. 2023-11-24T06:11:45.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp513 2023-11-24T06:11:46.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.082+0000 7f395e9d7700 1 Processor -- start 2023-11-24T06:11:46.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.082+0000 7f395e9d7700 1 -- start start 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395e9d7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f39580b4310 0x7f395806bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395e9d7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39580ce430 0x7f39580ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395e9d7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39580cedc0 con 0x7f395815e080 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395e9d7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39580d1200 con 0x7f39580ce430 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395e9d7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395815e460 con 0x7f39580b4310 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395cf74700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39580ce430 0x7f39580ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f3957fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f395815e080 0x7f39580d0900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42754/0 (socket says 172.21.15.97:42754) 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f3957fff700 1 -- 172.21.15.97:0/2841830647 learned_addr learned my addr 172.21.15.97:0/2841830647 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f3956ffd700 1 -- 172.21.15.97:0/2841830647 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941824390 0 0) 0x7f3948001110 con 0x7f395815e080 2023-11-24T06:11:46.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f3956ffd700 1 -- 172.21.15.97:0/2841830647 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39400035b0 con 0x7f395815e080 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f39577fe700 1 --2- 172.21.15.97:0/2841830647 >> v2:172.21.15.183:3300/0 conn(0x7f39580b4310 0x7f395806bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395cf74700 1 -- 172.21.15.97:0/2841830647 >> v2:172.21.15.183:3300/0 conn(0x7f39580b4310 msgr2=0x7f395806bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395cf74700 1 --2- 172.21.15.97:0/2841830647 >> v2:172.21.15.183:3300/0 conn(0x7f39580b4310 0x7f395806bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395cf74700 1 -- 172.21.15.97:0/2841830647 >> v1:172.21.15.97:6789/0 conn(0x7f395815e080 legacy=0x7f39580d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395cf74700 1 -- 172.21.15.97:0/2841830647 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395806ba50 con 0x7f39580ce430 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f395cf74700 1 --2- 172.21.15.97:0/2841830647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580ce430 0x7f39580ce880 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3958072420 tx=0x7f395000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833bdca9913ad7a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f3956ffd700 1 -- 172.21.15.97:0/2841830647 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395000e070 con 0x7f39580ce430 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f3956ffd700 1 -- 172.21.15.97:0/2841830647 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395000bb20 con 0x7f39580ce430 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.083+0000 7f3956ffd700 1 -- 172.21.15.97:0/2841830647 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950012430 con 0x7f39580ce430 2023-11-24T06:11:46.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.084+0000 7f395e9d7700 1 -- 172.21.15.97:0/2841830647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580ce430 msgr2=0x7f39580ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.084+0000 7f395e9d7700 1 --2- 172.21.15.97:0/2841830647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580ce430 0x7f39580ce880 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3958072420 tx=0x7f395000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:46.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.084+0000 7f395e9d7700 1 -- 172.21.15.97:0/2841830647 shutdown_connections 2023-11-24T06:11:46.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.084+0000 7f395e9d7700 1 --2- 172.21.15.97:0/2841830647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580ce430 0x7f39580ce880 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.084+0000 7f395e9d7700 1 --2- 172.21.15.97:0/2841830647 >> v2:172.21.15.183:3300/0 conn(0x7f39580b4310 0x7f395806bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.084+0000 7f395e9d7700 1 -- 172.21.15.97:0/2841830647 >> 172.21.15.97:0/2841830647 conn(0x7f39580b3780 msgr2=0x7f39580b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:46.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.084+0000 7f395e9d7700 1 -- 172.21.15.97:0/2841830647 shutdown_connections 2023-11-24T06:11:46.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.084+0000 7f395e9d7700 1 -- 172.21.15.97:0/2841830647 wait complete. 2023-11-24T06:11:46.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.085+0000 7f395e9d7700 1 Processor -- start 2023-11-24T06:11:46.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.085+0000 7f395e9d7700 1 -- start start 2023-11-24T06:11:46.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f395e9d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580b4310 0x7f395814f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f395e9d7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39580ce430 0x7f39580b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f395e9d7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395815e080 0x7f39580c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f395e9d7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39580b7410 con 0x7f39580b4310 2023-11-24T06:11:46.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f395e9d7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39580b5400 con 0x7f395815e080 2023-11-24T06:11:46.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f395e9d7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39580b56e0 con 0x7f39580ce430 2023-11-24T06:11:46.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f3957fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580b4310 0x7f395814f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f3957fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580b4310 0x7f395814f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50308/0 (socket says 172.21.15.97:50308) 2023-11-24T06:11:46.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f3957fff700 1 -- 172.21.15.97:0/571352633 learned_addr learned my addr 172.21.15.97:0/571352633 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:46.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f395cf74700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395815e080 0x7f39580c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f39577fe700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39580ce430 0x7f39580b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f3957fff700 1 -- 172.21.15.97:0/571352633 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39580ce430 msgr2=0x7f39580b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f3957fff700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39580ce430 0x7f39580b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f3957fff700 1 -- 172.21.15.97:0/571352633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395815e080 msgr2=0x7f39580c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f3957fff700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395815e080 0x7f39580c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.086+0000 7f3957fff700 1 -- 172.21.15.97:0/571352633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3950009a70 con 0x7f39580b4310 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.087+0000 7f3957fff700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580b4310 0x7f395814f020 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3948007fd0 tx=0x7f394800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.087+0000 7f3954ff9700 1 -- 172.21.15.97:0/571352633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948005d70 con 0x7f39580b4310 2023-11-24T06:11:46.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.087+0000 7f3954ff9700 1 -- 172.21.15.97:0/571352633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39480070d0 con 0x7f39580b4310 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.087+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39580b59c0 con 0x7f39580b4310 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.087+0000 7f3954ff9700 1 -- 172.21.15.97:0/571352633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948007460 con 0x7f39580b4310 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.087+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39580b61c0 con 0x7f39580b4310 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.088+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3958068fb0 con 0x7f39580b4310 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.088+0000 7f3954ff9700 1 -- 172.21.15.97:0/571352633 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f39480075c0 con 0x7f39580b4310 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.089+0000 7f3954ff9700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f394406b2c0 0x7f394406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.089+0000 7f39577fe700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f394406b2c0 0x7f394406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.089+0000 7f3954ff9700 1 -- 172.21.15.97:0/571352633 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f39480890f0 con 0x7f39580b4310 2023-11-24T06:11:46.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.089+0000 7f39577fe700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f394406b2c0 0x7f394406d780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3958074410 tx=0x7f394c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.096 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.093+0000 7f3954ff9700 1 -- 172.21.15.97:0/571352633 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3948053ba0 con 0x7f39580b4310 2023-11-24T06:11:46.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.313+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp513", "target": ["mon-mgr", ""]}) v1 -- 0x7f39580b5570 con 0x7f394406b2c0 2023-11-24T06:11:46.320 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:46.318+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:46.320 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:46.318+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:46.321 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:46.318+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:46.321 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:46.318+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:46.321 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:46.318+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:46.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.332+0000 7f3954ff9700 1 -- 172.21.15.97:0/571352633 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f39580b5570 con 0x7f394406b2c0 2023-11-24T06:11:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f394406b2c0 msgr2=0x7f394406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f394406b2c0 0x7f394406d780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3958074410 tx=0x7f394c008040 comp rx=0 tx=0).stop 2023-11-24T06:11:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580b4310 msgr2=0x7f395814f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580b4310 0x7f395814f020 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3948007fd0 tx=0x7f394800c860 comp rx=0 tx=0).stop 2023-11-24T06:11:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 shutdown_connections 2023-11-24T06:11:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f395815e080 0x7f39580c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39580b4310 0x7f395814f020 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f394406b2c0 0x7f394406d780 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 --2- 172.21.15.97:0/571352633 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f39580ce430 0x7f39580b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.334+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 >> 172.21.15.97:0/571352633 conn(0x7f39580b3780 msgr2=0x7f39580cd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.335+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 shutdown_connections 2023-11-24T06:11:46.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.335+0000 7f395e9d7700 1 -- 172.21.15.97:0/571352633 wait complete. 2023-11-24T06:11:46.349 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol7777 --group_name subvol_grp513 2023-11-24T06:11:46.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.471+0000 7f34b10a8700 1 Processor -- start 2023-11-24T06:11:46.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34b10a8700 1 -- start start 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34b10a8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34ac0b01e0 0x7f34ac06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34b10a8700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f34ac0b59c0 0x7f34ac0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34b10a8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 0x7f34ac0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34b10a8700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac1470a0 con 0x7f34ac0b01e0 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34b10a8700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac0b5e70 con 0x7f34ac0b59c0 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34b10a8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0748f0 con 0x7f34ac073f90 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34aad9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34ac0b01e0 0x7f34ac06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34aad9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34ac0b01e0 0x7f34ac06e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50330/0 (socket says 172.21.15.97:50330) 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.472+0000 7f34aad9d700 1 -- 172.21.15.97:0/36719861 learned_addr learned my addr 172.21.15.97:0/36719861 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34aa59c700 1 --2- 172.21.15.97:0/36719861 >> v2:172.21.15.183:3300/0 conn(0x7f34ac0b59c0 0x7f34ac0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34ab59e700 1 --2- 172.21.15.97:0/36719861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 0x7f34ac0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34aad9d700 1 -- 172.21.15.97:0/36719861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 msgr2=0x7f34ac0743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34aad9d700 1 --2- 172.21.15.97:0/36719861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 0x7f34ac0743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34aad9d700 1 -- 172.21.15.97:0/36719861 >> v2:172.21.15.183:3300/0 conn(0x7f34ac0b59c0 msgr2=0x7f34ac0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34aad9d700 1 --2- 172.21.15.97:0/36719861 >> v2:172.21.15.183:3300/0 conn(0x7f34ac0b59c0 0x7f34ac0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34aad9d700 1 -- 172.21.15.97:0/36719861 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac0b05c0 con 0x7f34ac0b01e0 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34aad9d700 1 --2- 172.21.15.97:0/36719861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b01e0 0x7f34ac06e9c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f34ac074fb0 tx=0x7f34a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11de5c11f39835e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34a9d9b700 1 -- 172.21.15.97:0/36719861 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a000b860 con 0x7f34ac0b01e0 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34a9d9b700 1 -- 172.21.15.97:0/36719861 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a000b9c0 con 0x7f34ac0b01e0 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.473+0000 7f34a9d9b700 1 -- 172.21.15.97:0/36719861 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a000bd50 con 0x7f34ac0b01e0 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 -- 172.21.15.97:0/36719861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b01e0 msgr2=0x7f34ac06e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 --2- 172.21.15.97:0/36719861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b01e0 0x7f34ac06e9c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f34ac074fb0 tx=0x7f34a0009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:46.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 -- 172.21.15.97:0/36719861 shutdown_connections 2023-11-24T06:11:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 --2- 172.21.15.97:0/36719861 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 0x7f34ac0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 --2- 172.21.15.97:0/36719861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b01e0 0x7f34ac06e9c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 --2- 172.21.15.97:0/36719861 >> v2:172.21.15.183:3300/0 conn(0x7f34ac0b59c0 0x7f34ac0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 -- 172.21.15.97:0/36719861 >> 172.21.15.97:0/36719861 conn(0x7f34ac0d0b40 msgr2=0x7f34ac0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 -- 172.21.15.97:0/36719861 shutdown_connections 2023-11-24T06:11:46.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.474+0000 7f34b10a8700 1 -- 172.21.15.97:0/36719861 wait complete. 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.475+0000 7f34b10a8700 1 Processor -- start 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.475+0000 7f34b10a8700 1 -- start start 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34b10a8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 0x7f34ac154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34b10a8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34ac0b01e0 0x7f34ac13fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34b10a8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b59c0 0x7f34ac160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34b10a8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac1427a0 con 0x7f34ac0b59c0 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34b10a8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac13bff0 con 0x7f34ac0b01e0 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34b10a8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac141b20 con 0x7f34ac073f90 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b59c0 0x7f34ac160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b59c0 0x7f34ac160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50340/0 (socket says 172.21.15.97:50340) 2023-11-24T06:11:46.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 -- 172.21.15.97:0/1864509523 learned_addr learned my addr 172.21.15.97:0/1864509523 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34aad9d700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 0x7f34ac154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34aa59c700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34ac0b01e0 0x7f34ac13fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 -- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 msgr2=0x7f34ac154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 0x7f34ac154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 -- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34ac0b01e0 msgr2=0x7f34ac13fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34ac0b01e0 0x7f34ac13fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 -- 172.21.15.97:0/1864509523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a0009550 con 0x7f34ac0b59c0 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34aa59c700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34ac0b01e0 0x7f34ac13fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.476+0000 7f34ab59e700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b59c0 0x7f34ac160890 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f34a4007e70 tx=0x7f34a400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.477+0000 7f34937fe700 1 -- 172.21.15.97:0/1864509523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400ee50 con 0x7f34ac0b59c0 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.477+0000 7f34937fe700 1 -- 172.21.15.97:0/1864509523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a4014420 con 0x7f34ac0b59c0 2023-11-24T06:11:46.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.477+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac141e00 con 0x7f34ac0b59c0 2023-11-24T06:11:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.477+0000 7f34937fe700 1 -- 172.21.15.97:0/1864509523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40147d0 con 0x7f34ac0b59c0 2023-11-24T06:11:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.477+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac1423c0 con 0x7f34ac0b59c0 2023-11-24T06:11:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.478+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac067f60 con 0x7f34ac0b59c0 2023-11-24T06:11:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.478+0000 7f34937fe700 1 -- 172.21.15.97:0/1864509523 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f34a4014930 con 0x7f34ac0b59c0 2023-11-24T06:11:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.478+0000 7f34937fe700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f349806b2c0 0x7f349806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.479+0000 7f34937fe700 1 -- 172.21.15.97:0/1864509523 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f34a408a470 con 0x7f34ac0b59c0 2023-11-24T06:11:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.479+0000 7f34aad9d700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f349806b2c0 0x7f349806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:46.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.479+0000 7f34aad9d700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f349806b2c0 0x7f349806d780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009c40 tx=0x7f34a0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:46.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.483+0000 7f34937fe700 1 -- 172.21.15.97:0/1864509523 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34a4054fb0 con 0x7f34ac0b59c0 2023-11-24T06:11:46.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.707+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol7777", "group_name": "subvol_grp513", "target": ["mon-mgr", ""]}) v1 -- 0x7f34ac06dee0 con 0x7f349806b2c0 2023-11-24T06:11:46.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.738+0000 7f34937fe700 1 -- 172.21.15.97:0/1864509523 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f34a000b660 con 0x7f349806b2c0 2023-11-24T06:11:46.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f349806b2c0 msgr2=0x7f349806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f349806b2c0 0x7f349806d780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009c40 tx=0x7f34a0008030 comp rx=0 tx=0).stop 2023-11-24T06:11:46.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b59c0 msgr2=0x7f34ac160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:46.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b59c0 0x7f34ac160890 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f34a4007e70 tx=0x7f34a400e3c0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 shutdown_connections 2023-11-24T06:11:46.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34ac0b59c0 0x7f34ac160890 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f349806b2c0 0x7f349806d780 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f34ac073f90 0x7f34ac154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 --2- 172.21.15.97:0/1864509523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34ac0b01e0 0x7f34ac13fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:46.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 >> 172.21.15.97:0/1864509523 conn(0x7f34ac0d0b40 msgr2=0x7f34ac0adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:46.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 shutdown_connections 2023-11-24T06:11:46.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:46.740+0000 7f34b10a8700 1 -- 172.21.15.97:0/1864509523 wait complete. 2023-11-24T06:11:46.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:46.755 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp513/.snap/ancestral_snap_1) 2023-11-24T06:11:46.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:46.812 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%i ./volumes/subvol_grp513) 2023-11-24T06:11:46.831 INFO:teuthology.orchestra.run.smithi097.stdout:1099511627777 2023-11-24T06:11:46.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:46.832 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls ./volumes/subvol_grp513/subvol7777/.snap/_ancestral_snap_1_1099511627777) 2023-11-24T06:11:46.891 INFO:teuthology.orchestra.run.smithi097.stdout:ecc85ad8-aed8-4abb-a94b-0d5a19e13ded 2023-11-24T06:11:46.891 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol7777 _ancestral_snap_1_1099511627777 subvol_grp513 2023-11-24T06:11:47.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.054+0000 7f6a7225f700 1 Processor -- start 2023-11-24T06:11:47.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.054+0000 7f6a7225f700 1 -- start start 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a7225f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c15e320 0x7f6a6c0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a7225f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6a6c06b700 0x7f6a6c167d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a7225f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0b6fb0 con 0x7f6a6c0b93c0 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a7225f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c0d4da0 con 0x7f6a6c06b700 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a7225f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0d2940 con 0x7f6a6c15e320 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c15e320 0x7f6a6c0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a6bfff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6a6c06b700 0x7f6a6c167d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a6bfff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6a6c06b700 0x7f6a6c167d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:44510/0 (socket says 172.21.15.97:44510) 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c15e320 0x7f6a6c0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:37382/0 (socket says 172.21.15.97:37382) 2023-11-24T06:11:47.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.055+0000 7f6a6bfff700 1 -- 172.21.15.97:0/2931637775 learned_addr learned my addr 172.21.15.97:0/2931637775 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6b7fe700 1 -- 172.21.15.97:0/2931637775 >> v2:172.21.15.183:3300/0 conn(0x7f6a6c06b700 msgr2=0x7f6a6c167d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6b7fe700 1 --2- 172.21.15.97:0/2931637775 >> v2:172.21.15.183:3300/0 conn(0x7f6a6c06b700 0x7f6a6c167d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6b7fe700 1 -- 172.21.15.97:0/2931637775 >> v1:172.21.15.183:6789/0 conn(0x7f6a6c0b93c0 legacy=0x7f6a6c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6b7fe700 1 -- 172.21.15.97:0/2931637775 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0d2c10 con 0x7f6a6c15e320 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6bfff700 1 --2- 172.21.15.97:0/2931637775 >> v2:172.21.15.183:3300/0 conn(0x7f6a6c06b700 0x7f6a6c167d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6b7fe700 1 --2- 172.21.15.97:0/2931637775 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c15e320 0x7f6a6c0d2400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009840 tx=0x7f6a54009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8900e0650d16e445 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6a7fc700 1 -- 172.21.15.97:0/2931637775 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54011830 con 0x7f6a6c15e320 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6a7fc700 1 -- 172.21.15.97:0/2931637775 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54011990 con 0x7f6a6c15e320 2023-11-24T06:11:47.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.056+0000 7f6a6a7fc700 1 -- 172.21.15.97:0/2931637775 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54011d20 con 0x7f6a6c15e320 2023-11-24T06:11:47.062 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.057+0000 7f6a7225f700 1 -- 172.21.15.97:0/2931637775 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c15e320 msgr2=0x7f6a6c0d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.057+0000 7f6a7225f700 1 --2- 172.21.15.97:0/2931637775 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c15e320 0x7f6a6c0d2400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009840 tx=0x7f6a54009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.057+0000 7f6a7225f700 1 -- 172.21.15.97:0/2931637775 shutdown_connections 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.057+0000 7f6a7225f700 1 --2- 172.21.15.97:0/2931637775 >> v2:172.21.15.183:3300/0 conn(0x7f6a6c06b700 0x7f6a6c167d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.057+0000 7f6a7225f700 1 --2- 172.21.15.97:0/2931637775 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c15e320 0x7f6a6c0d2400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.057+0000 7f6a7225f700 1 -- 172.21.15.97:0/2931637775 >> 172.21.15.97:0/2931637775 conn(0x7f6a6c0d3e50 msgr2=0x7f6a6c0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.057+0000 7f6a7225f700 1 -- 172.21.15.97:0/2931637775 shutdown_connections 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.057+0000 7f6a7225f700 1 -- 172.21.15.97:0/2931637775 wait complete. 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.058+0000 7f6a7225f700 1 Processor -- start 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.058+0000 7f6a7225f700 1 -- start start 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a7225f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c06b700 0x7f6a6c15d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a7225f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a6c0b93c0 0x7f6a6c0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a7225f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a6c15e320 0x7f6a6c0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a7225f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0cddd0 con 0x7f6a6c15e320 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a7225f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0ce0b0 con 0x7f6a6c0b93c0 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a7225f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0ce390 con 0x7f6a6c06b700 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a6c15e320 0x7f6a6c0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a6c15e320 0x7f6a6c0cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50356/0 (socket says 172.21.15.97:50356) 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 -- 172.21.15.97:0/2729858370 learned_addr learned my addr 172.21.15.97:0/2729858370 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6b7fe700 1 --2- 172.21.15.97:0/2729858370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c06b700 0x7f6a6c15d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6affd700 1 --2- 172.21.15.97:0/2729858370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a6c0b93c0 0x7f6a6c0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 -- 172.21.15.97:0/2729858370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c06b700 msgr2=0x7f6a6c15d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 --2- 172.21.15.97:0/2729858370 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a6c06b700 0x7f6a6c15d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.064 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 -- 172.21.15.97:0/2729858370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a6c0b93c0 msgr2=0x7f6a6c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 --2- 172.21.15.97:0/2729858370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a6c0b93c0 0x7f6a6c0cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 -- 172.21.15.97:0/2729858370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64009b30 con 0x7f6a6c15e320 2023-11-24T06:11:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a6bfff700 1 --2- 172.21.15.97:0/2729858370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a6c15e320 0x7f6a6c0cee70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400ef90 tx=0x7f6a6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a63fff700 1 -- 172.21.15.97:0/2729858370 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64018070 con 0x7f6a6c15e320 2023-11-24T06:11:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.059+0000 7f6a63fff700 1 -- 172.21.15.97:0/2729858370 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6400f040 con 0x7f6a6c15e320 2023-11-24T06:11:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.060+0000 7f6a7225f700 1 -- 172.21.15.97:0/2729858370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a540094a0 con 0x7f6a6c15e320 2023-11-24T06:11:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.060+0000 7f6a63fff700 1 -- 172.21.15.97:0/2729858370 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64003c60 con 0x7f6a6c15e320 2023-11-24T06:11:47.065 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.060+0000 7f6a7225f700 1 -- 172.21.15.97:0/2729858370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c0ce950 con 0x7f6a6c15e320 2023-11-24T06:11:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.060+0000 7f6a7225f700 1 -- 172.21.15.97:0/2729858370 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c0cfc10 con 0x7f6a6c15e320 2023-11-24T06:11:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.061+0000 7f6a63fff700 1 -- 172.21.15.97:0/2729858370 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6a6400c880 con 0x7f6a6c15e320 2023-11-24T06:11:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.061+0000 7f6a63fff700 1 --2- 172.21.15.97:0/2729858370 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a4c06b2c0 0x7f6a4c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.061+0000 7f6a63fff700 1 -- 172.21.15.97:0/2729858370 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6a64089f80 con 0x7f6a6c15e320 2023-11-24T06:11:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.061+0000 7f6a6b7fe700 1 --2- 172.21.15.97:0/2729858370 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a4c06b2c0 0x7f6a4c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.067 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.062+0000 7f6a6b7fe700 1 --2- 172.21.15.97:0/2729858370 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a4c06b2c0 0x7f6a4c06d780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6a54000c00 tx=0x7f6a5401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.066+0000 7f6a63fff700 1 -- 172.21.15.97:0/2729858370 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a640548f0 con 0x7f6a6c15e320 2023-11-24T06:11:47.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.294+0000 7f6a7225f700 1 -- 172.21.15.97:0/2729858370 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol7777", "snap_name": "_ancestral_snap_1_1099511627777", "group_name": "subvol_grp513", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a6c0ce220 con 0x7f6a4c06b2c0 2023-11-24T06:11:47.303 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.301+0000 7fc9c0830700 -1 mgr.server reply reply (22) Invalid argument snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T06:11:47.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.301+0000 7f6a63fff700 1 -- 172.21.15.97:0/2729858370 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: -22 snapshot name '_ancestral_snap_1_1099511627777' is invalid) v1 ==== 66+0+0 (secure 0 0 0) 0x7f6a6c0ce220 con 0x7f6a4c06b2c0 2023-11-24T06:11:47.305 INFO:teuthology.orchestra.run.smithi097.stderr:Error EINVAL: snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T06:11:47.308 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-24T06:11:47.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:47.309 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/subvol_grp513/.snap/ancestral_snap_1) 2023-11-24T06:11:47.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol7777 --group_name subvol_grp513 2023-11-24T06:11:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.505+0000 7f9bada96700 1 Processor -- start 2023-11-24T06:11:47.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.505+0000 7f9bada96700 1 -- start start 2023-11-24T06:11:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.505+0000 7f9bada96700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 0x7f9ba806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.505+0000 7f9bada96700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9ba80b59c0 0x7f9ba80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.505+0000 7f9bada96700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ba8073f90 0x7f9ba80743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.505+0000 7f9bada96700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba81470a0 con 0x7f9ba8073f90 2023-11-24T06:11:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.505+0000 7f9bada96700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba8074920 con 0x7f9ba80b59c0 2023-11-24T06:11:47.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.505+0000 7f9bada96700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba80b5e40 con 0x7f9ba80b01e0 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ba8073f90 0x7f9ba80743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ba8073f90 0x7f9ba80743e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50364/0 (socket says 172.21.15.97:50364) 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 -- 172.21.15.97:0/1395352347 learned_addr learned my addr 172.21.15.97:0/1395352347 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 -- 172.21.15.97:0/1395352347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 msgr2=0x7f9ba806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba77fe700 1 --2- 172.21.15.97:0/1395352347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 0x7f9ba806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba6ffd700 1 --2- 172.21.15.97:0/1395352347 >> v2:172.21.15.183:3300/0 conn(0x7f9ba80b59c0 0x7f9ba80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 --2- 172.21.15.97:0/1395352347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 0x7f9ba806e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 -- 172.21.15.97:0/1395352347 >> v2:172.21.15.183:3300/0 conn(0x7f9ba80b59c0 msgr2=0x7f9ba80b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 --2- 172.21.15.97:0/1395352347 >> v2:172.21.15.183:3300/0 conn(0x7f9ba80b59c0 0x7f9ba80b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 -- 172.21.15.97:0/1395352347 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba80b05c0 con 0x7f9ba8073f90 2023-11-24T06:11:47.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba7fff700 1 --2- 172.21.15.97:0/1395352347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba80743e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8073320 tx=0x7f9ba000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988fad801f5350ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba67fc700 1 -- 172.21.15.97:0/1395352347 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000e070 con 0x7f9ba8073f90 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba67fc700 1 -- 172.21.15.97:0/1395352347 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba000b9c0 con 0x7f9ba8073f90 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.506+0000 7f9ba67fc700 1 -- 172.21.15.97:0/1395352347 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba000bd70 con 0x7f9ba8073f90 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 -- 172.21.15.97:0/1395352347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 msgr2=0x7f9ba80743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 --2- 172.21.15.97:0/1395352347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba80743e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9ba8073320 tx=0x7f9ba000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 -- 172.21.15.97:0/1395352347 shutdown_connections 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 --2- 172.21.15.97:0/1395352347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba80743e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 --2- 172.21.15.97:0/1395352347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 0x7f9ba806e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 --2- 172.21.15.97:0/1395352347 >> v2:172.21.15.183:3300/0 conn(0x7f9ba80b59c0 0x7f9ba80b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 -- 172.21.15.97:0/1395352347 >> 172.21.15.97:0/1395352347 conn(0x7f9ba80d0b40 msgr2=0x7f9ba80d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:47.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 -- 172.21.15.97:0/1395352347 shutdown_connections 2023-11-24T06:11:47.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.507+0000 7f9bada96700 1 -- 172.21.15.97:0/1395352347 wait complete. 2023-11-24T06:11:47.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.508+0000 7f9bada96700 1 Processor -- start 2023-11-24T06:11:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.508+0000 7f9bada96700 1 -- start start 2023-11-24T06:11:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9bada96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9bada96700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 0x7f9ba8142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9bada96700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba80b59c0 0x7f9ba8071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9bada96700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba81f4480 con 0x7f9ba8073f90 2023-11-24T06:11:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9bada96700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba81f4760 con 0x7f9ba80b59c0 2023-11-24T06:11:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9bada96700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba81f4a40 con 0x7f9ba80b01e0 2023-11-24T06:11:47.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba815d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50366/0 (socket says 172.21.15.97:50366) 2023-11-24T06:11:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 -- 172.21.15.97:0/326538421 learned_addr learned my addr 172.21.15.97:0/326538421 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 -- 172.21.15.97:0/326538421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 msgr2=0x7f9ba8142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba6ffd700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 0x7f9ba8142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba7fff700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba80b59c0 0x7f9ba8071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 0x7f9ba8142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 -- 172.21.15.97:0/326538421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba80b59c0 msgr2=0x7f9ba8071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba80b59c0 0x7f9ba8071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 -- 172.21.15.97:0/326538421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba0009b30 con 0x7f9ba8073f90 2023-11-24T06:11:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9ba77fe700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba815d8d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c7f0 tx=0x7f9b9800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9b8ffff700 1 -- 172.21.15.97:0/326538421 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98011800 con 0x7f9ba8073f90 2023-11-24T06:11:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.509+0000 7f9b8ffff700 1 -- 172.21.15.97:0/326538421 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b98011960 con 0x7f9ba8073f90 2023-11-24T06:11:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.510+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba81f4d20 con 0x7f9ba8073f90 2023-11-24T06:11:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.510+0000 7f9b8ffff700 1 -- 172.21.15.97:0/326538421 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b98011cf0 con 0x7f9ba8073f90 2023-11-24T06:11:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.510+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba815e090 con 0x7f9ba8073f90 2023-11-24T06:11:47.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.510+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba80b5430 con 0x7f9ba8073f90 2023-11-24T06:11:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.512+0000 7f9b8ffff700 1 -- 172.21.15.97:0/326538421 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9b98024480 con 0x7f9ba8073f90 2023-11-24T06:11:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.512+0000 7f9b8ffff700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b9406b2c0 0x7f9b9406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.513+0000 7f9b8ffff700 1 -- 172.21.15.97:0/326538421 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9b9808a270 con 0x7f9ba8073f90 2023-11-24T06:11:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.513+0000 7f9ba6ffd700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b9406b2c0 0x7f9b9406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.513+0000 7f9ba6ffd700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b9406b2c0 0x7f9b9406d780 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80cfb30 tx=0x7f9b9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.516+0000 7f9b8ffff700 1 -- 172.21.15.97:0/326538421 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b98054c60 con 0x7f9ba8073f90 2023-11-24T06:11:47.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.740+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol7777", "group_name": "subvol_grp513", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ba81f48d0 con 0x7f9b9406b2c0 2023-11-24T06:11:47.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.759+0000 7f9b8ffff700 1 -- 172.21.15.97:0/326538421 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9ba81f48d0 con 0x7f9b9406b2c0 2023-11-24T06:11:47.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b9406b2c0 msgr2=0x7f9b9406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b9406b2c0 0x7f9b9406d780 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80cfb30 tx=0x7f9b9c008040 comp rx=0 tx=0).stop 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 msgr2=0x7f9ba815d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba815d8d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9b9800c7f0 tx=0x7f9b9800cbb0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 shutdown_connections 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ba80b59c0 0x7f9ba8071bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ba8073f90 0x7f9ba815d8d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9b9406b2c0 0x7f9b9406d780 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 --2- 172.21.15.97:0/326538421 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9ba80b01e0 0x7f9ba8142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 >> 172.21.15.97:0/326538421 conn(0x7f9ba80d0b40 msgr2=0x7f9ba80b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.761+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 shutdown_connections 2023-11-24T06:11:47.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.762+0000 7f9bada96700 1 -- 172.21.15.97:0/326538421 wait complete. 2023-11-24T06:11:47.767 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.764+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.767 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.764+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.767 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.764+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.767 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.764+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.767 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.764+0000 7fc9c3836700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:47.778 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T06:11:47.781 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.779+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.781 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.779+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.782 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.779+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.782 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.779+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.782 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:11:47.779+0000 7fc9c2033700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:11:47.813 INFO:teuthology.orchestra.run.smithi097.stdout:2 2023-11-24T06:11:47.814 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp513 2023-11-24T06:11:47.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.975+0000 7f1ff0917700 1 Processor -- start 2023-11-24T06:11:47.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.975+0000 7f1ff0917700 1 -- start start 2023-11-24T06:11:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1ff0917700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1fec0746a0 0x7f1fec0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1ff0917700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1fec0afc90 0x7f1fec06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1ff0917700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fec0b3c60 con 0x7f1fec15e160 2023-11-24T06:11:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1ff0917700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fec0b4500 con 0x7f1fec0afc90 2023-11-24T06:11:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1ff0917700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fec0b58c0 con 0x7f1fec0746a0 2023-11-24T06:11:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fead9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1fec0afc90 0x7f1fec06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fea59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1fec15e160 0x7f1fec073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42854/0 (socket says 172.21.15.97:42854) 2023-11-24T06:11:47.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fea59c700 1 -- 172.21.15.97:0/1204797327 learned_addr learned my addr 172.21.15.97:0/1204797327 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fe9d9b700 1 --2- 172.21.15.97:0/1204797327 >> v2:172.21.15.183:3300/0 conn(0x7f1fec0746a0 0x7f1fec0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fead9d700 1 -- 172.21.15.97:0/1204797327 >> v2:172.21.15.183:3300/0 conn(0x7f1fec0746a0 msgr2=0x7f1fec0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fead9d700 1 --2- 172.21.15.97:0/1204797327 >> v2:172.21.15.183:3300/0 conn(0x7f1fec0746a0 0x7f1fec0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fead9d700 1 -- 172.21.15.97:0/1204797327 >> v1:172.21.15.97:6789/0 conn(0x7f1fec15e160 legacy=0x7f1fec073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fead9d700 1 -- 172.21.15.97:0/1204797327 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec06bdc0 con 0x7f1fec0afc90 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fe959a700 1 -- 172.21.15.97:0/1204797327 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674869501 0 0) 0x7f1fec0b3c60 con 0x7f1fec15e160 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fe9d9b700 1 --2- 172.21.15.97:0/1204797327 >> v2:172.21.15.183:3300/0 conn(0x7f1fec0746a0 0x7f1fec0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.976+0000 7f1fead9d700 1 --2- 172.21.15.97:0/1204797327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec06d930 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0d0880 tx=0x7f1fe400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30f683924cd16ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1fe959a700 1 -- 172.21.15.97:0/1204797327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400e070 con 0x7f1fec0afc90 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1fe959a700 1 -- 172.21.15.97:0/1204797327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe400bbd0 con 0x7f1fec0afc90 2023-11-24T06:11:47.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1fe959a700 1 -- 172.21.15.97:0/1204797327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4012430 con 0x7f1fec0afc90 2023-11-24T06:11:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1ff0917700 1 -- 172.21.15.97:0/1204797327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 msgr2=0x7f1fec06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1ff0917700 1 --2- 172.21.15.97:0/1204797327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec06d930 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0d0880 tx=0x7f1fe400b330 comp rx=0 tx=0).stop 2023-11-24T06:11:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1ff0917700 1 -- 172.21.15.97:0/1204797327 shutdown_connections 2023-11-24T06:11:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1ff0917700 1 --2- 172.21.15.97:0/1204797327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec06d930 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1ff0917700 1 --2- 172.21.15.97:0/1204797327 >> v2:172.21.15.183:3300/0 conn(0x7f1fec0746a0 0x7f1fec0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1ff0917700 1 -- 172.21.15.97:0/1204797327 >> 172.21.15.97:0/1204797327 conn(0x7f1fec0d0bf0 msgr2=0x7f1fec0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1ff0917700 1 -- 172.21.15.97:0/1204797327 shutdown_connections 2023-11-24T06:11:47.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.977+0000 7f1ff0917700 1 -- 172.21.15.97:0/1204797327 wait complete. 2023-11-24T06:11:47.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.978+0000 7f1ff0917700 1 Processor -- start 2023-11-24T06:11:47.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1ff0917700 1 -- start start 2023-11-24T06:11:47.984 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1ff0917700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fec0746a0 0x7f1fec1445d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1ff0917700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec1600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1ff0917700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec15e160 0x7f1fec161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1ff0917700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec1613d0 con 0x7f1fec0afc90 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1ff0917700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec160b30 con 0x7f1fec15e160 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1ff0917700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec160e10 con 0x7f1fec0746a0 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1fe9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec1600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1fe9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec1600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50410/0 (socket says 172.21.15.97:50410) 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.979+0000 7f1fe9d9b700 1 -- 172.21.15.97:0/342060860 learned_addr learned my addr 172.21.15.97:0/342060860 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fea59c700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fec0746a0 0x7f1fec1445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fead9d700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec15e160 0x7f1fec161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fe9d9b700 1 -- 172.21.15.97:0/342060860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fec0746a0 msgr2=0x7f1fec1445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fe9d9b700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fec0746a0 0x7f1fec1445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fe9d9b700 1 -- 172.21.15.97:0/342060860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec15e160 msgr2=0x7f1fec161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fe9d9b700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec15e160 0x7f1fec161ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fe9d9b700 1 -- 172.21.15.97:0/342060860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe4009a70 con 0x7f1fec0afc90 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fe9d9b700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec1600e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000e810 tx=0x7f1fe000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fdaffd700 1 -- 172.21.15.97:0/342060860 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0013610 con 0x7f1fec0afc90 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fdaffd700 1 -- 172.21.15.97:0/342060860 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe0013770 con 0x7f1fec0afc90 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fec1610f0 con 0x7f1fec0afc90 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1fdaffd700 1 -- 172.21.15.97:0/342060860 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0013b20 con 0x7f1fec0afc90 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.980+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fec144b10 con 0x7f1fec0afc90 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.981+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fec068fb0 con 0x7f1fec0afc90 2023-11-24T06:11:47.986 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.981+0000 7f1fdaffd700 1 -- 172.21.15.97:0/342060860 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1fe0013c80 con 0x7f1fec0afc90 2023-11-24T06:11:47.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.982+0000 7f1fdaffd700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1fcc06b2c0 0x7f1fcc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:47.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.982+0000 7f1fea59c700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1fcc06b2c0 0x7f1fcc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:47.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.982+0000 7f1fdaffd700 1 -- 172.21.15.97:0/342060860 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1fe0089a70 con 0x7f1fec0afc90 2023-11-24T06:11:47.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.982+0000 7f1fea59c700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1fcc06b2c0 0x7f1fcc06d780 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000f60 tx=0x7f1fdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:47.989 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:47.986+0000 7f1fdaffd700 1 -- 172.21.15.97:0/342060860 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1fe00544a0 con 0x7f1fec0afc90 2023-11-24T06:11:48.210 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.207+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp513", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fec160f80 con 0x7f1fcc06b2c0 2023-11-24T06:11:48.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.212+0000 7f1fdaffd700 1 -- 172.21.15.97:0/342060860 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1fec160f80 con 0x7f1fcc06b2c0 2023-11-24T06:11:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1fcc06b2c0 msgr2=0x7f1fcc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1fcc06b2c0 0x7f1fcc06d780 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc000f60 tx=0x7f1fdc009040 comp rx=0 tx=0).stop 2023-11-24T06:11:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 msgr2=0x7f1fec1600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec1600e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000e810 tx=0x7f1fe000ebd0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 shutdown_connections 2023-11-24T06:11:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fec15e160 0x7f1fec161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1fcc06b2c0 0x7f1fcc06d780 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1fec0746a0 0x7f1fec1445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 --2- 172.21.15.97:0/342060860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1fec0afc90 0x7f1fec1600e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 >> 172.21.15.97:0/342060860 conn(0x7f1fec0d0bf0 msgr2=0x7f1fec0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 shutdown_connections 2023-11-24T06:11:48.218 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.214+0000 7f1ff0917700 1 -- 172.21.15.97:0/342060860 wait complete. 2023-11-24T06:11:48.228 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:11:48.229 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:11:48.274 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:11:48.321 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:11:48.321 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:48.322 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:48.339 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:48.339 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:48.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:48.557 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.555+0000 7f40df12e700 1 Processor -- start 2023-11-24T06:11:48.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.555+0000 7f40df12e700 1 -- start start 2023-11-24T06:11:48.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40df12e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40d80afe70 0x7f40d815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40df12e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f40d81624d0 0x7f40d8162940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40df12e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d806dd80 con 0x7f40d815bef0 2023-11-24T06:11:48.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40df12e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d815f880 con 0x7f40d80afe70 2023-11-24T06:11:48.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40df12e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d815ed90 con 0x7f40d81624d0 2023-11-24T06:11:48.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40d80afe70 0x7f40d815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40d80afe70 0x7f40d815e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50418/0 (socket says 172.21.15.97:50418) 2023-11-24T06:11:48.560 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d7fff700 1 -- 172.21.15.97:0/3453634686 learned_addr learned my addr 172.21.15.97:0/3453634686 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40dd6cb700 1 --2- 172.21.15.97:0/3453634686 >> v2:172.21.15.183:3300/0 conn(0x7f40d81624d0 0x7f40d8162940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d7fff700 1 -- 172.21.15.97:0/3453634686 >> v2:172.21.15.183:3300/0 conn(0x7f40d81624d0 msgr2=0x7f40d8162940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d7fff700 1 --2- 172.21.15.97:0/3453634686 >> v2:172.21.15.183:3300/0 conn(0x7f40d81624d0 0x7f40d8162940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d7fff700 1 -- 172.21.15.97:0/3453634686 >> v1:172.21.15.183:6789/0 conn(0x7f40d815bef0 legacy=0x7f40d80b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d7fff700 1 -- 172.21.15.97:0/3453634686 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d815c2d0 con 0x7f40d80afe70 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d7fff700 1 --2- 172.21.15.97:0/3453634686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80afe70 0x7f40d815e850 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f40d8071a50 tx=0x7f40c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f644bb685be00c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d77fe700 1 -- 172.21.15.97:0/3453634686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4004230 con 0x7f40d80afe70 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d77fe700 1 -- 172.21.15.97:0/3453634686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c4004390 con 0x7f40d80afe70 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.556+0000 7f40d77fe700 1 -- 172.21.15.97:0/3453634686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c40036a0 con 0x7f40d80afe70 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.557+0000 7f40df12e700 1 -- 172.21.15.97:0/3453634686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80afe70 msgr2=0x7f40d815e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.557+0000 7f40df12e700 1 --2- 172.21.15.97:0/3453634686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80afe70 0x7f40d815e850 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f40d8071a50 tx=0x7f40c4009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.557+0000 7f40df12e700 1 -- 172.21.15.97:0/3453634686 shutdown_connections 2023-11-24T06:11:48.561 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.557+0000 7f40df12e700 1 --2- 172.21.15.97:0/3453634686 >> v2:172.21.15.183:3300/0 conn(0x7f40d81624d0 0x7f40d8162940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.557+0000 7f40df12e700 1 --2- 172.21.15.97:0/3453634686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d80afe70 0x7f40d815e850 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.557+0000 7f40df12e700 1 -- 172.21.15.97:0/3453634686 >> 172.21.15.97:0/3453634686 conn(0x7f40d8071ac0 msgr2=0x7f40d8071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:48.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.557+0000 7f40df12e700 1 -- 172.21.15.97:0/3453634686 shutdown_connections 2023-11-24T06:11:48.562 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.557+0000 7f40df12e700 1 -- 172.21.15.97:0/3453634686 wait complete. 2023-11-24T06:11:48.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.558+0000 7f40df12e700 1 Processor -- start 2023-11-24T06:11:48.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.559+0000 7f40df12e700 1 -- start start 2023-11-24T06:11:48.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40df12e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80afe70 0x7f40d814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.564 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40df12e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d815bef0 0x7f40d80ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40df12e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d81624d0 0x7f40d80ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40df12e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81f41b0 con 0x7f40d81624d0 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40df12e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d815e790 con 0x7f40d80afe70 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40df12e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d815db80 con 0x7f40d815bef0 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d81624d0 0x7f40d80ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d81624d0 0x7f40d80ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50426/0 (socket says 172.21.15.97:50426) 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 -- 172.21.15.97:0/172867936 learned_addr learned my addr 172.21.15.97:0/172867936 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40d7fff700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d815bef0 0x7f40d80ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dceca700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80afe70 0x7f40d814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 -- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d815bef0 msgr2=0x7f40d80ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d815bef0 0x7f40d80ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.565 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 -- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80afe70 msgr2=0x7f40d814d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80afe70 0x7f40d814d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 -- 172.21.15.97:0/172867936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c4009550 con 0x7f40d81624d0 2023-11-24T06:11:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dceca700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80afe70 0x7f40d814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40d7fff700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d815bef0 0x7f40d80ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40dd6cb700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d81624d0 0x7f40d80ae6a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f40d000cda0 tx=0x7f40d000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40d57fa700 1 -- 172.21.15.97:0/172867936 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0003a20 con 0x7f40d81624d0 2023-11-24T06:11:48.566 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40d57fa700 1 -- 172.21.15.97:0/172867936 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d0003b80 con 0x7f40d81624d0 2023-11-24T06:11:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40d57fa700 1 -- 172.21.15.97:0/172867936 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0004020 con 0x7f40d81624d0 2023-11-24T06:11:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d815de60 con 0x7f40d81624d0 2023-11-24T06:11:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.560+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d815e420 con 0x7f40d81624d0 2023-11-24T06:11:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.561+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d815e920 con 0x7f40d81624d0 2023-11-24T06:11:48.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.562+0000 7f40d57fa700 1 -- 172.21.15.97:0/172867936 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f40d0004180 con 0x7f40d81624d0 2023-11-24T06:11:48.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.562+0000 7f40d57fa700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40bc06b350 0x7f40bc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.562+0000 7f40dceca700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40bc06b350 0x7f40bc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.563+0000 7f40d57fa700 1 -- 172.21.15.97:0/172867936 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f40d0012070 con 0x7f40d81624d0 2023-11-24T06:11:48.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.563+0000 7f40dceca700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40bc06b350 0x7f40bc06d810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f40cc001750 tx=0x7f40cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.569 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.567+0000 7f40d57fa700 1 -- 172.21.15.97:0/172867936 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40d0054ff0 con 0x7f40d81624d0 2023-11-24T06:11:48.758 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.756+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40d806fed0 con 0x7f40d81624d0 2023-11-24T06:11:48.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.757+0000 7f40d57fa700 1 -- 172.21.15.97:0/172867936 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16623 (secure 0 0 0) 0x7f40d0058610 con 0x7f40d81624d0 2023-11-24T06:11:48.760 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:48.760 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":72,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:37.197270+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:33.973262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:34.541598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40bc06b350 msgr2=0x7f40bc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40bc06b350 0x7f40bc06d810 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f40cc001750 tx=0x7f40cc009040 comp rx=0 tx=0).stop 2023-11-24T06:11:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d81624d0 msgr2=0x7f40d80ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d81624d0 0x7f40d80ae6a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f40d000cda0 tx=0x7f40d000a570 comp rx=0 tx=0).stop 2023-11-24T06:11:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 shutdown_connections 2023-11-24T06:11:48.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40d81624d0 0x7f40d80ae6a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40d80afe70 0x7f40d814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f40bc06b350 0x7f40bc06d810 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 --2- 172.21.15.97:0/172867936 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f40d815bef0 0x7f40d80ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 >> 172.21.15.97:0/172867936 conn(0x7f40d8071ac0 msgr2=0x7f40d8071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 shutdown_connections 2023-11-24T06:11:48.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.759+0000 7f40df12e700 1 -- 172.21.15.97:0/172867936 wait complete. 2023-11-24T06:11:48.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:33.973262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:34.541598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:48.775 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:48.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.896+0000 7ff1ad7d7700 1 Processor -- start 2023-11-24T06:11:48.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.896+0000 7ff1ad7d7700 1 -- start start 2023-11-24T06:11:48.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.896+0000 7ff1ad7d7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff1a815e040 0x7ff1a815e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.896+0000 7ff1ad7d7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a80b2b00 con 0x7ff1a8151660 2023-11-24T06:11:48.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.896+0000 7ff1ad7d7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a806e3a0 con 0x7ff1a806dec0 2023-11-24T06:11:48.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.896+0000 7ff1ad7d7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a815ea30 con 0x7ff1a815e040 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a6ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff1a8151660 0x7ff1a80cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42882/0 (socket says 172.21.15.97:42882) 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a6ffd700 1 -- 172.21.15.97:0/3563193003 learned_addr learned my addr 172.21.15.97:0/3563193003 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a77fe700 1 --2- 172.21.15.97:0/3563193003 >> v2:172.21.15.183:3300/0 conn(0x7ff1a815e040 0x7ff1a815e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850708861 0 0) 0x7ff1a80b2b00 con 0x7ff1a8151660 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1900035b0 con 0x7ff1a8151660 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1085807287 0 0) 0x7ff1900035b0 con 0x7ff1a8151660 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 >> v2:172.21.15.183:3300/0 conn(0x7ff1a815e040 msgr2=0x7ff1a815e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 --2- 172.21.15.97:0/3563193003 >> v2:172.21.15.183:3300/0 conn(0x7ff1a815e040 0x7ff1a815e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 >> v1:172.21.15.183:6789/0 conn(0x7ff1a806dec0 legacy=0x7ff1a8070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80b2da0 con 0x7ff1a8151660 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7ff19c001670 con 0x7ff1a8151660 2023-11-24T06:11:48.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 >> v1:172.21.15.97:6789/0 conn(0x7ff1a8151660 legacy=0x7ff1a80cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff190003a00 0x7ff190003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190004850 0x7ff190004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff190005210 0x7ff19000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815ea30 con 0x7ff190005210 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19000c0b0 con 0x7ff190004850 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19000c270 con 0x7ff190003a00 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a77fe700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff190005210 0x7ff19000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a67fc700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190004850 0x7ff190004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.897+0000 7ff1a6ffd700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff190003a00 0x7ff190003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a77fe700 1 -- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff190003a00 msgr2=0x7ff190003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a77fe700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff190003a00 0x7ff190003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.906 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a77fe700 1 -- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190004850 msgr2=0x7ff190004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a77fe700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190004850 0x7ff190004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a77fe700 1 -- 172.21.15.97:0/3563193003 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900035b0 con 0x7ff190005210 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a6ffd700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff190003a00 0x7ff190003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a77fe700 1 -- 172.21.15.97:0/3563193003 reap_dead start 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a77fe700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff190005210 0x7ff19000ba10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009fc0 tx=0x7ff190005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc40c38c2ee29a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0003a20 con 0x7ff190005210 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a000f040 con 0x7ff190005210 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1a5ffb700 1 -- 172.21.15.97:0/3563193003 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0003da0 con 0x7ff190005210 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/3563193003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff190005210 msgr2=0x7ff19000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1ad7d7700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff190005210 0x7ff19000ba10 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009fc0 tx=0x7ff190005880 comp rx=0 tx=0).stop 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/3563193003 shutdown_connections 2023-11-24T06:11:48.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1ad7d7700 1 --2- 172.21.15.97:0/3563193003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff190005210 0x7ff19000ba10 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/3563193003 >> 172.21.15.97:0/3563193003 conn(0x7ff1a806b410 msgr2=0x7ff1a806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/3563193003 shutdown_connections 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.898+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/3563193003 wait complete. 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.900+0000 7ff1ad7d7700 1 Processor -- start 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.900+0000 7ff1ad7d7700 1 -- start start 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.900+0000 7ff1ad7d7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190005210 0x7ff1a81599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.900+0000 7ff1ad7d7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1a80ce070 0x7ff1a80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.900+0000 7ff1ad7d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a8151a40 0x7ff1a8151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.900+0000 7ff1ad7d7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8153d60 con 0x7ff1a8151a40 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.900+0000 7ff1ad7d7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80cea30 con 0x7ff190005210 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.900+0000 7ff1ad7d7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8153150 con 0x7ff1a80ce070 2023-11-24T06:11:48.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a8151a40 0x7ff1a8151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a8151a40 0x7ff1a8151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50444/0 (socket says 172.21.15.97:50444) 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 -- 172.21.15.97:0/1687204341 learned_addr learned my addr 172.21.15.97:0/1687204341 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a67fc700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1a80ce070 0x7ff1a80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a6ffd700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190005210 0x7ff1a81599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 -- 172.21.15.97:0/1687204341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1a80ce070 msgr2=0x7ff1a80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1a80ce070 0x7ff1a80ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 -- 172.21.15.97:0/1687204341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190005210 msgr2=0x7ff1a81599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190005210 0x7ff1a81599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 -- 172.21.15.97:0/1687204341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8153430 con 0x7ff1a8151a40 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1a77fe700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a8151a40 0x7ff1a8151ec0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80b5110 tx=0x7ff1a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff18f7fe700 1 -- 172.21.15.97:0/1687204341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0007000 con 0x7ff1a8151a40 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff18f7fe700 1 -- 172.21.15.97:0/1687204341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a0003e10 con 0x7ff1a8151a40 2023-11-24T06:11:48.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff18f7fe700 1 -- 172.21.15.97:0/1687204341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0004470 con 0x7ff1a8151a40 2023-11-24T06:11:48.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a8153710 con 0x7ff1a8151a40 2023-11-24T06:11:48.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.901+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a8159f10 con 0x7ff1a8151a40 2023-11-24T06:11:48.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.902+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a8068480 con 0x7ff1a8151a40 2023-11-24T06:11:48.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.903+0000 7ff18f7fe700 1 -- 172.21.15.97:0/1687204341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7ff1a00045d0 con 0x7ff1a8151a40 2023-11-24T06:11:48.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.903+0000 7ff18f7fe700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff19406b350 0x7ff19406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:48.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.903+0000 7ff18f7fe700 1 -- 172.21.15.97:0/1687204341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff1a008af00 con 0x7ff1a8151a40 2023-11-24T06:11:48.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.903+0000 7ff1a6ffd700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff19406b350 0x7ff19406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:48.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.903+0000 7ff1a6ffd700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff19406b350 0x7ff19406d810 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff19c000c00 tx=0x7ff19c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:48.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:48.907+0000 7ff18f7fe700 1 -- 172.21.15.97:0/1687204341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1a0055a20 con 0x7ff1a8151a40 2023-11-24T06:11:49.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.097+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1a815a0a0 con 0x7ff1a8151a40 2023-11-24T06:11:49.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.098+0000 7ff18f7fe700 1 -- 172.21.15.97:0/1687204341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1a0018020 con 0x7ff1a8151a40 2023-11-24T06:11:49.101 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff19406b350 msgr2=0x7ff19406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff19406b350 0x7ff19406d810 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff19c000c00 tx=0x7ff19c007040 comp rx=0 tx=0).stop 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a8151a40 msgr2=0x7ff1a8151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a8151a40 0x7ff1a8151ec0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80b5110 tx=0x7ff1a0009750 comp rx=0 tx=0).stop 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 shutdown_connections 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1a8151a40 0x7ff1a8151ec0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff190005210 0x7ff1a81599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff19406b350 0x7ff19406d810 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.103 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 --2- 172.21.15.97:0/1687204341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff1a80ce070 0x7ff1a80ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 >> 172.21.15.97:0/1687204341 conn(0x7ff1a806b410 msgr2=0x7ff1a806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 shutdown_connections 2023-11-24T06:11:49.104 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.100+0000 7ff1ad7d7700 1 -- 172.21.15.97:0/1687204341 wait complete. 2023-11-24T06:11:49.115 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf150cb700 1 Processor -- start 2023-11-24T06:11:49.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf150cb700 1 -- start start 2023-11-24T06:11:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf150cb700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faf1015e250 0x7faf10073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf150cb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf100afea0 0x7faf100744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf150cb700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf100b48f0 con 0x7faf100b0560 2023-11-24T06:11:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf150cb700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1015e630 con 0x7faf1015e250 2023-11-24T06:11:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf150cb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100b4bd0 con 0x7faf100afea0 2023-11-24T06:11:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf0e59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faf100b0560 0x7faf10161cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42904/0 (socket says 172.21.15.97:42904) 2023-11-24T06:11:49.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.235+0000 7faf0e59c700 1 -- 172.21.15.97:0/3167461340 learned_addr learned my addr 172.21.15.97:0/3167461340 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707250106 0 0) 0x7faf100b48f0 con 0x7faf100b0560 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faef80035b0 con 0x7faf100b0560 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0f59e700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf100afea0 0x7faf100744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0ed9d700 1 --2- 172.21.15.97:0/3167461340 >> v2:172.21.15.183:3300/0 conn(0x7faf1015e250 0x7faf10073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2886184241 0 0) 0x7faef80035b0 con 0x7faf100b0560 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf100afea0 msgr2=0x7faf100744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf100afea0 0x7faf100744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 >> v2:172.21.15.183:3300/0 conn(0x7faf1015e250 msgr2=0x7faf10073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 --2- 172.21.15.97:0/3167461340 >> v2:172.21.15.183:3300/0 conn(0x7faf1015e250 0x7faf10073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf100b50d0 con 0x7faf100b0560 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7faf00001670 con 0x7faf100b0560 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 >> v1:172.21.15.97:6789/0 conn(0x7faf100b0560 legacy=0x7faf10161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef8003a00 0x7faef8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf100afea0 0x7faef8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef8004d90 0x7faef8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1015e630 con 0x7faef8004d90 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100b4bd0 con 0x7faef8003a00 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef800ba90 con 0x7faf100afea0 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0f59e700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef8004d90 0x7faef8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0ed9d700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef8003a00 0x7faef8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0e59c700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf100afea0 0x7faef8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0f59e700 1 -- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf100afea0 msgr2=0x7faef8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0f59e700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faf100afea0 0x7faef8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0f59e700 1 -- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef8003a00 msgr2=0x7faef8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0f59e700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef8003a00 0x7faef8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.236+0000 7faf0f59e700 1 -- 172.21.15.97:0/3167461340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80035b0 con 0x7faef8004d90 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf0f59e700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef8004d90 0x7faef8005210 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7faf08009fc0 tx=0x7faef8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3438e1f4344ad36 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800ec30 con 0x7faef8004d90 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0800f040 con 0x7faef8004d90 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf0dd9b700 1 -- 172.21.15.97:0/3167461340 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08014480 con 0x7faef8004d90 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf150cb700 1 -- 172.21.15.97:0/3167461340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef8004d90 msgr2=0x7faef8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf150cb700 1 --2- 172.21.15.97:0/3167461340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef8004d90 0x7faef8005210 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7faf08009fc0 tx=0x7faef8005880 comp rx=0 tx=0).stop 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf0f59e700 1 -- 172.21.15.97:0/3167461340 reap_dead start 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf150cb700 1 -- 172.21.15.97:0/3167461340 shutdown_connections 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf150cb700 1 -- 172.21.15.97:0/3167461340 >> 172.21.15.97:0/3167461340 conn(0x7faf10072960 msgr2=0x7faf1006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf150cb700 1 -- 172.21.15.97:0/3167461340 shutdown_connections 2023-11-24T06:11:49.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.237+0000 7faf150cb700 1 -- 172.21.15.97:0/3167461340 wait complete. 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.239+0000 7faf150cb700 1 Processor -- start 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.239+0000 7faf150cb700 1 -- start start 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.239+0000 7faf150cb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faef8004d90 0x7faf100c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.239+0000 7faf150cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf101f7e30 0x7faf101f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.239+0000 7faf150cb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf100ce810 0x7faf100cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.239+0000 7faf150cb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100d00c0 con 0x7faf101f7e30 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.239+0000 7faf150cb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100cf540 con 0x7faf100ce810 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.239+0000 7faf150cb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100cf820 con 0x7faef8004d90 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf101f7e30 0x7faf101f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf101f7e30 0x7faf101f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50464/0 (socket says 172.21.15.97:50464) 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 -- 172.21.15.97:0/657493159 learned_addr learned my addr 172.21.15.97:0/657493159 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:49.248 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0ed9d700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faef8004d90 0x7faf100c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0f59e700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf100ce810 0x7faf100cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 -- 172.21.15.97:0/657493159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faef8004d90 msgr2=0x7faf100c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faef8004d90 0x7faf100c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 -- 172.21.15.97:0/657493159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf100ce810 msgr2=0x7faf100cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf100ce810 0x7faf100cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 -- 172.21.15.97:0/657493159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf100cfb00 con 0x7faf101f7e30 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf0e59c700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf101f7e30 0x7faf101f82b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7faf00000c00 tx=0x7faf0000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faef77fe700 1 -- 172.21.15.97:0/657493159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00012590 con 0x7faf101f7e30 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faef77fe700 1 -- 172.21.15.97:0/657493159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf000126f0 con 0x7faf101f7e30 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf100cfde0 con 0x7faf101f7e30 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faef77fe700 1 -- 172.21.15.97:0/657493159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00012aa0 con 0x7faf101f7e30 2023-11-24T06:11:49.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.240+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf100cf1d0 con 0x7faf101f7e30 2023-11-24T06:11:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.241+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf10148e20 con 0x7faf101f7e30 2023-11-24T06:11:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.241+0000 7faef77fe700 1 -- 172.21.15.97:0/657493159 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7faf00012c00 con 0x7faf101f7e30 2023-11-24T06:11:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.242+0000 7faef77fe700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faefc06b350 0x7faefc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.242+0000 7faef77fe700 1 -- 172.21.15.97:0/657493159 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7faf00089c60 con 0x7faf101f7e30 2023-11-24T06:11:49.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.242+0000 7faf0ed9d700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faefc06b350 0x7faefc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.242+0000 7faf0ed9d700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faefc06b350 0x7faefc06d810 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faf04004670 tx=0x7faf04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.246+0000 7faef77fe700 1 -- 172.21.15.97:0/657493159 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf000546e0 con 0x7faf101f7e30 2023-11-24T06:11:49.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.438+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf100b62f0 con 0x7faf101f7e30 2023-11-24T06:11:49.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.438+0000 7faef77fe700 1 -- 172.21.15.97:0/657493159 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf00057d00 con 0x7faf101f7e30 2023-11-24T06:11:49.442 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:11:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faefc06b350 msgr2=0x7faefc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faefc06b350 0x7faefc06d810 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faf04004670 tx=0x7faf04008040 comp rx=0 tx=0).stop 2023-11-24T06:11:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf101f7e30 msgr2=0x7faf101f82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.444 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf101f7e30 0x7faf101f82b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7faf00000c00 tx=0x7faf0000eb70 comp rx=0 tx=0).stop 2023-11-24T06:11:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 shutdown_connections 2023-11-24T06:11:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf100ce810 0x7faf100cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faefc06b350 0x7faefc06d810 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.445 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faef8004d90 0x7faf100c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 --2- 172.21.15.97:0/657493159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf101f7e30 0x7faf101f82b0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 >> 172.21.15.97:0/657493159 conn(0x7faf10072960 msgr2=0x7faf100b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 shutdown_connections 2023-11-24T06:11:49.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.440+0000 7faf150cb700 1 -- 172.21.15.97:0/657493159 wait complete. 2023-11-24T06:11:49.456 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:49.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.578+0000 7fcad539f700 1 Processor -- start 2023-11-24T06:11:49.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.578+0000 7fcad539f700 1 -- start start 2023-11-24T06:11:49.587 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcad539f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcad015e1d0 0x7fcad00d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcad539f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad00d1aa0 con 0x7fcad00b5950 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcad539f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad00d2ff0 con 0x7fcad00cf240 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcad539f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad00d3b40 con 0x7fcad015e1d0 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacf7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcad00b5950 0x7fcad00b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42926/0 (socket says 172.21.15.97:42926) 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacf7fe700 1 -- 172.21.15.97:0/2215851246 learned_addr learned my addr 172.21.15.97:0/2215851246 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196149719 0 0) 0x7fcad00d1aa0 con 0x7fcad00b5950 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcaceffd700 1 --2- 172.21.15.97:0/2215851246 >> v2:172.21.15.183:3300/0 conn(0x7fcad015e1d0 0x7fcad00d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab80035b0 con 0x7fcad00b5950 2023-11-24T06:11:49.588 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3511706777 0 0) 0x7fcab80035b0 con 0x7fcad00b5950 2023-11-24T06:11:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 >> v2:172.21.15.183:3300/0 conn(0x7fcad015e1d0 msgr2=0x7fcad00d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacdffb700 1 --2- 172.21.15.97:0/2215851246 >> v2:172.21.15.183:3300/0 conn(0x7fcad015e1d0 0x7fcad00d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 >> v1:172.21.15.183:6789/0 conn(0x7fcad00cf240 legacy=0x7fcad00d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad00d1cf0 con 0x7fcad00b5950 2023-11-24T06:11:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.579+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fcac80019f0 con 0x7fcad00b5950 2023-11-24T06:11:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 >> v1:172.21.15.97:6789/0 conn(0x7fcad00b5950 legacy=0x7fcad00b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.589 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcab8003a00 0x7fcab8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8004850 0x7fcab8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab8005330 0x7fcab800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad00d3b40 con 0x7fcab8005330 2023-11-24T06:11:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab800c200 con 0x7fcab8004850 2023-11-24T06:11:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab800c3c0 con 0x7fcab8003a00 2023-11-24T06:11:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacf7fe700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab8005330 0x7fcab800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcaceffd700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcab8003a00 0x7fcab8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcace7fc700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8004850 0x7fcab8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacf7fe700 1 -- 172.21.15.97:0/2215851246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcab8003a00 msgr2=0x7fcab8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacf7fe700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcab8003a00 0x7fcab8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacf7fe700 1 -- 172.21.15.97:0/2215851246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8004850 msgr2=0x7fcab8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacf7fe700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8004850 0x7fcab8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacf7fe700 1 -- 172.21.15.97:0/2215851246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80035b0 con 0x7fcab8005330 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacf7fe700 1 -- 172.21.15.97:0/2215851246 reap_dead start 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacf7fe700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab8005330 0x7fcab800ba10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcac800a7f0 tx=0x7fcab8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec9a03b1349321f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.591 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac800a2f0 con 0x7fcab8005330 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac8012600 con 0x7fcab8005330 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.580+0000 7fcacdffb700 1 -- 172.21.15.97:0/2215851246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac80129b0 con 0x7fcab8005330 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.581+0000 7fcad539f700 1 -- 172.21.15.97:0/2215851246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab8005330 msgr2=0x7fcab800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.581+0000 7fcad539f700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab8005330 0x7fcab800ba10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fcac800a7f0 tx=0x7fcab8005880 comp rx=0 tx=0).stop 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.581+0000 7fcad539f700 1 -- 172.21.15.97:0/2215851246 shutdown_connections 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.581+0000 7fcad539f700 1 --2- 172.21.15.97:0/2215851246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcab8005330 0x7fcab800ba10 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.581+0000 7fcad539f700 1 -- 172.21.15.97:0/2215851246 >> 172.21.15.97:0/2215851246 conn(0x7fcad00d0950 msgr2=0x7fcad00d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.581+0000 7fcad539f700 1 -- 172.21.15.97:0/2215851246 shutdown_connections 2023-11-24T06:11:49.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.581+0000 7fcad539f700 1 -- 172.21.15.97:0/2215851246 wait complete. 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.582+0000 7fcad539f700 1 Processor -- start 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.582+0000 7fcad539f700 1 -- start start 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcad539f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8005330 0x7fcad015cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcad539f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad0160ca0 0x7fcad0161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcad539f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcad00c8b30 0x7fcad00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcad539f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad0160880 con 0x7fcad0160ca0 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcad539f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad01f49c0 con 0x7fcab8005330 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcad539f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad015fc40 con 0x7fcad00c8b30 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad0160ca0 0x7fcad0161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad0160ca0 0x7fcad0161120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50482/0 (socket says 172.21.15.97:50482) 2023-11-24T06:11:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 -- 172.21.15.97:0/817887570 learned_addr learned my addr 172.21.15.97:0/817887570 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcacf7fe700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcad00c8b30 0x7fcad00c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcaceffd700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8005330 0x7fcad015cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 -- 172.21.15.97:0/817887570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcad00c8b30 msgr2=0x7fcad00c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcad00c8b30 0x7fcad00c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 -- 172.21.15.97:0/817887570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8005330 msgr2=0x7fcad015cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8005330 0x7fcad015cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 -- 172.21.15.97:0/817887570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad015fee0 con 0x7fcad0160ca0 2023-11-24T06:11:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.583+0000 7fcace7fc700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad0160ca0 0x7fcad0161120 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcac000f610 tx=0x7fcac000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.584+0000 7fcab77fe700 1 -- 172.21.15.97:0/817887570 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac00052e0 con 0x7fcad0160ca0 2023-11-24T06:11:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.584+0000 7fcab77fe700 1 -- 172.21.15.97:0/817887570 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac0005440 con 0x7fcad0160ca0 2023-11-24T06:11:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.584+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad01601c0 con 0x7fcad0160ca0 2023-11-24T06:11:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.584+0000 7fcab77fe700 1 -- 172.21.15.97:0/817887570 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac00038a0 con 0x7fcad0160ca0 2023-11-24T06:11:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.584+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad015d260 con 0x7fcad0160ca0 2023-11-24T06:11:49.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.585+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad0068fb0 con 0x7fcad0160ca0 2023-11-24T06:11:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.585+0000 7fcab77fe700 1 -- 172.21.15.97:0/817887570 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7fcac0003a00 con 0x7fcad0160ca0 2023-11-24T06:11:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.586+0000 7fcab77fe700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcabc06b350 0x7fcabc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.596 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.586+0000 7fcaceffd700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcabc06b350 0x7fcabc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.586+0000 7fcab77fe700 1 -- 172.21.15.97:0/817887570 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcac00898f0 con 0x7fcad0160ca0 2023-11-24T06:11:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.586+0000 7fcaceffd700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcabc06b350 0x7fcabc06d810 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009be0 tx=0x7fcac4009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.590+0000 7fcab77fe700 1 -- 172.21.15.97:0/817887570 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcac0054440 con 0x7fcad0160ca0 2023-11-24T06:11:49.783 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.781+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcad00c77e0 con 0x7fcad0160ca0 2023-11-24T06:11:49.784 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.781+0000 7fcab77fe700 1 -- 172.21.15.97:0/817887570 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcac0003cb0 con 0x7fcad0160ca0 2023-11-24T06:11:49.784 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:11:49.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcabc06b350 msgr2=0x7fcabc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcabc06b350 0x7fcabc06d810 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009be0 tx=0x7fcac4009210 comp rx=0 tx=0).stop 2023-11-24T06:11:49.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad0160ca0 msgr2=0x7fcad0161120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.786 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad0160ca0 0x7fcad0161120 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fcac000f610 tx=0x7fcac000f920 comp rx=0 tx=0).stop 2023-11-24T06:11:49.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 shutdown_connections 2023-11-24T06:11:49.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcabc06b350 0x7fcabc06d810 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcad00c8b30 0x7fcad00c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcab8005330 0x7fcad015cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 --2- 172.21.15.97:0/817887570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad0160ca0 0x7fcad0161120 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 >> 172.21.15.97:0/817887570 conn(0x7fcad00d0950 msgr2=0x7fcad00ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 shutdown_connections 2023-11-24T06:11:49.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.783+0000 7fcad539f700 1 -- 172.21.15.97:0/817887570 wait complete. 2023-11-24T06:11:49.798 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:49.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.915+0000 7f4a42256700 1 Processor -- start 2023-11-24T06:11:49.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.915+0000 7f4a42256700 1 -- start start 2023-11-24T06:11:49.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.916+0000 7f4a42256700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4a3c0d1570 0x7f4a3c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.916+0000 7f4a42256700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c0d2d10 con 0x7f4a3c0b2fc0 2023-11-24T06:11:49.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.916+0000 7f4a42256700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c0cefd0 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.916+0000 7f4a42256700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3c0cf2b0 con 0x7f4a3c0d1570 2023-11-24T06:11:49.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.916+0000 7f4a3bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4a3c0b2fc0 0x7f4a3c0d3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42940/0 (socket says 172.21.15.97:42940) 2023-11-24T06:11:49.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.916+0000 7f4a3bfff700 1 -- 172.21.15.97:0/4140970847 learned_addr learned my addr 172.21.15.97:0/4140970847 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:49.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.916+0000 7f4a3affd700 1 --2- 172.21.15.97:0/4140970847 >> v2:172.21.15.183:3300/0 conn(0x7f4a3c0d1570 0x7f4a3c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3a7fc700 1 -- 172.21.15.97:0/4140970847 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262096945 0 0) 0x7f4a3c0d2d10 con 0x7f4a3c0b2fc0 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3a7fc700 1 -- 172.21.15.97:0/4140970847 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a280035b0 con 0x7f4a3c0b2fc0 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3a7fc700 1 -- 172.21.15.97:0/4140970847 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832826483 0 0) 0x7f4a3c0cefd0 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3a7fc700 1 -- 172.21.15.97:0/4140970847 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a3c0d2d10 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3a7fc700 1 -- 172.21.15.97:0/4140970847 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524974015 0 0) 0x7f4a280035b0 con 0x7f4a3c0b2fc0 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3affd700 1 -- 172.21.15.97:0/4140970847 >> v1:172.21.15.183:6789/0 conn(0x7f4a3c15dfc0 legacy=0x7f4a3c0d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3affd700 1 -- 172.21.15.97:0/4140970847 >> v1:172.21.15.97:6789/0 conn(0x7f4a3c0b2fc0 legacy=0x7f4a3c0d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3affd700 1 -- 172.21.15.97:0/4140970847 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0d2fb0 con 0x7f4a3c0d1570 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3affd700 1 --2- 172.21.15.97:0/4140970847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0d1570 0x7f4a3c0d34d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009840 tx=0x7f4a2c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37366e9dd0578f9b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3a7fc700 1 -- 172.21.15.97:0/4140970847 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a340019f0 con 0x7f4a3c0d1570 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3a7fc700 1 -- 172.21.15.97:0/4140970847 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c004030 con 0x7f4a3c0d1570 2023-11-24T06:11:49.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.917+0000 7f4a3a7fc700 1 -- 172.21.15.97:0/4140970847 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0043c0 con 0x7f4a3c0d1570 2023-11-24T06:11:49.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.918+0000 7f4a42256700 1 -- 172.21.15.97:0/4140970847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0d1570 msgr2=0x7f4a3c0d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.918+0000 7f4a42256700 1 --2- 172.21.15.97:0/4140970847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0d1570 0x7f4a3c0d34d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009840 tx=0x7f4a2c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.918+0000 7f4a42256700 1 -- 172.21.15.97:0/4140970847 shutdown_connections 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.918+0000 7f4a42256700 1 --2- 172.21.15.97:0/4140970847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0d1570 0x7f4a3c0d34d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.918+0000 7f4a42256700 1 -- 172.21.15.97:0/4140970847 >> 172.21.15.97:0/4140970847 conn(0x7f4a3c06b410 msgr2=0x7f4a3c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.918+0000 7f4a42256700 1 -- 172.21.15.97:0/4140970847 shutdown_connections 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.918+0000 7f4a42256700 1 -- 172.21.15.97:0/4140970847 wait complete. 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.919+0000 7f4a42256700 1 Processor -- start 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.919+0000 7f4a42256700 1 -- start start 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a42256700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0b2fc0 0x7f4a3c0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a42256700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a3c0d1570 0x7f4a3c14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a42256700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a3c15dfc0 0x7f4a3c0cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a42256700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0cb9b0 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a42256700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0cce60 con 0x7f4a3c0b2fc0 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a42256700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0cd140 con 0x7f4a3c0d1570 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a3c15dfc0 0x7f4a3c0cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a3c15dfc0 0x7f4a3c0cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50496/0 (socket says 172.21.15.97:50496) 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3bfff700 1 -- 172.21.15.97:0/2358758602 learned_addr learned my addr 172.21.15.97:0/2358758602 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3b7fe700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0b2fc0 0x7f4a3c0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3affd700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a3c0d1570 0x7f4a3c14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3bfff700 1 -- 172.21.15.97:0/2358758602 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a3c0d1570 msgr2=0x7f4a3c14b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3bfff700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a3c0d1570 0x7f4a3c14b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3bfff700 1 -- 172.21.15.97:0/2358758602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0b2fc0 msgr2=0x7f4a3c0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3bfff700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0b2fc0 0x7f4a3c0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.920+0000 7f4a3bfff700 1 -- 172.21.15.97:0/2358758602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c0094a0 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.921+0000 7f4a3bfff700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a3c15dfc0 0x7f4a3c0cc400 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400cba0 tx=0x7f4a3400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.921+0000 7f4a33fff700 1 -- 172.21.15.97:0/2358758602 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400aaa0 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.921+0000 7f4a33fff700 1 -- 172.21.15.97:0/2358758602 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3400ac20 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.921+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c0cd420 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.921+0000 7f4a33fff700 1 -- 172.21.15.97:0/2358758602 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a340070d0 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.921+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c0cbee0 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.922+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c067f60 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.927 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.922+0000 7f4a33fff700 1 -- 172.21.15.97:0/2358758602 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f4a34007230 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.922+0000 7f4a33fff700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a1c06b350 0x7f4a1c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.923+0000 7f4a33fff700 1 -- 172.21.15.97:0/2358758602 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4a340899c0 con 0x7f4a3c15dfc0 2023-11-24T06:11:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.923+0000 7f4a3b7fe700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a1c06b350 0x7f4a1c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:49.928 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.923+0000 7f4a3b7fe700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a1c06b350 0x7f4a1c06d810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4a240011e0 tx=0x7f4a24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:49.929 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:49.927+0000 7f4a33fff700 1 -- 172.21.15.97:0/2358758602 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a34054310 con 0x7f4a3c15dfc0 2023-11-24T06:11:50.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.132+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4a3c0cdd00 con 0x7f4a3c15dfc0 2023-11-24T06:11:50.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.132+0000 7f4a33fff700 1 -- 172.21.15.97:0/2358758602 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4a34018080 con 0x7f4a3c15dfc0 2023-11-24T06:11:50.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.134+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a1c06b350 msgr2=0x7f4a1c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.134+0000 7f4a42256700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a1c06b350 0x7f4a1c06d810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4a240011e0 tx=0x7f4a24009040 comp rx=0 tx=0).stop 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.134+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a3c15dfc0 msgr2=0x7f4a3c0cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.134+0000 7f4a42256700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a3c15dfc0 0x7f4a3c0cc400 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400cba0 tx=0x7f4a3400ceb0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.135+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 shutdown_connections 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.135+0000 7f4a42256700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a3c15dfc0 0x7f4a3c0cc400 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.135+0000 7f4a42256700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a3c0b2fc0 0x7f4a3c0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.135+0000 7f4a42256700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4a1c06b350 0x7f4a1c06d810 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.135+0000 7f4a42256700 1 --2- 172.21.15.97:0/2358758602 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4a3c0d1570 0x7f4a3c14b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.135+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 >> 172.21.15.97:0/2358758602 conn(0x7f4a3c06b410 msgr2=0x7f4a3c0ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:50.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.135+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 shutdown_connections 2023-11-24T06:11:50.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.135+0000 7f4a42256700 1 -- 172.21.15.97:0/2358758602 wait complete. 2023-11-24T06:11:50.139 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:11:50.149 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:11:50.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:50.150 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T06:11:50.191 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:11:50.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:50.192 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:11:50.248 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:50.248 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:50.270 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:50.270 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:50.327 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.445+0000 7fe5b71f5700 1 Processor -- start 2023-11-24T06:11:50.448 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.445+0000 7fe5b71f5700 1 -- start start 2023-11-24T06:11:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.446+0000 7fe5b71f5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5b00adf00 0x7fe5b006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.446+0000 7fe5b71f5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b00af950 con 0x7fe5b01446a0 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.446+0000 7fe5b71f5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b00afc30 con 0x7fe5b006c540 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.446+0000 7fe5b71f5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b006c9f0 con 0x7fe5b00adf00 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.446+0000 7fe5affff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5b00adf00 0x7fe5b006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.446+0000 7fe5b4f91700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe5b01446a0 0x7fe5b00b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42952/0 (socket says 172.21.15.97:42952) 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.446+0000 7fe5b4f91700 1 -- 172.21.15.97:0/2884115439 learned_addr learned my addr 172.21.15.97:0/2884115439 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5af7fe700 1 -- 172.21.15.97:0/2884115439 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394969463 0 0) 0x7fe5b00af950 con 0x7fe5b01446a0 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5af7fe700 1 -- 172.21.15.97:0/2884115439 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5a00035b0 con 0x7fe5b01446a0 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5affff700 1 -- 172.21.15.97:0/2884115439 >> v1:172.21.15.183:6789/0 conn(0x7fe5b006c540 legacy=0x7fe5b00b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5affff700 1 -- 172.21.15.97:0/2884115439 >> v1:172.21.15.97:6789/0 conn(0x7fe5b01446a0 legacy=0x7fe5b00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5affff700 1 -- 172.21.15.97:0/2884115439 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00aff10 con 0x7fe5b00adf00 2023-11-24T06:11:50.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5affff700 1 --2- 172.21.15.97:0/2884115439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b006c000 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009840 tx=0x7fe59c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f9a237b02c011a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5af7fe700 1 -- 172.21.15.97:0/2884115439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c011830 con 0x7fe5b00adf00 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5af7fe700 1 -- 172.21.15.97:0/2884115439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c011990 con 0x7fe5b00adf00 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.447+0000 7fe5af7fe700 1 -- 172.21.15.97:0/2884115439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c011d20 con 0x7fe5b00adf00 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.448+0000 7fe5b71f5700 1 -- 172.21.15.97:0/2884115439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 msgr2=0x7fe5b006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.448+0000 7fe5b71f5700 1 --2- 172.21.15.97:0/2884115439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b006c000 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009840 tx=0x7fe59c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.448+0000 7fe5b71f5700 1 -- 172.21.15.97:0/2884115439 shutdown_connections 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.448+0000 7fe5b71f5700 1 --2- 172.21.15.97:0/2884115439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b006c000 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.448+0000 7fe5b71f5700 1 -- 172.21.15.97:0/2884115439 >> 172.21.15.97:0/2884115439 conn(0x7fe5b0071b90 msgr2=0x7fe5b0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.448+0000 7fe5b71f5700 1 -- 172.21.15.97:0/2884115439 shutdown_connections 2023-11-24T06:11:50.452 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.448+0000 7fe5b71f5700 1 -- 172.21.15.97:0/2884115439 wait complete. 2023-11-24T06:11:50.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.449+0000 7fe5b71f5700 1 Processor -- start 2023-11-24T06:11:50.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.449+0000 7fe5b71f5700 1 -- start start 2023-11-24T06:11:50.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5b71f5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5b006c540 0x7fe5b015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5b71f5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b0143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5b71f5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b01446a0 0x7fe5b01437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5b71f5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b015bcc0 con 0x7fe5b00adf00 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5b71f5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b015e2f0 con 0x7fe5b01446a0 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5b71f5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b015b340 con 0x7fe5b006c540 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b0143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b0143280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50506/0 (socket says 172.21.15.97:50506) 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 -- 172.21.15.97:0/1000729183 learned_addr learned my addr 172.21.15.97:0/1000729183 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5b5792700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b01446a0 0x7fe5b01437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5b4f91700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5b006c540 0x7fe5b015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 -- 172.21.15.97:0/1000729183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5b006c540 msgr2=0x7fe5b015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5b006c540 0x7fe5b015ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 -- 172.21.15.97:0/1000729183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b01446a0 msgr2=0x7fe5b01437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b01446a0 0x7fe5b01437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 -- 172.21.15.97:0/1000729183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0094a0 con 0x7fe5b00adf00 2023-11-24T06:11:50.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.450+0000 7fe5affff700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b0143280 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009fd0 tx=0x7fe59c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.451+0000 7fe5ad7fa700 1 -- 172.21.15.97:0/1000729183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0036a0 con 0x7fe5b00adf00 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.451+0000 7fe5ad7fa700 1 -- 172.21.15.97:0/1000729183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c01f070 con 0x7fe5b00adf00 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.451+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b00d0330 con 0x7fe5b00adf00 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.451+0000 7fe5ad7fa700 1 -- 172.21.15.97:0/1000729183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c003a20 con 0x7fe5b00adf00 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.451+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b015b8e0 con 0x7fe5b00adf00 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.452+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b00753f0 con 0x7fe5b00adf00 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.452+0000 7fe5ad7fa700 1 -- 172.21.15.97:0/1000729183 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe59c003b80 con 0x7fe5b00adf00 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.452+0000 7fe5ad7fa700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe59406b4a0 0x7fe59406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.452+0000 7fe5ad7fa700 1 -- 172.21.15.97:0/1000729183 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe59c021070 con 0x7fe5b00adf00 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.452+0000 7fe5b4f91700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe59406b4a0 0x7fe59406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.453+0000 7fe5b4f91700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe59406b4a0 0x7fe59406d960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4007fd0 tx=0x7fe5a4007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.457+0000 7fe5ad7fa700 1 -- 172.21.15.97:0/1000729183 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe59c056020 con 0x7fe5b00adf00 2023-11-24T06:11:50.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.653+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5b015be00 con 0x7fe5b00adf00 2023-11-24T06:11:50.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.654+0000 7fe5ad7fa700 1 -- 172.21.15.97:0/1000729183 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16623 (secure 0 0 0) 0x7fe5b015be00 con 0x7fe5b00adf00 2023-11-24T06:11:50.657 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:50.657 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":72,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:37.197270+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:33.973262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:34.541598+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe59406b4a0 msgr2=0x7fe59406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe59406b4a0 0x7fe59406d960 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4007fd0 tx=0x7fe5a4007dc0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 msgr2=0x7fe5b0143280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b0143280 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009fd0 tx=0x7fe59c004750 comp rx=0 tx=0).stop 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 shutdown_connections 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5b01446a0 0x7fe5b01437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe59406b4a0 0x7fe59406d960 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe5b006c540 0x7fe5b015ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 --2- 172.21.15.97:0/1000729183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5b00adf00 0x7fe5b0143280 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 >> 172.21.15.97:0/1000729183 conn(0x7fe5b0071b90 msgr2=0x7fe5b00b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:50.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 shutdown_connections 2023-11-24T06:11:50.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.657+0000 7fe5b71f5700 1 -- 172.21.15.97:0/1000729183 wait complete. 2023-11-24T06:11:50.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:33.973262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:34.541598+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:50.672 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.792+0000 7f6a3bad6700 1 Processor -- start 2023-11-24T06:11:50.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.793+0000 7f6a3bad6700 1 -- start start 2023-11-24T06:11:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a3bad6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6a3415b370 0x7f6a3415b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a3bad6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a340ae5e0 con 0x7f6a341587f0 2023-11-24T06:11:50.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a3bad6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3415be30 con 0x7f6a340b3c60 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a3bad6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3415c110 con 0x7f6a3415b370 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a39872700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6a341587f0 0x7f6a3406aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42962/0 (socket says 172.21.15.97:42962) 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a39872700 1 -- 172.21.15.97:0/91909439 learned_addr learned my addr 172.21.15.97:0/91909439 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950611886 0 0) 0x7f6a340ae5e0 con 0x7f6a341587f0 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a39071700 1 --2- 172.21.15.97:0/91909439 >> v2:172.21.15.183:3300/0 conn(0x7f6a3415b370 0x7f6a3415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a200035b0 con 0x7f6a341587f0 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907781553 0 0) 0x7f6a200035b0 con 0x7f6a341587f0 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 >> v2:172.21.15.183:3300/0 conn(0x7f6a3415b370 msgr2=0x7f6a3415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 --2- 172.21.15.97:0/91909439 >> v2:172.21.15.183:3300/0 conn(0x7f6a3415b370 0x7f6a3415b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 >> v1:172.21.15.183:6789/0 conn(0x7f6a340b3c60 legacy=0x7f6a340af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340ae860 con 0x7f6a341587f0 2023-11-24T06:11:50.800 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f6a24001670 con 0x7f6a341587f0 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 >> v1:172.21.15.97:6789/0 conn(0x7f6a341587f0 legacy=0x7f6a3406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20003a30 0x7f6a20003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a20004850 0x7f6a20004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a20005210 0x7f6a2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3415c110 con 0x7f6a20005210 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2000c0e0 con 0x7f6a20004850 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2000c2a0 con 0x7f6a20003a30 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.794+0000 7f6a3a073700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a20005210 0x7f6a2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a39071700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a20004850 0x7f6a20004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a39872700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20003a30 0x7f6a20003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3a073700 1 -- 172.21.15.97:0/91909439 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20003a30 msgr2=0x7f6a20003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:50.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3a073700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20003a30 0x7f6a20003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3a073700 1 -- 172.21.15.97:0/91909439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a20004850 msgr2=0x7f6a20004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3a073700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a20004850 0x7f6a20004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3a073700 1 -- 172.21.15.97:0/91909439 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200035b0 con 0x7f6a20005210 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3a073700 1 -- 172.21.15.97:0/91909439 reap_dead start 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3a073700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a20005210 0x7f6a2000ba10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c190 tx=0x7f6a20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286296df9fb4a7d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c001db0 con 0x7f6a20005210 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c00ccf0 con 0x7f6a20005210 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a38870700 1 -- 172.21.15.97:0/91909439 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c003c40 con 0x7f6a20005210 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3bad6700 1 -- 172.21.15.97:0/91909439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a20005210 msgr2=0x7f6a2000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.795+0000 7f6a3bad6700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a20005210 0x7f6a2000ba10 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00c190 tx=0x7f6a20005880 comp rx=0 tx=0).stop 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.796+0000 7f6a3bad6700 1 -- 172.21.15.97:0/91909439 shutdown_connections 2023-11-24T06:11:50.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.796+0000 7f6a3bad6700 1 --2- 172.21.15.97:0/91909439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a20005210 0x7f6a2000ba10 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.796+0000 7f6a3bad6700 1 -- 172.21.15.97:0/91909439 >> 172.21.15.97:0/91909439 conn(0x7f6a340d0bf0 msgr2=0x7f6a340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.796+0000 7f6a3bad6700 1 -- 172.21.15.97:0/91909439 shutdown_connections 2023-11-24T06:11:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.796+0000 7f6a3bad6700 1 -- 172.21.15.97:0/91909439 wait complete. 2023-11-24T06:11:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.797+0000 7f6a3bad6700 1 Processor -- start 2023-11-24T06:11:50.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.797+0000 7f6a3bad6700 1 -- start start 2023-11-24T06:11:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.797+0000 7f6a3bad6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20005210 0x7f6a341540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3bad6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a341445d0 0x7f6a340aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3bad6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a3414f480 0x7f6a3415d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3bad6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340af3f0 con 0x7f6a3414f480 2023-11-24T06:11:50.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3bad6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a34144a50 con 0x7f6a341445d0 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3bad6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340af620 con 0x7f6a20005210 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a3414f480 0x7f6a3415d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a3414f480 0x7f6a3415d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50526/0 (socket says 172.21.15.97:50526) 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 -- 172.21.15.97:0/243153458 learned_addr learned my addr 172.21.15.97:0/243153458 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a39071700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a341445d0 0x7f6a340aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a39872700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20005210 0x7f6a341540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 -- 172.21.15.97:0/243153458 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20005210 msgr2=0x7f6a341540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20005210 0x7f6a341540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 -- 172.21.15.97:0/243153458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a341445d0 msgr2=0x7f6a340aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a341445d0 0x7f6a340aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.806 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 -- 172.21.15.97:0/243153458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340af900 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3a073700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a3414f480 0x7f6a3415d4d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6a340b4a50 tx=0x7f6a2c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/243153458 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c003e10 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/243153458 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c005070 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/243153458 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0053e0 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a340afbe0 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.798+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a340b01a0 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.799+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a340b0330 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.800+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/243153458 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6a2c005540 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.800+0000 7f6a2a7fc700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a1806b470 0x7f6a1806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.800+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/243153458 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6a2c08b380 con 0x7f6a3414f480 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.800+0000 7f6a39872700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a1806b470 0x7f6a1806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:50.807 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.800+0000 7f6a39872700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a1806b470 0x7f6a1806d930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:50.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.804+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/243153458 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a2c055d50 con 0x7f6a3414f480 2023-11-24T06:11:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.994+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a34068fb0 con 0x7f6a3414f480 2023-11-24T06:11:50.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.994+0000 7f6a2a7fc700 1 -- 172.21.15.97:0/243153458 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a2c059370 con 0x7f6a3414f480 2023-11-24T06:11:50.997 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a1806b470 msgr2=0x7f6a1806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a1806b470 0x7f6a1806d930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000c00 tx=0x7f6a24007980 comp rx=0 tx=0).stop 2023-11-24T06:11:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a3414f480 msgr2=0x7f6a3415d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a3414f480 0x7f6a3415d4d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6a340b4a50 tx=0x7f6a2c0014a0 comp rx=0 tx=0).stop 2023-11-24T06:11:50.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 shutdown_connections 2023-11-24T06:11:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a3414f480 0x7f6a3415d4d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a1806b470 0x7f6a1806d930 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a20005210 0x7f6a341540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 --2- 172.21.15.97:0/243153458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a341445d0 0x7f6a340aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 >> 172.21.15.97:0/243153458 conn(0x7f6a340d0bf0 msgr2=0x7f6a341571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 shutdown_connections 2023-11-24T06:11:51.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:50.996+0000 7f6a3bad6700 1 -- 172.21.15.97:0/243153458 wait complete. 2023-11-24T06:11:51.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.131+0000 7f247d63e700 1 Processor -- start 2023-11-24T06:11:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.131+0000 7f247d63e700 1 -- start start 2023-11-24T06:11:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f247d63e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f24780746a0 0x7f24780b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f247d63e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24780afc90 0x7f247806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f247d63e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24780b3c60 con 0x7f247815e160 2023-11-24T06:11:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f247d63e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24780b4500 con 0x7f24780afc90 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f247d63e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24780b58c0 con 0x7f24780746a0 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f24777fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24780afc90 0x7f247806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f24777fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24780afc90 0x7f247806d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50534/0 (socket says 172.21.15.97:50534) 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f24777fe700 1 -- 172.21.15.97:0/3302627997 learned_addr learned my addr 172.21.15.97:0/3302627997 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f2476ffd700 1 --1- 172.21.15.97:0/3302627997 >> v1:172.21.15.97:6789/0 conn(0x7f247815e160 0x7f2478073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:42982/0 (socket says 172.21.15.97:42982) 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f24777fe700 1 -- 172.21.15.97:0/3302627997 >> v2:172.21.15.183:3300/0 conn(0x7f24780746a0 msgr2=0x7f24780b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f24767fc700 1 --2- 172.21.15.97:0/3302627997 >> v2:172.21.15.183:3300/0 conn(0x7f24780746a0 0x7f24780b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f24777fe700 1 --2- 172.21.15.97:0/3302627997 >> v2:172.21.15.183:3300/0 conn(0x7f24780746a0 0x7f24780b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f24777fe700 1 -- 172.21.15.97:0/3302627997 >> v1:172.21.15.97:6789/0 conn(0x7f247815e160 legacy=0x7f2478073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f2475ffb700 1 -- 172.21.15.97:0/3302627997 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 622132237 0 0) 0x7f24780b3c60 con 0x7f247815e160 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.132+0000 7f24777fe700 1 -- 172.21.15.97:0/3302627997 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247806bdc0 con 0x7f24780afc90 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f24777fe700 1 --2- 172.21.15.97:0/3302627997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780afc90 0x7f247806d930 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f24780d0880 tx=0x7f247000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41920ebc31c3c033 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f2475ffb700 1 -- 172.21.15.97:0/3302627997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000e070 con 0x7f24780afc90 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f2475ffb700 1 -- 172.21.15.97:0/3302627997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247000b9c0 con 0x7f24780afc90 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f2475ffb700 1 -- 172.21.15.97:0/3302627997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000bd70 con 0x7f24780afc90 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f247d63e700 1 -- 172.21.15.97:0/3302627997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780afc90 msgr2=0x7f247806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f247d63e700 1 --2- 172.21.15.97:0/3302627997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780afc90 0x7f247806d930 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f24780d0880 tx=0x7f247000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f247d63e700 1 -- 172.21.15.97:0/3302627997 shutdown_connections 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f247d63e700 1 --2- 172.21.15.97:0/3302627997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780afc90 0x7f247806d930 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f247d63e700 1 --2- 172.21.15.97:0/3302627997 >> v2:172.21.15.183:3300/0 conn(0x7f24780746a0 0x7f24780b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.133+0000 7f247d63e700 1 -- 172.21.15.97:0/3302627997 >> 172.21.15.97:0/3302627997 conn(0x7f24780d0bf0 msgr2=0x7f24780d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:51.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.134+0000 7f247d63e700 1 -- 172.21.15.97:0/3302627997 shutdown_connections 2023-11-24T06:11:51.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.134+0000 7f247d63e700 1 -- 172.21.15.97:0/3302627997 wait complete. 2023-11-24T06:11:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.135+0000 7f247d63e700 1 Processor -- start 2023-11-24T06:11:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.135+0000 7f247d63e700 1 -- start start 2023-11-24T06:11:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f247d63e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780746a0 0x7f2478152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f247d63e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24780afc90 0x7f24781522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f247d63e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f247815e160 0x7f24781565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f247d63e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478154270 con 0x7f24780746a0 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f247d63e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478153660 con 0x7f24780afc90 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f247d63e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2478153940 con 0x7f247815e160 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780746a0 0x7f2478152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780746a0 0x7f2478152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50536/0 (socket says 172.21.15.97:50536) 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 -- 172.21.15.97:0/3069704175 learned_addr learned my addr 172.21.15.97:0/3069704175 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f24777fe700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f247815e160 0x7f24781565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f24767fc700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24780afc90 0x7f24781522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 -- 172.21.15.97:0/3069704175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f247815e160 msgr2=0x7f24781565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f247815e160 0x7f24781565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 -- 172.21.15.97:0/3069704175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24780afc90 msgr2=0x7f24781522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24780afc90 0x7f24781522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 -- 172.21.15.97:0/3069704175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470009b30 con 0x7f24780746a0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f2476ffd700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780746a0 0x7f2478152fd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f246c00c550 tx=0x7f246c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f245f7fe700 1 -- 172.21.15.97:0/3069704175 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c004c80 con 0x7f24780746a0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f245f7fe700 1 -- 172.21.15.97:0/3069704175 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246c004e00 con 0x7f24780746a0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2478153c20 con 0x7f24780746a0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f245f7fe700 1 -- 172.21.15.97:0/3069704175 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c007310 con 0x7f24780746a0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.136+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2478159850 con 0x7f24780746a0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.137+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2478068fb0 con 0x7f24780746a0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.138+0000 7f245f7fe700 1 -- 172.21.15.97:0/3069704175 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f246c007470 con 0x7f24780746a0 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.138+0000 7f245f7fe700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f246406b470 0x7f246406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.138+0000 7f245f7fe700 1 -- 172.21.15.97:0/3069704175 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f246c089df0 con 0x7f24780746a0 2023-11-24T06:11:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.138+0000 7f24767fc700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f246406b470 0x7f246406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.138+0000 7f24767fc700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f246406b470 0x7f246406d930 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f24781f7040 tx=0x7f2468008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.142+0000 7f245f7fe700 1 -- 172.21.15.97:0/3069704175 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f246c054580 con 0x7f24780746a0 2023-11-24T06:11:51.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.335+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f247806c140 con 0x7f24780746a0 2023-11-24T06:11:51.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.335+0000 7f245f7fe700 1 -- 172.21.15.97:0/3069704175 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7f246c057ba0 con 0x7f24780746a0 2023-11-24T06:11:51.338 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:11:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f246406b470 msgr2=0x7f246406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f246406b470 0x7f246406d930 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f24781f7040 tx=0x7f2468008040 comp rx=0 tx=0).stop 2023-11-24T06:11:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780746a0 msgr2=0x7f2478152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780746a0 0x7f2478152fd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f246c00c550 tx=0x7f246c00c910 comp rx=0 tx=0).stop 2023-11-24T06:11:51.340 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 shutdown_connections 2023-11-24T06:11:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f246406b470 0x7f246406d930 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f247815e160 0x7f24781565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24780746a0 0x7f2478152fd0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 --2- 172.21.15.97:0/3069704175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24780afc90 0x7f24781522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 >> 172.21.15.97:0/3069704175 conn(0x7f24780d0bf0 msgr2=0x7f24780b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 shutdown_connections 2023-11-24T06:11:51.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.337+0000 7f247d63e700 1 -- 172.21.15.97:0/3069704175 wait complete. 2023-11-24T06:11:51.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:51.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.473+0000 7fcc5e944700 1 Processor -- start 2023-11-24T06:11:51.475 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.473+0000 7fcc5e944700 1 -- start start 2023-11-24T06:11:51.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5e944700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc581587f0 0x7fcc5806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5e944700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fcc5815b370 0x7fcc5815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5e944700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc580b3c60 0x7fcc580b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5e944700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc580b4610 con 0x7fcc580b3c60 2023-11-24T06:11:51.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5e944700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5806b530 con 0x7fcc5815b370 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5e944700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580ae5e0 con 0x7fcc581587f0 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc580b3c60 0x7fcc580b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcc580b3c60 0x7fcc580b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50548/0 (socket says 172.21.15.97:50548) 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 -- 172.21.15.97:0/732803097 learned_addr learned my addr 172.21.15.97:0/732803097 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc577fe700 1 --2- 172.21.15.97:0/732803097 >> v2:172.21.15.183:3300/0 conn(0x7fcc5815b370 0x7fcc5815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc57fff700 1 --2- 172.21.15.97:0/732803097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc581587f0 0x7fcc5806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 -- 172.21.15.97:0/732803097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc581587f0 msgr2=0x7fcc5806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 --2- 172.21.15.97:0/732803097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc581587f0 0x7fcc5806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 -- 172.21.15.97:0/732803097 >> v2:172.21.15.183:3300/0 conn(0x7fcc5815b370 msgr2=0x7fcc5815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 --2- 172.21.15.97:0/732803097 >> v2:172.21.15.183:3300/0 conn(0x7fcc5815b370 0x7fcc5815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 -- 172.21.15.97:0/732803097 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5815be30 con 0x7fcc580b3c60 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.474+0000 7fcc5cee1700 1 --2- 172.21.15.97:0/732803097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc580b3c60 0x7fcc580b40d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcc58072e10 tx=0x7fcc5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2372b3b9f25922 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc56ffd700 1 -- 172.21.15.97:0/732803097 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000e070 con 0x7fcc580b3c60 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc56ffd700 1 -- 172.21.15.97:0/732803097 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5000b9c0 con 0x7fcc580b3c60 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc56ffd700 1 -- 172.21.15.97:0/732803097 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000bd70 con 0x7fcc580b3c60 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 -- 172.21.15.97:0/732803097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc580b3c60 msgr2=0x7fcc580b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 --2- 172.21.15.97:0/732803097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc580b3c60 0x7fcc580b40d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcc58072e10 tx=0x7fcc5000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 -- 172.21.15.97:0/732803097 shutdown_connections 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 --2- 172.21.15.97:0/732803097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc580b3c60 0x7fcc580b40d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 --2- 172.21.15.97:0/732803097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc581587f0 0x7fcc5806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 --2- 172.21.15.97:0/732803097 >> v2:172.21.15.183:3300/0 conn(0x7fcc5815b370 0x7fcc5815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 -- 172.21.15.97:0/732803097 >> 172.21.15.97:0/732803097 conn(0x7fcc580d0bf0 msgr2=0x7fcc580d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:51.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 -- 172.21.15.97:0/732803097 shutdown_connections 2023-11-24T06:11:51.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.475+0000 7fcc5e944700 1 -- 172.21.15.97:0/732803097 wait complete. 2023-11-24T06:11:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.476+0000 7fcc5e944700 1 Processor -- start 2023-11-24T06:11:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc5e944700 1 -- start start 2023-11-24T06:11:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc5e944700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc580b3c60 0x7fcc58155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc5e944700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc581587f0 0x7fcc5809c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc5e944700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc5815b370 0x7fcc5806eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc5e944700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5809bf70 con 0x7fcc581587f0 2023-11-24T06:11:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc5e944700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5809b360 con 0x7fcc5815b370 2023-11-24T06:11:51.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc5e944700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5809b640 con 0x7fcc580b3c60 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc581587f0 0x7fcc5809c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc577fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc581587f0 0x7fcc5809c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50552/0 (socket says 172.21.15.97:50552) 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.477+0000 7fcc577fe700 1 -- 172.21.15.97:0/984831417 learned_addr learned my addr 172.21.15.97:0/984831417 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc57fff700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc580b3c60 0x7fcc58155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc5cee1700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc5815b370 0x7fcc5806eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc577fe700 1 -- 172.21.15.97:0/984831417 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc580b3c60 msgr2=0x7fcc58155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc577fe700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc580b3c60 0x7fcc58155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc577fe700 1 -- 172.21.15.97:0/984831417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc5815b370 msgr2=0x7fcc5806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc577fe700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc5815b370 0x7fcc5806eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc577fe700 1 -- 172.21.15.97:0/984831417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50009b30 con 0x7fcc581587f0 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc577fe700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc581587f0 0x7fcc5809c400 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00d800 tx=0x7fcc4c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc54ff9700 1 -- 172.21.15.97:0/984831417 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c0125f0 con 0x7fcc581587f0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc54ff9700 1 -- 172.21.15.97:0/984831417 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c012750 con 0x7fcc581587f0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc54ff9700 1 -- 172.21.15.97:0/984831417 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c012b00 con 0x7fcc581587f0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5809b920 con 0x7fcc581587f0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.478+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5809aed0 con 0x7fcc581587f0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.479+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc58068fb0 con 0x7fcc581587f0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.480+0000 7fcc54ff9700 1 -- 172.21.15.97:0/984831417 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fcc4c012c60 con 0x7fcc581587f0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.480+0000 7fcc54ff9700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcc4406b470 0x7fcc4406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.480+0000 7fcc54ff9700 1 -- 172.21.15.97:0/984831417 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcc4c08a0a0 con 0x7fcc581587f0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.480+0000 7fcc57fff700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcc4406b470 0x7fcc4406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.480+0000 7fcc57fff700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcc4406b470 0x7fcc4406d930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcc48006b20 tx=0x7fcc48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.487 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.484+0000 7fcc54ff9700 1 -- 172.21.15.97:0/984831417 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc4c0548c0 con 0x7fcc581587f0 2023-11-24T06:11:51.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.677+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc5809acb0 con 0x7fcc581587f0 2023-11-24T06:11:51.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.677+0000 7fcc54ff9700 1 -- 172.21.15.97:0/984831417 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc4c057ee0 con 0x7fcc581587f0 2023-11-24T06:11:51.680 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:11:51.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcc4406b470 msgr2=0x7fcc4406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcc4406b470 0x7fcc4406d930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcc48006b20 tx=0x7fcc48008040 comp rx=0 tx=0).stop 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc581587f0 msgr2=0x7fcc5809c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc581587f0 0x7fcc5809c400 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00d800 tx=0x7fcc4c00db10 comp rx=0 tx=0).stop 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 shutdown_connections 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc5815b370 0x7fcc5806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fcc4406b470 0x7fcc4406d930 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fcc580b3c60 0x7fcc58155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 --2- 172.21.15.97:0/984831417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc581587f0 0x7fcc5809c400 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.679+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 >> 172.21.15.97:0/984831417 conn(0x7fcc580d0bf0 msgr2=0x7fcc580af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.680+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 shutdown_connections 2023-11-24T06:11:51.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.680+0000 7fcc5e944700 1 -- 172.21.15.97:0/984831417 wait complete. 2023-11-24T06:11:51.694 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:11:51.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.814+0000 7f769a811700 1 Processor -- start 2023-11-24T06:11:51.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.814+0000 7f769a811700 1 -- start start 2023-11-24T06:11:51.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.814+0000 7f769a811700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769415ac50 0x7f769406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f769a811700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f76940b50c0 0x7f76940cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f769a811700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769406bf50 con 0x7f769406f460 2023-11-24T06:11:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f769a811700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769415b030 con 0x7f76940b50c0 2023-11-24T06:11:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f769a811700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769406e810 con 0x7f769415ac50 2023-11-24T06:11:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f76937fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f769406f460 0x7f76940b49b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43010/0 (socket says 172.21.15.97:43010) 2023-11-24T06:11:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f76937fe700 1 -- 172.21.15.97:0/1656609404 learned_addr learned my addr 172.21.15.97:0/1656609404 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:51.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7698dae700 1 --2- 172.21.15.97:0/1656609404 >> v2:172.21.15.183:3300/0 conn(0x7f76940b50c0 0x7f76940cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953158971 0 0) 0x7f769406bf50 con 0x7f769406f460 2023-11-24T06:11:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f767c0035b0 con 0x7f769406f460 2023-11-24T06:11:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7693fff700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769415ac50 0x7f769406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 940551290 0 0) 0x7f767c0035b0 con 0x7f769406f460 2023-11-24T06:11:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769415ac50 msgr2=0x7f769406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769415ac50 0x7f769406c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 >> v2:172.21.15.183:3300/0 conn(0x7f76940b50c0 msgr2=0x7f76940cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 --2- 172.21.15.97:0/1656609404 >> v2:172.21.15.183:3300/0 conn(0x7f76940b50c0 0x7f76940cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769406b340 con 0x7f769406f460 2023-11-24T06:11:51.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f7688001670 con 0x7f769406f460 2023-11-24T06:11:51.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 >> v1:172.21.15.97:6789/0 conn(0x7f769406f460 legacy=0x7f76940b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f767c003a00 0x7f767c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769415ac50 0x7f767c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f767c004d90 0x7f767c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769415b030 con 0x7f767c004d90 2023-11-24T06:11:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769406e810 con 0x7f767c003a00 2023-11-24T06:11:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c00dc20 con 0x7f769415ac50 2023-11-24T06:11:51.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7698dae700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f767c004d90 0x7f767c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.815+0000 7f7693fff700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f767c003a00 0x7f767c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f76937fe700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769415ac50 0x7f767c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7698dae700 1 -- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769415ac50 msgr2=0x7f767c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7698dae700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f769415ac50 0x7f767c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.832 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7698dae700 1 -- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f767c003a00 msgr2=0x7f767c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:51.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7698dae700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f767c003a00 0x7f767c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7698dae700 1 -- 172.21.15.97:0/1656609404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0035b0 con 0x7f767c004d90 2023-11-24T06:11:51.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7698dae700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f767c004d90 0x7f767c005210 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f768c009f90 tx=0x7f767c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca363a2f33763e7e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c00ec30 con 0x7f767c004d90 2023-11-24T06:11:51.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c00f040 con 0x7f767c004d90 2023-11-24T06:11:51.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7692ffd700 1 -- 172.21.15.97:0/1656609404 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c014480 con 0x7f767c004d90 2023-11-24T06:11:51.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f769a811700 1 -- 172.21.15.97:0/1656609404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f767c004d90 msgr2=0x7f767c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f769a811700 1 --2- 172.21.15.97:0/1656609404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f767c004d90 0x7f767c005210 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f768c009f90 tx=0x7f767c005880 comp rx=0 tx=0).stop 2023-11-24T06:11:51.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f7698dae700 1 -- 172.21.15.97:0/1656609404 reap_dead start 2023-11-24T06:11:51.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f769a811700 1 -- 172.21.15.97:0/1656609404 shutdown_connections 2023-11-24T06:11:51.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.816+0000 7f769a811700 1 -- 172.21.15.97:0/1656609404 >> 172.21.15.97:0/1656609404 conn(0x7f76940b3130 msgr2=0x7f76940d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:51.834 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.817+0000 7f769a811700 1 -- 172.21.15.97:0/1656609404 shutdown_connections 2023-11-24T06:11:51.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.817+0000 7f769a811700 1 -- 172.21.15.97:0/1656609404 wait complete. 2023-11-24T06:11:51.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.818+0000 7f769a811700 1 Processor -- start 2023-11-24T06:11:51.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.818+0000 7f769a811700 1 -- start start 2023-11-24T06:11:51.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f769a811700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769414a100 0x7f7694157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f769a811700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7694157d40 0x7f76941556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f769a811700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76941efb90 0x7f7694155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f769a811700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769415cd80 con 0x7f7694157d40 2023-11-24T06:11:51.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f769a811700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7694158510 con 0x7f769414a100 2023-11-24T06:11:51.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f769a811700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7694158230 con 0x7f76941efb90 2023-11-24T06:11:51.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7694157d40 0x7f76941556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7694157d40 0x7f76941556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50572/0 (socket says 172.21.15.97:50572) 2023-11-24T06:11:51.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 -- 172.21.15.97:0/2039102991 learned_addr learned my addr 172.21.15.97:0/2039102991 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:51.836 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f7698dae700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76941efb90 0x7f7694155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 -- 172.21.15.97:0/2039102991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76941efb90 msgr2=0x7f7694155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:51.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76941efb90 0x7f7694155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 -- 172.21.15.97:0/2039102991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769414a100 msgr2=0x7f7694157800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:11:51.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769414a100 0x7f7694157800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:51.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 -- 172.21.15.97:0/2039102991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769415c6c0 con 0x7f7694157d40 2023-11-24T06:11:51.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f76937fe700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7694157d40 0x7f76941556d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7688007290 tx=0x7f76880075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f7690ff9700 1 -- 172.21.15.97:0/2039102991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76880077e0 con 0x7f7694157d40 2023-11-24T06:11:51.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f7690ff9700 1 -- 172.21.15.97:0/2039102991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768800f850 con 0x7f7694157d40 2023-11-24T06:11:51.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769415c9a0 con 0x7f7694157d40 2023-11-24T06:11:51.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f7690ff9700 1 -- 172.21.15.97:0/2039102991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768800fbe0 con 0x7f7694157d40 2023-11-24T06:11:51.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.819+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769414a8b0 con 0x7f7694157d40 2023-11-24T06:11:51.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.820+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769415c520 con 0x7f7694157d40 2023-11-24T06:11:51.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.821+0000 7f7690ff9700 1 -- 172.21.15.97:0/2039102991 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f768800fd40 con 0x7f7694157d40 2023-11-24T06:11:51.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.821+0000 7f7690ff9700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f768006b470 0x7f768006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:51.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.821+0000 7f7690ff9700 1 -- 172.21.15.97:0/2039102991 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f768808a0c0 con 0x7f7694157d40 2023-11-24T06:11:51.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.821+0000 7f7693fff700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f768006b470 0x7f768006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:51.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.821+0000 7f7693fff700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f768006b470 0x7f768006d930 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f76840099e0 tx=0x7f7684009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:51.840 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:51.825+0000 7f7690ff9700 1 -- 172.21.15.97:0/2039102991 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76880bce90 con 0x7f7694157d40 2023-11-24T06:11:52.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.028+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f769415cb30 con 0x7f7694157d40 2023-11-24T06:11:52.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.028+0000 7f7690ff9700 1 -- 172.21.15.97:0/2039102991 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v72) v1 ==== 80+0+0 (secure 0 0 0) 0x7f769415c520 con 0x7f7694157d40 2023-11-24T06:11:52.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f768006b470 msgr2=0x7f768006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f768006b470 0x7f768006d930 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f76840099e0 tx=0x7f7684009210 comp rx=0 tx=0).stop 2023-11-24T06:11:52.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7694157d40 msgr2=0x7f76941556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7694157d40 0x7f76941556d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7688007290 tx=0x7f76880075a0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 shutdown_connections 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f768006b470 0x7f768006d930 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f76941efb90 0x7f7694155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f769414a100 0x7f7694157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 --2- 172.21.15.97:0/2039102991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7694157d40 0x7f76941556d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 >> 172.21.15.97:0/2039102991 conn(0x7f76940b3130 msgr2=0x7f76940b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.030+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 shutdown_connections 2023-11-24T06:11:52.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.031+0000 7f769a811700 1 -- 172.21.15.97:0/2039102991 wait complete. 2023-11-24T06:11:52.035 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:11:52.045 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:11:52.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:52.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T06:11:52.094 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:11:52.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:11:52.094 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:11:52.113 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:11:52.113 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:52.131 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:11:52.131 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:11:52.131 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:11:52.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:11:52.173 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.294+0000 7f1b925ca700 1 Processor -- start 2023-11-24T06:11:52.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.294+0000 7f1b925ca700 1 -- start start 2023-11-24T06:11:52.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b925ca700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c06c710 0x7f1b8c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b925ca700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1b8c0af4d0 0x7f1b8c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b925ca700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8c0ae430 con 0x7f1b8c15e080 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b925ca700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8c0af0b0 con 0x7f1b8c0af4d0 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b925ca700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0b05a0 con 0x7f1b8c06c710 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b8b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c06c710 0x7f1b8c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b90b67700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1b8c0af4d0 0x7f1b8c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b8bfff700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f1b8c15e080 0x7f1b8c06c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:52352/0 (socket says 172.21.15.97:52352) 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.295+0000 7f1b8bfff700 1 -- 172.21.15.97:0/703444847 learned_addr learned my addr 172.21.15.97:0/703444847 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b8b7fe700 1 -- 172.21.15.97:0/703444847 >> v2:172.21.15.183:3300/0 conn(0x7f1b8c0af4d0 msgr2=0x7f1b8c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b8b7fe700 1 --2- 172.21.15.97:0/703444847 >> v2:172.21.15.183:3300/0 conn(0x7f1b8c0af4d0 0x7f1b8c0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b8b7fe700 1 -- 172.21.15.97:0/703444847 >> v1:172.21.15.183:6789/0 conn(0x7f1b8c15e080 legacy=0x7f1b8c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b8b7fe700 1 -- 172.21.15.97:0/703444847 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c0ae710 con 0x7f1b8c06c710 2023-11-24T06:11:52.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b90b67700 1 --2- 172.21.15.97:0/703444847 >> v2:172.21.15.183:3300/0 conn(0x7f1b8c0af4d0 0x7f1b8c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b8b7fe700 1 --2- 172.21.15.97:0/703444847 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c06c710 0x7f1b8c0b5050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009840 tx=0x7f1b7c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801c61db8e49eae7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b8affd700 1 -- 172.21.15.97:0/703444847 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c011830 con 0x7f1b8c06c710 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b8affd700 1 -- 172.21.15.97:0/703444847 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c011990 con 0x7f1b8c06c710 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.296+0000 7f1b8affd700 1 -- 172.21.15.97:0/703444847 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c011d20 con 0x7f1b8c06c710 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.297+0000 7f1b925ca700 1 -- 172.21.15.97:0/703444847 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c06c710 msgr2=0x7f1b8c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.297+0000 7f1b925ca700 1 --2- 172.21.15.97:0/703444847 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c06c710 0x7f1b8c0b5050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009840 tx=0x7f1b7c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.297+0000 7f1b925ca700 1 -- 172.21.15.97:0/703444847 shutdown_connections 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.297+0000 7f1b925ca700 1 --2- 172.21.15.97:0/703444847 >> v2:172.21.15.183:3300/0 conn(0x7f1b8c0af4d0 0x7f1b8c0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.297+0000 7f1b925ca700 1 --2- 172.21.15.97:0/703444847 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c06c710 0x7f1b8c0b5050 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.297+0000 7f1b925ca700 1 -- 172.21.15.97:0/703444847 >> 172.21.15.97:0/703444847 conn(0x7f1b8c071b90 msgr2=0x7f1b8c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.297+0000 7f1b925ca700 1 -- 172.21.15.97:0/703444847 shutdown_connections 2023-11-24T06:11:52.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.297+0000 7f1b925ca700 1 -- 172.21.15.97:0/703444847 wait complete. 2023-11-24T06:11:52.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.298+0000 7f1b925ca700 1 Processor -- start 2023-11-24T06:11:52.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.298+0000 7f1b925ca700 1 -- start start 2023-11-24T06:11:52.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b925ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c06c710 0x7f1b8c1603a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b925ca700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c0af4d0 0x7f1b8c146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b925ca700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b8c15e080 0x7f1b8c1465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b925ca700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c154180 con 0x7f1b8c06c710 2023-11-24T06:11:52.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b925ca700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0d0480 con 0x7f1b8c15e080 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b925ca700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c146b10 con 0x7f1b8c0af4d0 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c06c710 0x7f1b8c1603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c06c710 0x7f1b8c1603a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50584/0 (socket says 172.21.15.97:50584) 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 -- 172.21.15.97:0/4144194673 learned_addr learned my addr 172.21.15.97:0/4144194673 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 -- 172.21.15.97:0/4144194673 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c0af4d0 msgr2=0x7f1b8c146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c0af4d0 0x7f1b8c146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 -- 172.21.15.97:0/4144194673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b8c15e080 msgr2=0x7f1b8c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b8c15e080 0x7f1b8c1465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 -- 172.21.15.97:0/4144194673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b84009a70 con 0x7f1b8c06c710 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b8bfff700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c06c710 0x7f1b8c1603a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c4e0 tx=0x7f1b8000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b88ff9700 1 -- 172.21.15.97:0/4144194673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80005d70 con 0x7f1b8c06c710 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b88ff9700 1 -- 172.21.15.97:0/4144194673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b800070d0 con 0x7f1b8c06c710 2023-11-24T06:11:52.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7c0094a0 con 0x7f1b8c06c710 2023-11-24T06:11:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.299+0000 7f1b88ff9700 1 -- 172.21.15.97:0/4144194673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80007440 con 0x7f1b8c06c710 2023-11-24T06:11:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.300+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c0cfec0 con 0x7f1b8c06c710 2023-11-24T06:11:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.300+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8c0753f0 con 0x7f1b8c06c710 2023-11-24T06:11:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.301+0000 7f1b88ff9700 1 -- 172.21.15.97:0/4144194673 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1b800075a0 con 0x7f1b8c06c710 2023-11-24T06:11:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.301+0000 7f1b88ff9700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b7806b470 0x7f1b7806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.301+0000 7f1b8b7fe700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b7806b470 0x7f1b7806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.301+0000 7f1b88ff9700 1 -- 172.21.15.97:0/4144194673 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1b800892f0 con 0x7f1b8c06c710 2023-11-24T06:11:52.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.302+0000 7f1b8b7fe700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b7806b470 0x7f1b7806d930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009b90 tx=0x7f1b7c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.306+0000 7f1b88ff9700 1 -- 172.21.15.97:0/4144194673 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b80053c10 con 0x7f1b8c06c710 2023-11-24T06:11:52.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.541+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b8c153c40 con 0x7f1b8c06c710 2023-11-24T06:11:52.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.542+0000 7f1b88ff9700 1 -- 172.21.15.97:0/4144194673 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f1b8c153c40 con 0x7f1b8c06c710 2023-11-24T06:11:52.545 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:52.545 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5863,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2765643379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2765643379},{"type":"v1","addr":"172.21.15.183:6834","nonce":2765643379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1367625334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367625334},{"type":"v1","addr":"172.21.15.97:6836","nonce":1367625334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/807372372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":807372372},{"type":"v1","addr":"172.21.15.97:6837","nonce":807372372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:35.581019+0000","modified":"2023-11-24T06:11:38.607438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3452869588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3452869588},{"type":"v1","addr":"172.21.15.183:6835","nonce":3452869588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:11:52.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b7806b470 msgr2=0x7f1b7806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.547 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b7806b470 0x7f1b7806d930 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009b90 tx=0x7f1b7c010000 comp rx=0 tx=0).stop 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c06c710 msgr2=0x7f1b8c1603a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c06c710 0x7f1b8c1603a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c4e0 tx=0x7f1b8000c8a0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 shutdown_connections 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b8c15e080 0x7f1b8c1465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8c06c710 0x7f1b8c1603a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1b7806b470 0x7f1b7806d930 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 --2- 172.21.15.97:0/4144194673 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1b8c0af4d0 0x7f1b8c146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.544+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 >> 172.21.15.97:0/4144194673 conn(0x7f1b8c071b90 msgr2=0x7f1b8c0af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.545+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 shutdown_connections 2023-11-24T06:11:52.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.545+0000 7f1b925ca700 1 -- 172.21.15.97:0/4144194673 wait complete. 2023-11-24T06:11:52.549 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 47 2023-11-24T06:11:52.560 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.679+0000 7fa4537f0700 1 Processor -- start 2023-11-24T06:11:52.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.679+0000 7fa4537f0700 1 -- start start 2023-11-24T06:11:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa4537f0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c15bdd0 0x7fa44c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa4537f0700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa44c0d1c70 0x7fa44c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa4537f0700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c0b22d0 con 0x7fa44c0b3370 2023-11-24T06:11:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa4537f0700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c0b2f50 con 0x7fa44c0d1c70 2023-11-24T06:11:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa4537f0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0d2640 con 0x7fa44c15bdd0 2023-11-24T06:11:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa450d8b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa44c0b3370 0x7fa44c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43048/0 (socket says 172.21.15.97:43048) 2023-11-24T06:11:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa450d8b700 1 -- 172.21.15.97:0/195203986 learned_addr learned my addr 172.21.15.97:0/195203986 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:52.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa45158c700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c15bdd0 0x7fa44c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa451d8d700 1 --2- 172.21.15.97:0/195203986 >> v2:172.21.15.183:3300/0 conn(0x7fa44c0d1c70 0x7fa44c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534665339 0 0) 0x7fa44c0b22d0 con 0x7fa44c0b3370 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.680+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4380035b0 con 0x7fa44c0b3370 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 825414108 0 0) 0x7fa4380035b0 con 0x7fa44c0b3370 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c15bdd0 msgr2=0x7fa44c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c15bdd0 0x7fa44c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 >> v2:172.21.15.183:3300/0 conn(0x7fa44c0d1c70 msgr2=0x7fa44c0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 --2- 172.21.15.97:0/195203986 >> v2:172.21.15.183:3300/0 conn(0x7fa44c0d1c70 0x7fa44c0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0b25b0 con 0x7fa44c0b3370 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fa448001670 con 0x7fa44c0b3370 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 >> v1:172.21.15.97:6789/0 conn(0x7fa44c0b3370 legacy=0x7fa44c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c15bdd0 0x7fa438003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438003f80 0x7fa438004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa438004d90 0x7fa438005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0b2f50 con 0x7fa438004d90 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0d2640 con 0x7fa438003f80 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43800ba60 con 0x7fa44c15bdd0 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa451d8d700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa438004d90 0x7fa438005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa450d8b700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438003f80 0x7fa438004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa45158c700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c15bdd0 0x7fa438003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa451d8d700 1 -- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c15bdd0 msgr2=0x7fa438003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa451d8d700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c15bdd0 0x7fa438003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa451d8d700 1 -- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438003f80 msgr2=0x7fa438004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa451d8d700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438003f80 0x7fa438004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa451d8d700 1 -- 172.21.15.97:0/195203986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380035b0 con 0x7fa438004d90 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa451d8d700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa438004d90 0x7fa438005210 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa444007360 tx=0x7fa438005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3a6f4c25db70eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444007e80 con 0x7fa438004d90 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444014400 con 0x7fa438004d90 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.681+0000 7fa443fff700 1 -- 172.21.15.97:0/195203986 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444014790 con 0x7fa438004d90 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.682+0000 7fa4537f0700 1 -- 172.21.15.97:0/195203986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa438004d90 msgr2=0x7fa438005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.682+0000 7fa4537f0700 1 --2- 172.21.15.97:0/195203986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa438004d90 0x7fa438005210 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fa444007360 tx=0x7fa438005880 comp rx=0 tx=0).stop 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.682+0000 7fa451d8d700 1 -- 172.21.15.97:0/195203986 reap_dead start 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.682+0000 7fa4537f0700 1 -- 172.21.15.97:0/195203986 shutdown_connections 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.682+0000 7fa4537f0700 1 -- 172.21.15.97:0/195203986 >> 172.21.15.97:0/195203986 conn(0x7fa44c0731a0 msgr2=0x7fa44c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.682+0000 7fa4537f0700 1 -- 172.21.15.97:0/195203986 shutdown_connections 2023-11-24T06:11:52.688 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.682+0000 7fa4537f0700 1 -- 172.21.15.97:0/195203986 wait complete. 2023-11-24T06:11:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.683+0000 7fa4537f0700 1 Processor -- start 2023-11-24T06:11:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.683+0000 7fa4537f0700 1 -- start start 2023-11-24T06:11:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa4537f0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438004d90 0x7fa44c0b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa4537f0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa44c0ca5e0 0x7fa44c0c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa4537f0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c152ad0 0x7fa44c0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa4537f0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0c9dd0 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa4537f0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0b5ff0 con 0x7fa438004d90 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa4537f0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c0b62d0 con 0x7fa44c152ad0 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa44c0ca5e0 0x7fa44c0c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa44c0ca5e0 0x7fa44c0c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50606/0 (socket says 172.21.15.97:50606) 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 -- 172.21.15.97:0/2484231097 learned_addr learned my addr 172.21.15.97:0/2484231097 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa451d8d700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c152ad0 0x7fa44c0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa45158c700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438004d90 0x7fa44c0b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 -- 172.21.15.97:0/2484231097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c152ad0 msgr2=0x7fa44c0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c152ad0 0x7fa44c0cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 -- 172.21.15.97:0/2484231097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438004d90 msgr2=0x7fa44c0b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438004d90 0x7fa44c0b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 -- 172.21.15.97:0/2484231097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c0c8db0 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.684+0000 7fa450d8b700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa44c0ca5e0 0x7fa44c0c9890 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa448007310 tx=0x7fa448007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.685+0000 7fa441ffb700 1 -- 172.21.15.97:0/2484231097 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44800f6c0 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.685+0000 7fa441ffb700 1 -- 172.21.15.97:0/2484231097 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44800f840 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.685+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c0c9090 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.685+0000 7fa441ffb700 1 -- 172.21.15.97:0/2484231097 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44800fbf0 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.685+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c0caa60 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.686+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c0cac20 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.686+0000 7fa441ffb700 1 -- 172.21.15.97:0/2484231097 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fa44800fd50 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.687+0000 7fa441ffb700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa43006b470 0x7fa43006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.687+0000 7fa45158c700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa43006b470 0x7fa43006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.687+0000 7fa441ffb700 1 -- 172.21.15.97:0/2484231097 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fa44808a040 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.687+0000 7fa45158c700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa43006b470 0x7fa43006d930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009c60 tx=0x7fa43c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:52.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.691+0000 7fa441ffb700 1 -- 172.21.15.97:0/2484231097 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa448058300 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.885+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa44c0c95b0 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.887+0000 7fa441ffb700 1 -- 172.21.15.97:0/2484231097 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16734 (secure 0 0 0) 0x7fa44c0c95b0 con 0x7fa44c0ca5e0 2023-11-24T06:11:52.890 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:52.890 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":73,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:52.252147+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:33.973262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:34.541598+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":73,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa43006b470 msgr2=0x7fa43006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa43006b470 0x7fa43006d930 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009c60 tx=0x7fa43c009210 comp rx=0 tx=0).stop 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa44c0ca5e0 msgr2=0x7fa44c0c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa44c0ca5e0 0x7fa44c0c9890 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa448007310 tx=0x7fa448007620 comp rx=0 tx=0).stop 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 shutdown_connections 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa43006b470 0x7fa43006d930 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa44c152ad0 0x7fa44c0cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa438004d90 0x7fa44c0b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 --2- 172.21.15.97:0/2484231097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa44c0ca5e0 0x7fa44c0c9890 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:52.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 >> 172.21.15.97:0/2484231097 conn(0x7fa44c0731a0 msgr2=0x7fa44c0cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:52.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 shutdown_connections 2023-11-24T06:11:52.893 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:52.889+0000 7fa4537f0700 1 -- 172.21.15.97:0/2484231097 wait complete. 2023-11-24T06:11:52.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:11:33.973262+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:11:34.541598+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 73, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:52.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:11:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.024+0000 7fc86250e700 1 Processor -- start 2023-11-24T06:11:53.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.024+0000 7fc86250e700 1 -- start start 2023-11-24T06:11:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc86250e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc85c15e160 0x7fc85c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc86250e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc85c0afc90 0x7fc85c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc86250e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85c0b3c60 con 0x7fc85c0744d0 2023-11-24T06:11:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc86250e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c0b4500 con 0x7fc85c0afc90 2023-11-24T06:11:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc86250e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85c074940 con 0x7fc85c15e160 2023-11-24T06:11:53.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc85c0afc90 0x7fc85c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc85c0afc90 0x7fc85c073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50612/0 (socket says 172.21.15.97:50612) 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85b7fe700 1 -- 172.21.15.97:0/1693596167 learned_addr learned my addr 172.21.15.97:0/1693596167 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85bfff700 1 --2- 172.21.15.97:0/1693596167 >> v2:172.21.15.183:3300/0 conn(0x7fc85c15e160 0x7fc85c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85b7fe700 1 -- 172.21.15.97:0/1693596167 >> v2:172.21.15.183:3300/0 conn(0x7fc85c15e160 msgr2=0x7fc85c0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85b7fe700 1 --2- 172.21.15.97:0/1693596167 >> v2:172.21.15.183:3300/0 conn(0x7fc85c15e160 0x7fc85c0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85b7fe700 1 -- 172.21.15.97:0/1693596167 >> v1:172.21.15.183:6789/0 conn(0x7fc85c0744d0 legacy=0x7fc85c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85b7fe700 1 -- 172.21.15.97:0/1693596167 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0b58c0 con 0x7fc85c0afc90 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.025+0000 7fc85b7fe700 1 --2- 172.21.15.97:0/1693596167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0afc90 0x7fc85c073f90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc85c09efa0 tx=0x7fc84c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b71982c4fb582a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc85affd700 1 -- 172.21.15.97:0/1693596167 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c011830 con 0x7fc85c0afc90 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc85affd700 1 -- 172.21.15.97:0/1693596167 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c011990 con 0x7fc85c0afc90 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc85affd700 1 -- 172.21.15.97:0/1693596167 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c011d20 con 0x7fc85c0afc90 2023-11-24T06:11:53.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc86250e700 1 -- 172.21.15.97:0/1693596167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0afc90 msgr2=0x7fc85c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc86250e700 1 --2- 172.21.15.97:0/1693596167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0afc90 0x7fc85c073f90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc85c09efa0 tx=0x7fc84c009c00 comp rx=0 tx=0).stop 2023-11-24T06:11:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc86250e700 1 -- 172.21.15.97:0/1693596167 shutdown_connections 2023-11-24T06:11:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc86250e700 1 --2- 172.21.15.97:0/1693596167 >> v2:172.21.15.183:3300/0 conn(0x7fc85c15e160 0x7fc85c0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc86250e700 1 --2- 172.21.15.97:0/1693596167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0afc90 0x7fc85c073f90 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc86250e700 1 -- 172.21.15.97:0/1693596167 >> 172.21.15.97:0/1693596167 conn(0x7fc85c0d0bf0 msgr2=0x7fc85c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc86250e700 1 -- 172.21.15.97:0/1693596167 shutdown_connections 2023-11-24T06:11:53.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.026+0000 7fc86250e700 1 -- 172.21.15.97:0/1693596167 wait complete. 2023-11-24T06:11:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.027+0000 7fc86250e700 1 Processor -- start 2023-11-24T06:11:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.027+0000 7fc86250e700 1 -- start start 2023-11-24T06:11:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc86250e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0744d0 0x7fc85c149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc86250e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc85c0afc90 0x7fc85c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc86250e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c15e160 0x7fc85c0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc86250e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0ca3e0 con 0x7fc85c0744d0 2023-11-24T06:11:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc86250e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c14a650 con 0x7fc85c15e160 2023-11-24T06:11:53.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc86250e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0c6690 con 0x7fc85c0afc90 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0744d0 0x7fc85c149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc85bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0744d0 0x7fc85c149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50616/0 (socket says 172.21.15.97:50616) 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.028+0000 7fc85bfff700 1 -- 172.21.15.97:0/1837889081 learned_addr learned my addr 172.21.15.97:0/1837889081 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc85b7fe700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc85c0afc90 0x7fc85c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc85bfff700 1 -- 172.21.15.97:0/1837889081 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc85c0afc90 msgr2=0x7fc85c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc860aab700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c15e160 0x7fc85c0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc85bfff700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc85c0afc90 0x7fc85c0c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc85bfff700 1 -- 172.21.15.97:0/1837889081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c15e160 msgr2=0x7fc85c0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc85bfff700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c15e160 0x7fc85c0c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc85bfff700 1 -- 172.21.15.97:0/1837889081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c009550 con 0x7fc85c0744d0 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc85bfff700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0744d0 0x7fc85c149db0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e9b0 tx=0x7fc85000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc858ff9700 1 -- 172.21.15.97:0/1837889081 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850009710 con 0x7fc85c0744d0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc858ff9700 1 -- 172.21.15.97:0/1837889081 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85000c850 con 0x7fc85c0744d0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c0c6970 con 0x7fc85c0744d0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc858ff9700 1 -- 172.21.15.97:0/1837889081 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000cbc0 con 0x7fc85c0744d0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.029+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c14ab80 con 0x7fc85c0744d0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.030+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c0c6b60 con 0x7fc85c0744d0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.030+0000 7fc858ff9700 1 -- 172.21.15.97:0/1837889081 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fc85000cd20 con 0x7fc85c0744d0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.031+0000 7fc858ff9700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc84806b470 0x7fc84806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.031+0000 7fc858ff9700 1 -- 172.21.15.97:0/1837889081 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fc85008a1a0 con 0x7fc85c0744d0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.031+0000 7fc85b7fe700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc84806b470 0x7fc84806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.031+0000 7fc85b7fe700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc84806b470 0x7fc84806d930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0d1520 tx=0x7fc84c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.035+0000 7fc858ff9700 1 -- 172.21.15.97:0/1837889081 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc850054980 con 0x7fc85c0744d0 2023-11-24T06:11:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.224+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc85c068fb0 con 0x7fc85c0744d0 2023-11-24T06:11:53.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.225+0000 7fc858ff9700 1 -- 172.21.15.97:0/1837889081 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc850057fa0 con 0x7fc85c0744d0 2023-11-24T06:11:53.228 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc84806b470 msgr2=0x7fc84806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc84806b470 0x7fc84806d930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0d1520 tx=0x7fc84c008030 comp rx=0 tx=0).stop 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0744d0 msgr2=0x7fc85c149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0744d0 0x7fc85c149db0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e9b0 tx=0x7fc85000ed70 comp rx=0 tx=0).stop 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 shutdown_connections 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc85c15e160 0x7fc85c0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc85c0744d0 0x7fc85c149db0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc84806b470 0x7fc84806d930 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.230 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 --2- 172.21.15.97:0/1837889081 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc85c0afc90 0x7fc85c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 >> 172.21.15.97:0/1837889081 conn(0x7fc85c0d0bf0 msgr2=0x7fc85c06eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 shutdown_connections 2023-11-24T06:11:53.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.227+0000 7fc86250e700 1 -- 172.21.15.97:0/1837889081 wait complete. 2023-11-24T06:11:53.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:11:53.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.362+0000 7fa06dcd6700 1 Processor -- start 2023-11-24T06:11:53.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.362+0000 7fa06dcd6700 1 -- start start 2023-11-24T06:11:53.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa06dcd6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa0680746a0 0x7fa0680b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa06dcd6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680afc90 0x7fa06806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.368 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa06dcd6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0680b3c60 con 0x7fa06815e160 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa06dcd6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0680b4500 con 0x7fa0680746a0 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa06dcd6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0680b58c0 con 0x7fa0680afc90 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0677fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa06815e160 0x7fa068073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43076/0 (socket says 172.21.15.97:43076) 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0677fe700 1 -- 172.21.15.97:0/2289202324 learned_addr learned my addr 172.21.15.97:0/2289202324 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601766236 0 0) 0x7fa0680b3c60 con 0x7fa06815e160 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0500035b0 con 0x7fa06815e160 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa066ffd700 1 --2- 172.21.15.97:0/2289202324 >> v2:172.21.15.183:3300/0 conn(0x7fa0680746a0 0x7fa0680b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa067fff700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680afc90 0x7fa06806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 764750426 0 0) 0x7fa0500035b0 con 0x7fa06815e160 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680afc90 msgr2=0x7fa06806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:53.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0667fc700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680afc90 0x7fa06806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 >> v2:172.21.15.183:3300/0 conn(0x7fa0680746a0 msgr2=0x7fa0680b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0667fc700 1 --2- 172.21.15.97:0/2289202324 >> v2:172.21.15.183:3300/0 conn(0x7fa0680746a0 0x7fa0680b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.363+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0680b0120 con 0x7fa06815e160 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fa05c001670 con 0x7fa06815e160 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 >> v1:172.21.15.97:6789/0 conn(0x7fa06815e160 legacy=0x7fa068073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680afc90 0x7fa050003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa050003f80 0x7fa050004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa050004d90 0x7fa050005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0680b4500 con 0x7fa050004d90 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0680b58c0 con 0x7fa050003f80 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05000ba10 con 0x7fa0680afc90 2023-11-24T06:11:53.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa067fff700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa050004d90 0x7fa050005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0677fe700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680afc90 0x7fa050003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa066ffd700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa050003f80 0x7fa050004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa067fff700 1 -- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680afc90 msgr2=0x7fa050003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa067fff700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0680afc90 0x7fa050003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa067fff700 1 -- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa050003f80 msgr2=0x7fa050004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa067fff700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa050003f80 0x7fa050004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa067fff700 1 -- 172.21.15.97:0/2289202324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500035b0 con 0x7fa050004d90 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa067fff700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa050004d90 0x7fa050005210 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa060009fc0 tx=0x7fa050005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0182647e2ce2c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000ebd0 con 0x7fa050004d90 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06000f040 con 0x7fa050004d90 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.364+0000 7fa0667fc700 1 -- 172.21.15.97:0/2289202324 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa060014480 con 0x7fa050004d90 2023-11-24T06:11:53.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.365+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2289202324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa050004d90 msgr2=0x7fa050005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.365+0000 7fa06dcd6700 1 --2- 172.21.15.97:0/2289202324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa050004d90 0x7fa050005210 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa060009fc0 tx=0x7fa050005880 comp rx=0 tx=0).stop 2023-11-24T06:11:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.365+0000 7fa067fff700 1 -- 172.21.15.97:0/2289202324 reap_dead start 2023-11-24T06:11:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.365+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2289202324 shutdown_connections 2023-11-24T06:11:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.365+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2289202324 >> 172.21.15.97:0/2289202324 conn(0x7fa0680d0bf0 msgr2=0x7fa0680d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.365+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2289202324 shutdown_connections 2023-11-24T06:11:53.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.365+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2289202324 wait complete. 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.366+0000 7fa06dcd6700 1 Processor -- start 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.367+0000 7fa06dcd6700 1 -- start start 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.367+0000 7fa06dcd6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa06814b580 0x7fa0680b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.367+0000 7fa06dcd6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0681570d0 0x7fa0680b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.367+0000 7fa06dcd6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa068157d70 0x7fa0681581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa06dcd6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068156030 con 0x7fa06814b580 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa06dcd6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa068156310 con 0x7fa068157d70 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa06dcd6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0681565f0 con 0x7fa0681570d0 2023-11-24T06:11:53.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa06814b580 0x7fa0680b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa06814b580 0x7fa0680b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50636/0 (socket says 172.21.15.97:50636) 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 -- 172.21.15.97:0/2805476061 learned_addr learned my addr 172.21.15.97:0/2805476061 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa067fff700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa068157d70 0x7fa0681581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa066ffd700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0681570d0 0x7fa0680b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 -- 172.21.15.97:0/2805476061 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0681570d0 msgr2=0x7fa0680b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0681570d0 0x7fa0680b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 -- 172.21.15.97:0/2805476061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa068157d70 msgr2=0x7fa0681581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa068157d70 0x7fa0681581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 -- 172.21.15.97:0/2805476061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0681568d0 con 0x7fa06814b580 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa0677fe700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa06814b580 0x7fa0680b0e00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa05c001aa0 tx=0x7fa05c00f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa04ffff700 1 -- 172.21.15.97:0/2805476061 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c004e00 con 0x7fa06814b580 2023-11-24T06:11:53.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa04ffff700 1 -- 172.21.15.97:0/2805476061 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c00fa90 con 0x7fa06814b580 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa068156bb0 con 0x7fa06814b580 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa04ffff700 1 -- 172.21.15.97:0/2805476061 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00fe40 con 0x7fa06814b580 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.368+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06814b960 con 0x7fa06814b580 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.369+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa068074a80 con 0x7fa06814b580 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.370+0000 7fa04ffff700 1 -- 172.21.15.97:0/2805476061 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fa05c01d460 con 0x7fa06814b580 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.370+0000 7fa04ffff700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa05406b470 0x7fa05406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.370+0000 7fa066ffd700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa05406b470 0x7fa05406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.370+0000 7fa04ffff700 1 -- 172.21.15.97:0/2805476061 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fa05c08a2a0 con 0x7fa06814b580 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.371+0000 7fa066ffd700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa05406b470 0x7fa05406d930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0680b1c30 tx=0x7fa058008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.375+0000 7fa04ffff700 1 -- 172.21.15.97:0/2805476061 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa05c054b10 con 0x7fa06814b580 2023-11-24T06:11:53.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.564+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa068068fb0 con 0x7fa06814b580 2023-11-24T06:11:53.567 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.565+0000 7fa04ffff700 1 -- 172.21.15.97:0/2805476061 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa05c058130 con 0x7fa06814b580 2023-11-24T06:11:53.567 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:11:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa05406b470 msgr2=0x7fa05406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa05406b470 0x7fa05406d930 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0680b1c30 tx=0x7fa058008040 comp rx=0 tx=0).stop 2023-11-24T06:11:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa06814b580 msgr2=0x7fa0680b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa06814b580 0x7fa0680b0e00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa05c001aa0 tx=0x7fa05c00f4b0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 shutdown_connections 2023-11-24T06:11:53.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa068157d70 0x7fa0681581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa06814b580 0x7fa0680b0e00 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa05406b470 0x7fa05406d930 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 --2- 172.21.15.97:0/2805476061 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa0681570d0 0x7fa0680b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 >> 172.21.15.97:0/2805476061 conn(0x7fa0680d0bf0 msgr2=0x7fa06815d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:53.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 shutdown_connections 2023-11-24T06:11:53.571 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.567+0000 7fa06dcd6700 1 -- 172.21.15.97:0/2805476061 wait complete. 2023-11-24T06:11:53.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:11:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.703+0000 7f4dba941700 1 Processor -- start 2023-11-24T06:11:53.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.703+0000 7f4dba941700 1 -- start start 2023-11-24T06:11:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4dba941700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4db415e160 0x7f4db40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4dba941700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40afc90 0x7f4db4073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4dba941700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4db40744d0 0x7f4db406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4dba941700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db40b4500 con 0x7f4db415e160 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4dba941700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db4074960 con 0x7f4db40744d0 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4dba941700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40b3c60 con 0x7f4db40afc90 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4db415e160 0x7f4db40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4db415e160 0x7f4db40b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50644/0 (socket says 172.21.15.97:50644) 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 -- 172.21.15.97:0/3130296973 learned_addr learned my addr 172.21.15.97:0/3130296973 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db8ede700 1 --2- 172.21.15.97:0/3130296973 >> v2:172.21.15.183:3300/0 conn(0x7f4db40744d0 0x7f4db406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db37fe700 1 --2- 172.21.15.97:0/3130296973 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40afc90 0x7f4db4073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 -- 172.21.15.97:0/3130296973 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40afc90 msgr2=0x7f4db4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 --2- 172.21.15.97:0/3130296973 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40afc90 0x7f4db4073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 -- 172.21.15.97:0/3130296973 >> v2:172.21.15.183:3300/0 conn(0x7f4db40744d0 msgr2=0x7f4db406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 --2- 172.21.15.97:0/3130296973 >> v2:172.21.15.183:3300/0 conn(0x7f4db40744d0 0x7f4db406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 -- 172.21.15.97:0/3130296973 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db40b58c0 con 0x7f4db415e160 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db3fff700 1 --2- 172.21.15.97:0/3130296973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db40b5da0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4db40d1cd0 tx=0x7f4da4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7d5c0278543ad9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.704+0000 7f4db2ffd700 1 -- 172.21.15.97:0/3130296973 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da400b870 con 0x7f4db415e160 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4db2ffd700 1 -- 172.21.15.97:0/3130296973 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da400b9d0 con 0x7f4db415e160 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4db2ffd700 1 -- 172.21.15.97:0/3130296973 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da400bd60 con 0x7f4db415e160 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 -- 172.21.15.97:0/3130296973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 msgr2=0x7f4db40b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 --2- 172.21.15.97:0/3130296973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db40b5da0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4db40d1cd0 tx=0x7f4da4009bd0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 -- 172.21.15.97:0/3130296973 shutdown_connections 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 --2- 172.21.15.97:0/3130296973 >> v2:172.21.15.183:3300/0 conn(0x7f4db40744d0 0x7f4db406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 --2- 172.21.15.97:0/3130296973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db40b5da0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 --2- 172.21.15.97:0/3130296973 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40afc90 0x7f4db4073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 -- 172.21.15.97:0/3130296973 >> 172.21.15.97:0/3130296973 conn(0x7f4db40d0bf0 msgr2=0x7f4db40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 -- 172.21.15.97:0/3130296973 shutdown_connections 2023-11-24T06:11:53.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.705+0000 7f4dba941700 1 -- 172.21.15.97:0/3130296973 wait complete. 2023-11-24T06:11:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.706+0000 7f4dba941700 1 Processor -- start 2023-11-24T06:11:53.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.706+0000 7f4dba941700 1 -- start start 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4dba941700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40744d0 0x7f4db4154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4dba941700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4db40afc90 0x7f4db415e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4dba941700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db41f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4dba941700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41f4f30 con 0x7f4db415e160 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4dba941700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41f4970 con 0x7f4db40afc90 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4dba941700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41f4c50 con 0x7f4db40744d0 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4db8ede700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db41f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4db8ede700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db41f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50654/0 (socket says 172.21.15.97:50654) 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4db8ede700 1 -- 172.21.15.97:0/3876121341 learned_addr learned my addr 172.21.15.97:0/3876121341 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4db37fe700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4db40afc90 0x7f4db415e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.707+0000 7f4db3fff700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40744d0 0x7f4db4154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db8ede700 1 -- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40744d0 msgr2=0x7f4db4154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db8ede700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40744d0 0x7f4db4154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db8ede700 1 -- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4db40afc90 msgr2=0x7f4db415e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db8ede700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4db40afc90 0x7f4db415e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db8ede700 1 -- 172.21.15.97:0/3876121341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da4009550 con 0x7f4db415e160 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db3fff700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40744d0 0x7f4db4154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db8ede700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db41f3f50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4dac007e40 tx=0x7f4dac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db0ff9700 1 -- 172.21.15.97:0/3876121341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00ede0 con 0x7f4db415e160 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db0ff9700 1 -- 172.21.15.97:0/3876121341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac014420 con 0x7f4db415e160 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db4156030 con 0x7f4db415e160 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4db0ff9700 1 -- 172.21.15.97:0/3876121341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0147d0 con 0x7f4db415e160 2023-11-24T06:11:53.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.708+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db415ead0 con 0x7f4db415e160 2023-11-24T06:11:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.709+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db4068fb0 con 0x7f4db415e160 2023-11-24T06:11:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.709+0000 7f4db0ff9700 1 -- 172.21.15.97:0/3876121341 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4dac014930 con 0x7f4db415e160 2023-11-24T06:11:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.710+0000 7f4db0ff9700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4da006b470 0x7f4da006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.710+0000 7f4db0ff9700 1 -- 172.21.15.97:0/3876121341 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4dac08a730 con 0x7f4db415e160 2023-11-24T06:11:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.710+0000 7f4db3fff700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4da006b470 0x7f4da006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.710+0000 7f4db3fff700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4da006b470 0x7f4da006d930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4da4014040 tx=0x7f4da4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:53.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.714+0000 7f4db0ff9700 1 -- 172.21.15.97:0/3876121341 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dac054fb0 con 0x7f4db415e160 2023-11-24T06:11:53.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.905+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4db40742b0 con 0x7f4db415e160 2023-11-24T06:11:53.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.905+0000 7f4db0ff9700 1 -- 172.21.15.97:0/3876121341 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4dac0585d0 con 0x7f4db415e160 2023-11-24T06:11:53.910 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4da006b470 msgr2=0x7f4da006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4da006b470 0x7f4da006d930 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4da4014040 tx=0x7f4da4008030 comp rx=0 tx=0).stop 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 msgr2=0x7f4db41f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db41f3f50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4dac007e40 tx=0x7f4dac00e3c0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 shutdown_connections 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4db415e160 0x7f4db41f3f50 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4da006b470 0x7f4da006d930 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4db40744d0 0x7f4db4154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 --2- 172.21.15.97:0/3876121341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4db40afc90 0x7f4db415e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 >> 172.21.15.97:0/3876121341 conn(0x7f4db40d0bf0 msgr2=0x7f4db406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:53.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 shutdown_connections 2023-11-24T06:11:53.912 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:53.907+0000 7f4dba941700 1 -- 172.21.15.97:0/3876121341 wait complete. 2023-11-24T06:11:53.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:54.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.039+0000 7f10aa1cd700 1 Processor -- start 2023-11-24T06:11:54.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.039+0000 7f10aa1cd700 1 -- start start 2023-11-24T06:11:54.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10aa1cd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10aa1cd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f10a40af4d0 0x7f10a40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10aa1cd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10a406c000 0x7f10a406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10aa1cd700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40af0b0 con 0x7f10a406c000 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10aa1cd700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a406c9f0 con 0x7f10a40af4d0 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10aa1cd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b05a0 con 0x7f10a415e080 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10a406c000 0x7f10a406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10a406c000 0x7f10a406c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50662/0 (socket says 172.21.15.97:50662) 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 -- 172.21.15.97:0/1721720763 learned_addr learned my addr 172.21.15.97:0/1721720763 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a37fe700 1 --2- 172.21.15.97:0/1721720763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 -- 172.21.15.97:0/1721720763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 msgr2=0x7f10a40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a2ffd700 1 --2- 172.21.15.97:0/1721720763 >> v2:172.21.15.183:3300/0 conn(0x7f10a40af4d0 0x7f10a40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 --2- 172.21.15.97:0/1721720763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 -- 172.21.15.97:0/1721720763 >> v2:172.21.15.183:3300/0 conn(0x7f10a40af4d0 msgr2=0x7f10a40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:54.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 --2- 172.21.15.97:0/1721720763 >> v2:172.21.15.183:3300/0 conn(0x7f10a40af4d0 0x7f10a40afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 -- 172.21.15.97:0/1721720763 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40ae430 con 0x7f10a406c000 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.040+0000 7f10a3fff700 1 --2- 172.21.15.97:0/1721720763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a406c4b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f10a406d380 tx=0x7f109c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e2506ba4938cc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10a27fc700 1 -- 172.21.15.97:0/1721720763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00e070 con 0x7f10a406c000 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10a27fc700 1 -- 172.21.15.97:0/1721720763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c00b980 con 0x7f10a406c000 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10a27fc700 1 -- 172.21.15.97:0/1721720763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00bd30 con 0x7f10a406c000 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1721720763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 msgr2=0x7f10a406c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1721720763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a406c4b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f10a406d380 tx=0x7f109c00b330 comp rx=0 tx=0).stop 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1721720763 shutdown_connections 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1721720763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a406c4b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1721720763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1721720763 >> v2:172.21.15.183:3300/0 conn(0x7f10a40af4d0 0x7f10a40afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1721720763 >> 172.21.15.97:0/1721720763 conn(0x7f10a4071b90 msgr2=0x7f10a4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1721720763 shutdown_connections 2023-11-24T06:11:54.047 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.041+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1721720763 wait complete. 2023-11-24T06:11:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.043+0000 7f10aa1cd700 1 Processor -- start 2023-11-24T06:11:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.043+0000 7f10aa1cd700 1 -- start start 2023-11-24T06:11:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10aa1cd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a41425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10aa1cd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a40af4d0 0x7f10a4141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10aa1cd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10aa1cd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a41519d0 con 0x7f10a406c000 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10aa1cd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40cf100 con 0x7f10a40af4d0 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10aa1cd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a415a4d0 con 0x7f10a415e080 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a41425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a41425e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50664/0 (socket says 172.21.15.97:50664) 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 -- 172.21.15.97:0/1370615390 learned_addr learned my addr 172.21.15.97:0/1370615390 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a2ffd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a40af4d0 0x7f10a4141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a3fff700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 -- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 msgr2=0x7f10a40cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 -- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a40af4d0 msgr2=0x7f10a4141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a40af4d0 0x7f10a4141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 -- 172.21.15.97:0/1370615390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c009b30 con 0x7f10a406c000 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a2ffd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a40af4d0 0x7f10a4141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a3fff700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10a37fe700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a41425e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f108c00c7f0 tx=0x7f108c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f109bfff700 1 -- 172.21.15.97:0/1370615390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c011800 con 0x7f10a406c000 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f109bfff700 1 -- 172.21.15.97:0/1370615390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c011960 con 0x7f10a406c000 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.044+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a4157280 con 0x7f10a406c000 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.045+0000 7f109bfff700 1 -- 172.21.15.97:0/1370615390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c011cf0 con 0x7f10a406c000 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.045+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a4157610 con 0x7f10a406c000 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.046+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a40753f0 con 0x7f10a406c000 2023-11-24T06:11:54.051 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.046+0000 7f109bfff700 1 -- 172.21.15.97:0/1370615390 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f108c018360 con 0x7f10a406c000 2023-11-24T06:11:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.046+0000 7f109bfff700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f108406b4a0 0x7f108406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.046+0000 7f109bfff700 1 -- 172.21.15.97:0/1370615390 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f108c08a160 con 0x7f10a406c000 2023-11-24T06:11:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.046+0000 7f10a2ffd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f108406b4a0 0x7f108406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.052 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.047+0000 7f10a2ffd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f108406b4a0 0x7f108406d960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1094006fd0 tx=0x7f1094008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.051+0000 7f109bfff700 1 -- 172.21.15.97:0/1370615390 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f108c0547a0 con 0x7f10a406c000 2023-11-24T06:11:54.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.282+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10a4157410 con 0x7f10a406c000 2023-11-24T06:11:54.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.283+0000 7f109bfff700 1 -- 172.21.15.97:0/1370615390 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f108c057dc0 con 0x7f10a406c000 2023-11-24T06:11:54.286 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:54.286 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5863,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2765643379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2765643379},{"type":"v1","addr":"172.21.15.183:6834","nonce":2765643379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1367625334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367625334},{"type":"v1","addr":"172.21.15.97:6836","nonce":1367625334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/807372372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":807372372},{"type":"v1","addr":"172.21.15.97:6837","nonce":807372372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:35.581019+0000","modified":"2023-11-24T06:11:38.607438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3452869588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3452869588},{"type":"v1","addr":"172.21.15.183:6835","nonce":3452869588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:11:54.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f108406b4a0 msgr2=0x7f108406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f108406b4a0 0x7f108406d960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1094006fd0 tx=0x7f1094008040 comp rx=0 tx=0).stop 2023-11-24T06:11:54.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 msgr2=0x7f10a41425e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a41425e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f108c00c7f0 tx=0x7f108c00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 shutdown_connections 2023-11-24T06:11:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f108406b4a0 0x7f108406d960 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f10a415e080 0x7f10a40cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10a406c000 0x7f10a41425e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 --2- 172.21.15.97:0/1370615390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10a40af4d0 0x7f10a4141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 >> 172.21.15.97:0/1370615390 conn(0x7f10a4071b90 msgr2=0x7f10a40b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 shutdown_connections 2023-11-24T06:11:54.289 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.285+0000 7f10aa1cd700 1 -- 172.21.15.97:0/1370615390 wait complete. 2023-11-24T06:11:54.290 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 47 2023-11-24T06:11:54.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:54.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.419+0000 7f8120248700 1 Processor -- start 2023-11-24T06:11:54.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.420+0000 7f8120248700 1 -- start start 2023-11-24T06:11:54.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.420+0000 7f8120248700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f811815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.420+0000 7f8120248700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f81180d0790 0x7f811815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.420+0000 7f8120248700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811815fa50 con 0x7f8118062310 2023-11-24T06:11:54.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.420+0000 7f8120248700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811815fca0 con 0x7f81180d0790 2023-11-24T06:11:54.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.420+0000 7f8120248700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811815ff80 con 0x7f811815c7d0 2023-11-24T06:11:54.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811e7e5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f81180d0790 0x7f811815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811dfe4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f811815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811d7e3700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f8118062310 0x7f81180d0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:52446/0 (socket says 172.21.15.97:52446) 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811d7e3700 1 -- 172.21.15.97:0/4138409678 learned_addr learned my addr 172.21.15.97:0/4138409678 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811e7e5700 1 -- 172.21.15.97:0/4138409678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 msgr2=0x7f811815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811e7e5700 1 --2- 172.21.15.97:0/4138409678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f811815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811e7e5700 1 -- 172.21.15.97:0/4138409678 >> v1:172.21.15.183:6789/0 conn(0x7f8118062310 legacy=0x7f81180d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811e7e5700 1 -- 172.21.15.97:0/4138409678 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8118160260 con 0x7f81180d0790 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811dfe4700 1 --2- 172.21.15.97:0/4138409678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f811815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811e7e5700 1 --2- 172.21.15.97:0/4138409678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 0x7f811815d7d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8110009d90 tx=0x7f811000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9583927d8c31aef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811cfe2700 1 -- 172.21.15.97:0/4138409678 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811000e070 con 0x7f81180d0790 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811cfe2700 1 -- 172.21.15.97:0/4138409678 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811000bb20 con 0x7f81180d0790 2023-11-24T06:11:54.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.421+0000 7f811cfe2700 1 -- 172.21.15.97:0/4138409678 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110012430 con 0x7f81180d0790 2023-11-24T06:11:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.422+0000 7f8120248700 1 -- 172.21.15.97:0/4138409678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 msgr2=0x7f811815d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.422+0000 7f8120248700 1 --2- 172.21.15.97:0/4138409678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 0x7f811815d7d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8110009d90 tx=0x7f811000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.422+0000 7f8120248700 1 -- 172.21.15.97:0/4138409678 shutdown_connections 2023-11-24T06:11:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.422+0000 7f8120248700 1 --2- 172.21.15.97:0/4138409678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 0x7f811815d7d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.422+0000 7f8120248700 1 --2- 172.21.15.97:0/4138409678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f811815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.422+0000 7f8120248700 1 -- 172.21.15.97:0/4138409678 >> 172.21.15.97:0/4138409678 conn(0x7f81180b5640 msgr2=0x7f81180b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:54.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.422+0000 7f8120248700 1 -- 172.21.15.97:0/4138409678 shutdown_connections 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.422+0000 7f8120248700 1 -- 172.21.15.97:0/4138409678 wait complete. 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.423+0000 7f8120248700 1 Processor -- start 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.423+0000 7f8120248700 1 -- start start 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f8120248700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8118062310 0x7f8118148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f8120248700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 0x7f81180ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f8120248700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f81180cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f8120248700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8118148a70 con 0x7f8118062310 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f8120248700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81180ccf70 con 0x7f81180d0790 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f8120248700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81180cd250 con 0x7f811815c7d0 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811dfe4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8118062310 0x7f8118148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.429 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811dfe4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8118062310 0x7f8118148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50682/0 (socket says 172.21.15.97:50682) 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811dfe4700 1 -- 172.21.15.97:0/1736401962 learned_addr learned my addr 172.21.15.97:0/1736401962 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811d7e3700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 0x7f81180ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811e7e5700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f81180cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811dfe4700 1 -- 172.21.15.97:0/1736401962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 msgr2=0x7f81180cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811dfe4700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f81180cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811dfe4700 1 -- 172.21.15.97:0/1736401962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 msgr2=0x7f81180ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811dfe4700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 0x7f81180ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.424+0000 7f811dfe4700 1 -- 172.21.15.97:0/1736401962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81080094a0 con 0x7f8118062310 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.425+0000 7f811dfe4700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8118062310 0x7f8118148530 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f810800ebe0 tx=0x7f810800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.425+0000 7f810e7fc700 1 -- 172.21.15.97:0/1736401962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108009760 con 0x7f8118062310 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.425+0000 7f810e7fc700 1 -- 172.21.15.97:0/1736401962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810800c850 con 0x7f8118062310 2023-11-24T06:11:54.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.425+0000 7f810e7fc700 1 -- 172.21.15.97:0/1736401962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800cbe0 con 0x7f8118062310 2023-11-24T06:11:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.425+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8110009a70 con 0x7f8118062310 2023-11-24T06:11:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.425+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81180cd810 con 0x7f8118062310 2023-11-24T06:11:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.426+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81180cde60 con 0x7f8118062310 2023-11-24T06:11:54.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.426+0000 7f810e7fc700 1 -- 172.21.15.97:0/1736401962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f810800cd40 con 0x7f8118062310 2023-11-24T06:11:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.427+0000 7f810e7fc700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f80fc06b470 0x7f80fc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.427+0000 7f811d7e3700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f80fc06b470 0x7f80fc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.427+0000 7f810e7fc700 1 -- 172.21.15.97:0/1736401962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f810808a0f0 con 0x7f8118062310 2023-11-24T06:11:54.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.427+0000 7f811d7e3700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f80fc06b470 0x7f80fc06d930 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f811806eb10 tx=0x7f8114009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.431+0000 7f810e7fc700 1 -- 172.21.15.97:0/1736401962 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8108054890 con 0x7f8118062310 2023-11-24T06:11:54.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:11:54.439 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:11:54.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.631+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81180d05d0 con 0x7f8118062310 2023-11-24T06:11:54.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.632+0000 7f810e7fc700 1 -- 172.21.15.97:0/1736401962 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16734 (secure 0 0 0) 0x7f8108057eb0 con 0x7f8118062310 2023-11-24T06:11:54.635 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:54.635 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":73,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:52.252147+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:33.973262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:34.541598+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":73,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f80fc06b470 msgr2=0x7f80fc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f80fc06b470 0x7f80fc06d930 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f811806eb10 tx=0x7f8114009040 comp rx=0 tx=0).stop 2023-11-24T06:11:54.637 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8118062310 msgr2=0x7f8118148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8118062310 0x7f8118148530 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f810800ebe0 tx=0x7f810800eef0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 shutdown_connections 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f80fc06b470 0x7f80fc06d930 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f811815c7d0 0x7f81180cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8118062310 0x7f8118148530 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 --2- 172.21.15.97:0/1736401962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81180d0790 0x7f81180ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 >> 172.21.15.97:0/1736401962 conn(0x7f81180b5640 msgr2=0x7f8118156320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 shutdown_connections 2023-11-24T06:11:54.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.634+0000 7f8120248700 1 -- 172.21.15.97:0/1736401962 wait complete. 2023-11-24T06:11:54.649 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:11:54.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:54.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.767+0000 7f46f19f1700 1 Processor -- start 2023-11-24T06:11:54.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.767+0000 7f46f19f1700 1 -- start start 2023-11-24T06:11:54.772 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46f19f1700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46ec0b2b60 0x7f46ec0ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46f19f1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec0cf0d0 0x7f46ec0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46f19f1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46ec06bc00 con 0x7f46ec15e0f0 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46f19f1700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46ec0cf560 con 0x7f46ec0b2b60 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46f19f1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec15e4d0 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46ea7fc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46ec0b2b60 0x7f46ec0ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46eaffd700 1 --1- >> v1:172.21.15.183:6789/0 conn(0x7f46ec15e0f0 0x7f46ec0b9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:52460/0 (socket says 172.21.15.97:52460) 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46eb7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec0cf0d0 0x7f46ec0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46eaffd700 1 -- 172.21.15.97:0/2080708805 learned_addr learned my addr 172.21.15.97:0/2080708805 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.768+0000 7f46ea7fc700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f46ec0b2b60 0x7f46ec0ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:44848/0 (socket says 172.21.15.97:44848) 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46ea7fc700 1 -- 172.21.15.97:0/2080708805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec0cf0d0 msgr2=0x7f46ec0d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:54.773 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46ea7fc700 1 --2- 172.21.15.97:0/2080708805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec0cf0d0 0x7f46ec0d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46ea7fc700 1 -- 172.21.15.97:0/2080708805 >> v1:172.21.15.183:6789/0 conn(0x7f46ec15e0f0 legacy=0x7f46ec0b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46e9ffb700 1 -- 172.21.15.97:0/2080708805 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137381497 0 0) 0x7f46ec06bc00 con 0x7f46ec15e0f0 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46ea7fc700 1 -- 172.21.15.97:0/2080708805 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec06aff0 con 0x7f46ec0b2b60 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46eb7fe700 1 --2- 172.21.15.97:0/2080708805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec0cf0d0 0x7f46ec0d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46ea7fc700 1 --2- 172.21.15.97:0/2080708805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 0x7f46ec0ceb90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009840 tx=0x7f46dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c59781a3a53085d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46e9ffb700 1 -- 172.21.15.97:0/2080708805 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc011830 con 0x7f46ec0b2b60 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46e9ffb700 1 -- 172.21.15.97:0/2080708805 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc011990 con 0x7f46ec0b2b60 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.769+0000 7f46e9ffb700 1 -- 172.21.15.97:0/2080708805 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc011d20 con 0x7f46ec0b2b60 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.770+0000 7f46f19f1700 1 -- 172.21.15.97:0/2080708805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 msgr2=0x7f46ec0ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.770+0000 7f46f19f1700 1 --2- 172.21.15.97:0/2080708805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 0x7f46ec0ceb90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009840 tx=0x7f46dc009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:54.774 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.770+0000 7f46f19f1700 1 -- 172.21.15.97:0/2080708805 shutdown_connections 2023-11-24T06:11:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.770+0000 7f46f19f1700 1 --2- 172.21.15.97:0/2080708805 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec0cf0d0 0x7f46ec0d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.770+0000 7f46f19f1700 1 --2- 172.21.15.97:0/2080708805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 0x7f46ec0ceb90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.770+0000 7f46f19f1700 1 -- 172.21.15.97:0/2080708805 >> 172.21.15.97:0/2080708805 conn(0x7f46ec0b5ce0 msgr2=0x7f46ec0b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.770+0000 7f46f19f1700 1 -- 172.21.15.97:0/2080708805 shutdown_connections 2023-11-24T06:11:54.775 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.770+0000 7f46f19f1700 1 -- 172.21.15.97:0/2080708805 wait complete. 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.771+0000 7f46f19f1700 1 Processor -- start 2023-11-24T06:11:54.777 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.771+0000 7f46f19f1700 1 -- start start 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46f19f1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 0x7f46ec155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46f19f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46ec0cf0d0 0x7f46ec15e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46f19f1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec15e0f0 0x7f46ec160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46f19f1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec15fc00 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46f19f1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec15ef80 con 0x7f46ec0b2b60 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46f19f1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ec15f260 con 0x7f46ec15e0f0 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46ea7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46ec0cf0d0 0x7f46ec15e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46ea7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46ec0cf0d0 0x7f46ec15e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50694/0 (socket says 172.21.15.97:50694) 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46ea7fc700 1 -- 172.21.15.97:0/3140556608 learned_addr learned my addr 172.21.15.97:0/3140556608 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46eaffd700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 0x7f46ec155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.778 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46eb7fe700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec15e0f0 0x7f46ec160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46ea7fc700 1 -- 172.21.15.97:0/3140556608 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec15e0f0 msgr2=0x7f46ec160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46ea7fc700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec15e0f0 0x7f46ec160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46ea7fc700 1 -- 172.21.15.97:0/3140556608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 msgr2=0x7f46ec155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46ea7fc700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 0x7f46ec155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.772+0000 7f46ea7fc700 1 -- 172.21.15.97:0/3140556608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4009a70 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.773+0000 7f46ea7fc700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46ec0cf0d0 0x7f46ec15e610 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009fd0 tx=0x7f46dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.773+0000 7f46d3fff700 1 -- 172.21.15.97:0/3140556608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0038a0 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.773+0000 7f46d3fff700 1 -- 172.21.15.97:0/3140556608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc01f070 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.773+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc0094a0 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.773+0000 7f46d3fff700 1 -- 172.21.15.97:0/3140556608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc003c40 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.773+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec15f820 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.774+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ec15f9b0 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.779 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.774+0000 7f46d3fff700 1 -- 172.21.15.97:0/3140556608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f46dc026480 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.774+0000 7f46d3fff700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46d806b470 0x7f46d806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:54.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.775+0000 7f46d3fff700 1 -- 172.21.15.97:0/3140556608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f46dc021070 con 0x7f46ec0cf0d0 2023-11-24T06:11:54.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.775+0000 7f46eaffd700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46d806b470 0x7f46d806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:54.780 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.775+0000 7f46eaffd700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46d806b470 0x7f46d806d930 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000f60 tx=0x7f46e0007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:54.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:54.779+0000 7f46d3fff700 1 -- 172.21.15.97:0/3140556608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46dc056200 con 0x7f46ec0cf0d0 2023-11-24T06:11:55.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.012+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46ec0751e0 con 0x7f46ec0cf0d0 2023-11-24T06:11:55.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.012+0000 7f46d3fff700 1 -- 172.21.15.97:0/3140556608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f46dc059820 con 0x7f46ec0cf0d0 2023-11-24T06:11:55.015 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:55.015 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5863,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2765643379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2765643379},{"type":"v1","addr":"172.21.15.183:6834","nonce":2765643379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1367625334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367625334},{"type":"v1","addr":"172.21.15.97:6836","nonce":1367625334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/807372372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":807372372},{"type":"v1","addr":"172.21.15.97:6837","nonce":807372372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:35.581019+0000","modified":"2023-11-24T06:11:38.607438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3452869588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3452869588},{"type":"v1","addr":"172.21.15.183:6835","nonce":3452869588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:11:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46d806b470 msgr2=0x7f46d806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46d806b470 0x7f46d806d930 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000f60 tx=0x7f46e0007c90 comp rx=0 tx=0).stop 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46ec0cf0d0 msgr2=0x7f46ec15e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46ec0cf0d0 0x7f46ec15e610 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009fd0 tx=0x7f46dc004750 comp rx=0 tx=0).stop 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 shutdown_connections 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f46d806b470 0x7f46d806d930 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f46ec15e0f0 0x7f46ec160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46ec0b2b60 0x7f46ec155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 --2- 172.21.15.97:0/3140556608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46ec0cf0d0 0x7f46ec15e610 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 >> 172.21.15.97:0/3140556608 conn(0x7f46ec0b5ce0 msgr2=0x7f46ec0b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 shutdown_connections 2023-11-24T06:11:55.018 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.015+0000 7f46f19f1700 1 -- 172.21.15.97:0/3140556608 wait complete. 2023-11-24T06:11:55.019 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 47 2023-11-24T06:11:55.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:11:55.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.147+0000 7f78900ef700 1 Processor -- start 2023-11-24T06:11:55.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.148+0000 7f78900ef700 1 -- start start 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.148+0000 7f78900ef700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78880afe70 0x7f788815e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.148+0000 7f78900ef700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f78881624d0 0x7f7888162930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.148+0000 7f78900ef700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788806dd80 con 0x7f788815bef0 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f78900ef700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788815f880 con 0x7f78881624d0 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f78900ef700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788815ed90 con 0x7f78880afe70 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788de8b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f788815bef0 0x7f78880b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43152/0 (socket says 172.21.15.97:43152) 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788de8b700 1 -- 172.21.15.97:0/3897812434 learned_addr learned my addr 172.21.15.97:0/3897812434 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788e68c700 1 --2- 172.21.15.97:0/3897812434 >> v2:172.21.15.183:3300/0 conn(0x7f78881624d0 0x7f7888162930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788d68a700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78880afe70 0x7f788815e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476574621 0 0) 0x7f788806dd80 con 0x7f788815bef0 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78740035b0 con 0x7f788815bef0 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16209911 0 0) 0x7f78740035b0 con 0x7f788815bef0 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78880afe70 msgr2=0x7f788815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78880afe70 0x7f788815e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 >> v2:172.21.15.183:3300/0 conn(0x7f78881624d0 msgr2=0x7f7888162930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 --2- 172.21.15.97:0/3897812434 >> v2:172.21.15.183:3300/0 conn(0x7f78881624d0 0x7f7888162930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788815c2d0 con 0x7f788815bef0 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f7878001670 con 0x7f788815bef0 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 >> v1:172.21.15.97:6789/0 conn(0x7f788815bef0 legacy=0x7f78880b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78880afe70 0x7f7874003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f7874004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7874004d90 0x7f7874005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788815f880 con 0x7f7874003f80 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788815ed90 con 0x7f7874004d90 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787400ba60 con 0x7f78880afe70 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788d68a700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f7874004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788e68c700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7874004d90 0x7f7874005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.149+0000 7f788de8b700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78880afe70 0x7f7874003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788d68a700 1 -- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78880afe70 msgr2=0x7f7874003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788d68a700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f78880afe70 0x7f7874003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788d68a700 1 -- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7874004d90 msgr2=0x7f7874005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788d68a700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7874004d90 0x7f7874005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788d68a700 1 -- 172.21.15.97:0/3897812434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740035b0 con 0x7f7874003f80 2023-11-24T06:11:55.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788e68c700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7874004d90 0x7f7874005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788d68a700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f7874004850 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7884009b90 tx=0x7f7874005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8022267283b604c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788400e6c0 con 0x7f7874003f80 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788400e820 con 0x7f7874003f80 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788ce89700 1 -- 172.21.15.97:0/3897812434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788400eb90 con 0x7f7874003f80 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f78900ef700 1 -- 172.21.15.97:0/3897812434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 msgr2=0x7f7874004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f78900ef700 1 --2- 172.21.15.97:0/3897812434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f7874004850 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7884009b90 tx=0x7f7874005880 comp rx=0 tx=0).stop 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f788e68c700 1 -- 172.21.15.97:0/3897812434 reap_dead start 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f78900ef700 1 -- 172.21.15.97:0/3897812434 shutdown_connections 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f78900ef700 1 -- 172.21.15.97:0/3897812434 >> 172.21.15.97:0/3897812434 conn(0x7f7888071ac0 msgr2=0x7f7888071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.150+0000 7f78900ef700 1 -- 172.21.15.97:0/3897812434 shutdown_connections 2023-11-24T06:11:55.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.151+0000 7f78900ef700 1 -- 172.21.15.97:0/3897812434 wait complete. 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.152+0000 7f78900ef700 1 Processor -- start 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.152+0000 7f78900ef700 1 -- start start 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.152+0000 7f78900ef700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f788815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.152+0000 7f78900ef700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78881496b0 0x7f788815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.152+0000 7f78900ef700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f788815dc30 0x7f7888158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.152+0000 7f78900ef700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788815e0c0 con 0x7f7874003f80 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.152+0000 7f78900ef700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788815b070 con 0x7f78881496b0 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.152+0000 7f78900ef700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7888159d70 con 0x7f788815dc30 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f788815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f788815ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50712/0 (socket says 172.21.15.97:50712) 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 -- 172.21.15.97:0/1582090165 learned_addr learned my addr 172.21.15.97:0/1582090165 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 -- 172.21.15.97:0/1582090165 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f788815dc30 msgr2=0x7f7888158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f788815dc30 0x7f7888158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 -- 172.21.15.97:0/1582090165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78881496b0 msgr2=0x7f788815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78881496b0 0x7f788815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 -- 172.21.15.97:0/1582090165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788815a050 con 0x7f7874003f80 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f788de8b700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f788815ab30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7878000c00 tx=0x7f787800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f787e7fc700 1 -- 172.21.15.97:0/1582090165 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878012590 con 0x7f7874003f80 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f787e7fc700 1 -- 172.21.15.97:0/1582090165 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78780126f0 con 0x7f7874003f80 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788815a330 con 0x7f7874003f80 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f787e7fc700 1 -- 172.21.15.97:0/1582090165 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878012aa0 con 0x7f7874003f80 2023-11-24T06:11:55.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.153+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7888159270 con 0x7f7874003f80 2023-11-24T06:11:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.154+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7888067f60 con 0x7f7874003f80 2023-11-24T06:11:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.155+0000 7f787e7fc700 1 -- 172.21.15.97:0/1582090165 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f7878012c00 con 0x7f7874003f80 2023-11-24T06:11:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.155+0000 7f787e7fc700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f786c06b470 0x7f786c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.155+0000 7f787e7fc700 1 -- 172.21.15.97:0/1582090165 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f7878089f50 con 0x7f7874003f80 2023-11-24T06:11:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.155+0000 7f788d68a700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f786c06b470 0x7f786c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.155+0000 7f788d68a700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f786c06b470 0x7f786c06d930 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7884000c00 tx=0x7f7884013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.159+0000 7f787e7fc700 1 -- 172.21.15.97:0/1582090165 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7878054730 con 0x7f7874003f80 2023-11-24T06:11:55.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.392+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f788815a850 con 0x7f7874003f80 2023-11-24T06:11:55.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.392+0000 7f787e7fc700 1 -- 172.21.15.97:0/1582090165 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f788815a850 con 0x7f7874003f80 2023-11-24T06:11:55.395 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:55.396 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5863,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2765643379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2765643379},{"type":"v1","addr":"172.21.15.183:6834","nonce":2765643379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1367625334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367625334},{"type":"v1","addr":"172.21.15.97:6836","nonce":1367625334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/807372372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":807372372},{"type":"v1","addr":"172.21.15.97:6837","nonce":807372372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:11:35.581019+0000","modified":"2023-11-24T06:11:38.607438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6474},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6474":{"gid":6474,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3452869588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3452869588},{"type":"v1","addr":"172.21.15.183:6835","nonce":3452869588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f786c06b470 msgr2=0x7f786c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f786c06b470 0x7f786c06d930 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7884000c00 tx=0x7f7884013000 comp rx=0 tx=0).stop 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 msgr2=0x7f788815ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f788815ab30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f7878000c00 tx=0x7f787800ebd0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 shutdown_connections 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f786c06b470 0x7f786c06d930 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f788815dc30 0x7f7888158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7874003f80 0x7f788815ab30 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 --2- 172.21.15.97:0/1582090165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f78881496b0 0x7f788815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 >> 172.21.15.97:0/1582090165 conn(0x7f7888071ac0 msgr2=0x7f78880b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 shutdown_connections 2023-11-24T06:11:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.395+0000 7f78900ef700 1 -- 172.21.15.97:0/1582090165 wait complete. 2023-11-24T06:11:55.399 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 47 2023-11-24T06:11:55.410 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:55.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.527+0000 7f068f7b4700 1 Processor -- start 2023-11-24T06:11:55.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.527+0000 7f068f7b4700 1 -- start start 2023-11-24T06:11:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.528+0000 7f068f7b4700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f06880b3370 0x7f06881604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.528+0000 7f068f7b4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880d1c70 0x7f06880d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.528+0000 7f068f7b4700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06880b22d0 con 0x7f068815bdd0 2023-11-24T06:11:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.528+0000 7f068f7b4700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06880b2f50 con 0x7f06880b3370 2023-11-24T06:11:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.528+0000 7f068f7b4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06880d2640 con 0x7f06880d1c70 2023-11-24T06:11:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.528+0000 7f068d550700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f068815bdd0 0x7f06880b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43172/0 (socket says 172.21.15.97:43172) 2023-11-24T06:11:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.528+0000 7f068d550700 1 -- 172.21.15.97:0/3082090860 learned_addr learned my addr 172.21.15.97:0/3082090860 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:55.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f068dd51700 1 --2- 172.21.15.97:0/3082090860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880d1c70 0x7f06880d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f068cd4f700 1 --2- 172.21.15.97:0/3082090860 >> v2:172.21.15.183:3300/0 conn(0x7f06880b3370 0x7f06881604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f067ffff700 1 -- 172.21.15.97:0/3082090860 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540880338 0 0) 0x7f06880b22d0 con 0x7f068815bdd0 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f067ffff700 1 -- 172.21.15.97:0/3082090860 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06740035b0 con 0x7f068815bdd0 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f067ffff700 1 -- 172.21.15.97:0/3082090860 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2503312855 0 0) 0x7f06740035b0 con 0x7f068815bdd0 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f068dd51700 1 -- 172.21.15.97:0/3082090860 >> v2:172.21.15.183:3300/0 conn(0x7f06880b3370 msgr2=0x7f06881604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f068dd51700 1 --2- 172.21.15.97:0/3082090860 >> v2:172.21.15.183:3300/0 conn(0x7f06880b3370 0x7f06881604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f068dd51700 1 -- 172.21.15.97:0/3082090860 >> v1:172.21.15.97:6789/0 conn(0x7f068815bdd0 legacy=0x7f06880b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f068dd51700 1 -- 172.21.15.97:0/3082090860 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880b25b0 con 0x7f06880d1c70 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f068cd4f700 1 --2- 172.21.15.97:0/3082090860 >> v2:172.21.15.183:3300/0 conn(0x7f06880b3370 0x7f06881604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f068dd51700 1 --2- 172.21.15.97:0/3082090860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880d1c70 0x7f06880d2100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0680009d90 tx=0x7f068000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb398d8a7224f68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f067ffff700 1 -- 172.21.15.97:0/3082090860 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068000e070 con 0x7f06880d1c70 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f067ffff700 1 -- 172.21.15.97:0/3082090860 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0680003a20 con 0x7f06880d1c70 2023-11-24T06:11:55.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.529+0000 7f067ffff700 1 -- 172.21.15.97:0/3082090860 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678001670 con 0x7f06880d1c70 2023-11-24T06:11:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.530+0000 7f068f7b4700 1 -- 172.21.15.97:0/3082090860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880d1c70 msgr2=0x7f06880d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.530+0000 7f068f7b4700 1 --2- 172.21.15.97:0/3082090860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880d1c70 0x7f06880d2100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0680009d90 tx=0x7f068000b330 comp rx=0 tx=0).stop 2023-11-24T06:11:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.530+0000 7f068f7b4700 1 -- 172.21.15.97:0/3082090860 shutdown_connections 2023-11-24T06:11:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.530+0000 7f068f7b4700 1 --2- 172.21.15.97:0/3082090860 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880d1c70 0x7f06880d2100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.530+0000 7f068f7b4700 1 --2- 172.21.15.97:0/3082090860 >> v2:172.21.15.183:3300/0 conn(0x7f06880b3370 0x7f06881604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.530+0000 7f068f7b4700 1 -- 172.21.15.97:0/3082090860 >> 172.21.15.97:0/3082090860 conn(0x7f06880731a0 msgr2=0x7f068809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:55.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.530+0000 7f068f7b4700 1 -- 172.21.15.97:0/3082090860 shutdown_connections 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.530+0000 7f068f7b4700 1 -- 172.21.15.97:0/3082090860 wait complete. 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.531+0000 7f068f7b4700 1 Processor -- start 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.531+0000 7f068f7b4700 1 -- start start 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068f7b4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880b3370 0x7f06880c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068f7b4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06880d1c70 0x7f06881570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068f7b4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f068815bdd0 0x7f068806f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068f7b4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880c7fc0 con 0x7f068815bdd0 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068f7b4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880c7340 con 0x7f06880d1c70 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068f7b4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06880c7620 con 0x7f06880b3370 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f068815bdd0 0x7f068806f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f068815bdd0 0x7f068806f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50730/0 (socket says 172.21.15.97:50730) 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 -- 172.21.15.97:0/2308667763 learned_addr learned my addr 172.21.15.97:0/2308667763 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:55.537 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068cd4f700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06880d1c70 0x7f06881570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068d550700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880b3370 0x7f06880c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 -- 172.21.15.97:0/2308667763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880b3370 msgr2=0x7f06880c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880b3370 0x7f06880c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 -- 172.21.15.97:0/2308667763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06880d1c70 msgr2=0x7f06881570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06880d1c70 0x7f06881570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 -- 172.21.15.97:0/2308667763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840094a0 con 0x7f068815bdd0 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.532+0000 7f068dd51700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f068815bdd0 0x7f068806f4e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f06880d0700 tx=0x7f0680004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.533+0000 7f067dffb700 1 -- 172.21.15.97:0/2308667763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680004540 con 0x7f068815bdd0 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.533+0000 7f067dffb700 1 -- 172.21.15.97:0/2308667763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06800046e0 con 0x7f068815bdd0 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.533+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0680009a70 con 0x7f068815bdd0 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.533+0000 7f067dffb700 1 -- 172.21.15.97:0/2308667763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068001c440 con 0x7f068815bdd0 2023-11-24T06:11:55.538 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.533+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06880c7be0 con 0x7f068815bdd0 2023-11-24T06:11:55.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.534+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0688067f60 con 0x7f068815bdd0 2023-11-24T06:11:55.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.534+0000 7f067dffb700 1 -- 172.21.15.97:0/2308667763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f068001c5a0 con 0x7f068815bdd0 2023-11-24T06:11:55.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.535+0000 7f067dffb700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f066c06b470 0x7f066c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.535+0000 7f068d550700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f066c06b470 0x7f066c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.535+0000 7f067dffb700 1 -- 172.21.15.97:0/2308667763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f068008bb10 con 0x7f068815bdd0 2023-11-24T06:11:55.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.535+0000 7f068d550700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f066c06b470 0x7f066c06d930 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f06780011e0 tx=0x7f0678001da0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.543 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.541+0000 7f067dffb700 1 -- 172.21.15.97:0/2308667763 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0680056270 con 0x7f068815bdd0 2023-11-24T06:11:55.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.734+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06880c8100 con 0x7f068815bdd0 2023-11-24T06:11:55.737 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.735+0000 7f067dffb700 1 -- 172.21.15.97:0/2308667763 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16734 (secure 0 0 0) 0x7f06880c8100 con 0x7f068815bdd0 2023-11-24T06:11:55.738 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:55.738 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":73,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:52.252147+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:11:33.973262+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T06:11:34.541598+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":73,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f066c06b470 msgr2=0x7f066c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f066c06b470 0x7f066c06d930 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f06780011e0 tx=0x7f0678001da0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f068815bdd0 msgr2=0x7f068806f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f068815bdd0 0x7f068806f4e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f06880d0700 tx=0x7f0680004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 shutdown_connections 2023-11-24T06:11:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f068815bdd0 0x7f068806f4e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f066c06b470 0x7f066c06d930 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f06880b3370 0x7f06880c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 --2- 172.21.15.97:0/2308667763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06880d1c70 0x7f06881570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 >> 172.21.15.97:0/2308667763 conn(0x7f06880731a0 msgr2=0x7f06880702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 shutdown_connections 2023-11-24T06:11:55.741 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.737+0000 7f068f7b4700 1 -- 172.21.15.97:0/2308667763 wait complete. 2023-11-24T06:11:55.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:11:55.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.872+0000 7f8a53b66700 1 Processor -- start 2023-11-24T06:11:55.875 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.872+0000 7f8a53b66700 1 -- start start 2023-11-24T06:11:55.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a53b66700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a4c0b3370 0x7f8a4c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.876 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a53b66700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c0b22d0 con 0x7f8a4c15bdd0 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a53b66700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4c0b25b0 con 0x7f8a4c0d1c70 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a53b66700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4c0b2f50 con 0x7f8a4c0b3370 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a51101700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a4c0b3370 0x7f8a4c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a51902700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8a4c15bdd0 0x7f8a4c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43186/0 (socket says 172.21.15.97:43186) 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a51902700 1 -- 172.21.15.97:0/534422545 learned_addr learned my addr 172.21.15.97:0/534422545 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a51101700 1 -- 172.21.15.97:0/534422545 >> v1:172.21.15.183:6789/0 conn(0x7f8a4c0d1c70 legacy=0x7f8a4c165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a51101700 1 -- 172.21.15.97:0/534422545 >> v1:172.21.15.97:6789/0 conn(0x7f8a4c15bdd0 legacy=0x7f8a4c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.873+0000 7f8a51101700 1 -- 172.21.15.97:0/534422545 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4c0b2890 con 0x7f8a4c0b3370 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a51101700 1 --2- 172.21.15.97:0/534422545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1604e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009840 tx=0x7f8a48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d63e929df32a669 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a50900700 1 -- 172.21.15.97:0/534422545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48004230 con 0x7f8a4c0b3370 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a50900700 1 -- 172.21.15.97:0/534422545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a48004390 con 0x7f8a4c0b3370 2023-11-24T06:11:55.877 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a50900700 1 -- 172.21.15.97:0/534422545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a480036a0 con 0x7f8a4c0b3370 2023-11-24T06:11:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a53b66700 1 -- 172.21.15.97:0/534422545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 msgr2=0x7f8a4c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a53b66700 1 --2- 172.21.15.97:0/534422545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1604e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009840 tx=0x7f8a48009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a53b66700 1 -- 172.21.15.97:0/534422545 shutdown_connections 2023-11-24T06:11:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a53b66700 1 --2- 172.21.15.97:0/534422545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1604e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a53b66700 1 -- 172.21.15.97:0/534422545 >> 172.21.15.97:0/534422545 conn(0x7f8a4c0731a0 msgr2=0x7f8a4c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a53b66700 1 -- 172.21.15.97:0/534422545 shutdown_connections 2023-11-24T06:11:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.874+0000 7f8a53b66700 1 -- 172.21.15.97:0/534422545 wait complete. 2023-11-24T06:11:55.880 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a53b66700 1 Processor -- start 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a53b66700 1 -- start start 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a53b66700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a53b66700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a4c0d1c70 0x7f8a4c1f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a53b66700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a4c15bdd0 0x7f8a4c0c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a53b66700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c15eb90 con 0x7f8a4c0b3370 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a53b66700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c1f61d0 con 0x7f8a4c15bdd0 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a53b66700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c1f63e0 con 0x7f8a4c0d1c70 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a51902700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a51902700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50740/0 (socket says 172.21.15.97:50740) 2023-11-24T06:11:55.881 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.876+0000 7f8a51902700 1 -- 172.21.15.97:0/1127403226 learned_addr learned my addr 172.21.15.97:0/1127403226 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a51902700 1 -- 172.21.15.97:0/1127403226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a4c0d1c70 msgr2=0x7f8a4c1f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a51902700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a4c0d1c70 0x7f8a4c1f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a51902700 1 -- 172.21.15.97:0/1127403226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a4c15bdd0 msgr2=0x7f8a4c0c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a51902700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a4c15bdd0 0x7f8a4c0c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a51902700 1 -- 172.21.15.97:0/1127403226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a480094a0 con 0x7f8a4c0b3370 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a51902700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1572d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00c430 tx=0x7f8a3c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a427fc700 1 -- 172.21.15.97:0/1127403226 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c005d70 con 0x7f8a4c0b3370 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a427fc700 1 -- 172.21.15.97:0/1127403226 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c0070d0 con 0x7f8a4c0b3370 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a427fc700 1 -- 172.21.15.97:0/1127403226 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c007460 con 0x7f8a4c0b3370 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c1f66c0 con 0x7f8a4c0b3370 2023-11-24T06:11:55.882 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.877+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c1e5900 con 0x7f8a4c0b3370 2023-11-24T06:11:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.878+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c1f6be0 con 0x7f8a4c0b3370 2023-11-24T06:11:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.878+0000 7f8a427fc700 1 -- 172.21.15.97:0/1127403226 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8a3c0075c0 con 0x7f8a4c0b3370 2023-11-24T06:11:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.879+0000 7f8a427fc700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a3006b470 0x7f8a3006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:55.883 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.879+0000 7f8a427fc700 1 -- 172.21.15.97:0/1127403226 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f8a3c089500 con 0x7f8a4c0b3370 2023-11-24T06:11:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.879+0000 7f8a51101700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a3006b470 0x7f8a3006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:55.884 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.879+0000 7f8a51101700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a3006b470 0x7f8a3006d930 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c15e260 tx=0x7f8a48010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:55.886 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:55.883+0000 7f8a427fc700 1 -- 172.21.15.97:0/1127403226 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a4c1f6be0 con 0x7f8a4c0b3370 2023-11-24T06:11:56.120 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.117+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8a4c1e5760 con 0x7f8a4c0b3370 2023-11-24T06:11:56.120 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-11-24T06:11:56.118+0000 7fbb14a83700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:11:56.121 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.119+0000 7f8a427fc700 1 -- 172.21.15.97:0/1127403226 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8a4c1e5760 con 0x7f8a4c0b3370 2023-11-24T06:11:56.124 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a3006b470 msgr2=0x7f8a3006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a3006b470 0x7f8a3006d930 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c15e260 tx=0x7f8a48010000 comp rx=0 tx=0).stop 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 msgr2=0x7f8a4c1572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1572d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00c430 tx=0x7f8a3c00c7f0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 shutdown_connections 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a4c15bdd0 0x7f8a4c0c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0b3370 0x7f8a4c1572d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f8a3006b470 0x7f8a3006d930 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 --2- 172.21.15.97:0/1127403226 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f8a4c0d1c70 0x7f8a4c1f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 >> 172.21.15.97:0/1127403226 conn(0x7f8a4c0731a0 msgr2=0x7f8a4c0b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 shutdown_connections 2023-11-24T06:11:56.125 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.122+0000 7f8a53b66700 1 -- 172.21.15.97:0/1127403226 wait complete. 2023-11-24T06:11:56.126 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:11:56.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:11:56.164 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-11-24T06:11:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.257+0000 7f0c9ca0d700 1 Processor -- start 2023-11-24T06:11:56.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.257+0000 7f0c9ca0d700 1 -- start start 2023-11-24T06:11:56.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c9ca0d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c9ca0d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0c9815e040 0x7f0c9815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c9ca0d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9815e9e0 con 0x7f0c9806dec0 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c9ca0d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9806ee40 con 0x7f0c9815e040 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c9ca0d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980b2b00 con 0x7f0c98151660 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c96d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0c9815e040 0x7f0c9815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c96d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0c9815e040 0x7f0c9815e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50748/0 (socket says 172.21.15.97:50748) 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c96d9d700 1 -- 172.21.15.97:0/452948946 learned_addr learned my addr 172.21.15.97:0/452948946 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.258+0000 7f0c9659c700 1 --2- 172.21.15.97:0/452948946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c96d9d700 1 -- 172.21.15.97:0/452948946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 msgr2=0x7f0c980cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c96d9d700 1 --2- 172.21.15.97:0/452948946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c96d9d700 1 -- 172.21.15.97:0/452948946 >> v1:172.21.15.183:6789/0 conn(0x7f0c9806dec0 legacy=0x7f0c98070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c96d9d700 1 -- 172.21.15.97:0/452948946 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9806e3a0 con 0x7f0c9815e040 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c96d9d700 1 --2- 172.21.15.97:0/452948946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9815e040 0x7f0c9815e4a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000d180 tx=0x7f0c9000d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ae82d41d6808f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c9559a700 1 -- 172.21.15.97:0/452948946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90010070 con 0x7f0c9815e040 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c9559a700 1 -- 172.21.15.97:0/452948946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9000dd00 con 0x7f0c9815e040 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c9559a700 1 -- 172.21.15.97:0/452948946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000b310 con 0x7f0c9815e040 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/452948946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9815e040 msgr2=0x7f0c9815e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.259+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/452948946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9815e040 0x7f0c9815e4a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000d180 tx=0x7f0c9000d490 comp rx=0 tx=0).stop 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.260+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/452948946 shutdown_connections 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.260+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/452948946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9815e040 0x7f0c9815e4a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.264 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.260+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/452948946 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.260+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/452948946 >> 172.21.15.97:0/452948946 conn(0x7f0c9806b410 msgr2=0x7f0c9806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.260+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/452948946 shutdown_connections 2023-11-24T06:11:56.265 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.260+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/452948946 wait complete. 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.261+0000 7f0c9ca0d700 1 Processor -- start 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.261+0000 7f0c9ca0d700 1 -- start start 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9ca0d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9806dec0 0x7f0c9814f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9ca0d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9ca0d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c9815e040 0x7f0c980cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9ca0d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9814f870 con 0x7f0c9806dec0 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9ca0d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981544f0 con 0x7f0c9815e040 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9ca0d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c980cb7b0 con 0x7f0c98151660 2023-11-24T06:11:56.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9806dec0 0x7f0c9814f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9806dec0 0x7f0c9814f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50752/0 (socket says 172.21.15.97:50752) 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 -- 172.21.15.97:0/1078475084 learned_addr learned my addr 172.21.15.97:0/1078475084 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c96d9d700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c9815e040 0x7f0c980cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c95d9b700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 -- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 msgr2=0x7f0c980cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 -- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c9815e040 msgr2=0x7f0c980cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c9815e040 0x7f0c980cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 -- 172.21.15.97:0/1078475084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9000bd40 con 0x7f0c9806dec0 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c95d9b700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c96d9d700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c9815e040 0x7f0c980cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:56.269 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9659c700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9806dec0 0x7f0c9814f330 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e9e0 tx=0x7f0c8800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c86ffd700 1 -- 172.21.15.97:0/1078475084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88009710 con 0x7f0c9806dec0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c86ffd700 1 -- 172.21.15.97:0/1078475084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8800c850 con 0x7f0c9806dec0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98159520 con 0x7f0c9806dec0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c86ffd700 1 -- 172.21.15.97:0/1078475084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8800cbc0 con 0x7f0c9806dec0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.262+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c98158a40 con 0x7f0c9806dec0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.263+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c98068480 con 0x7f0c9806dec0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.264+0000 7f0c86ffd700 1 -- 172.21.15.97:0/1078475084 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0c8800cd20 con 0x7f0c9806dec0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.264+0000 7f0c86ffd700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c7806b470 0x7f0c7806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.264+0000 7f0c86ffd700 1 -- 172.21.15.97:0/1078475084 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f0c8808a1d0 con 0x7f0c9806dec0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.264+0000 7f0c95d9b700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c7806b470 0x7f0c7806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:56.270 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.265+0000 7f0c95d9b700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c7806b470 0x7f0c7806d930 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0c98072b20 tx=0x7f0c8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:56.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.269+0000 7f0c86ffd700 1 -- 172.21.15.97:0/1078475084 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c88054980 con 0x7f0c9806dec0 2023-11-24T06:11:56.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:56.500+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0c98068fb0 con 0x7f0c9806dec0 2023-11-24T06:11:57.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.124+0000 7f0c86ffd700 1 -- 172.21.15.97:0/1078475084 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0c88057fa0 con 0x7f0c9806dec0 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c7806b470 msgr2=0x7f0c7806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c7806b470 0x7f0c7806d930 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0c98072b20 tx=0x7f0c8c009040 comp rx=0 tx=0).stop 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9806dec0 msgr2=0x7f0c9814f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9806dec0 0x7f0c9814f330 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800e9e0 tx=0x7f0c8800eda0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 shutdown_connections 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c9815e040 0x7f0c980cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c9806dec0 0x7f0c9814f330 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0c7806b470 0x7f0c7806d930 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.130 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 --2- 172.21.15.97:0/1078475084 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0c98151660 0x7f0c980cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 >> 172.21.15.97:0/1078475084 conn(0x7f0c9806b410 msgr2=0x7f0c98070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:57.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 shutdown_connections 2023-11-24T06:11:57.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.127+0000 7f0c9ca0d700 1 -- 172.21.15.97:0/1078475084 wait complete. 2023-11-24T06:11:57.144 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:11:57.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.271+0000 7ff58e066700 1 Processor -- start 2023-11-24T06:11:57.274 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.271+0000 7ff58e066700 1 -- start start 2023-11-24T06:11:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff58e066700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5880d1b00 0x7ff5880d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff58e066700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5880d2450 con 0x7ff58815de80 2023-11-24T06:11:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff58e066700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58806f810 con 0x7ff58815e260 2023-11-24T06:11:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff58e066700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58806b580 con 0x7ff5880d1b00 2023-11-24T06:11:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5877fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff58815de80 0x7ff5880cfd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43212/0 (socket says 172.21.15.97:43212) 2023-11-24T06:11:57.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5877fe700 1 -- 172.21.15.97:0/370442857 learned_addr learned my addr 172.21.15.97:0/370442857 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519187688 0 0) 0x7ff5880d2450 con 0x7ff58815de80 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff587fff700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5880d1b00 0x7ff5880d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5740035b0 con 0x7ff58815de80 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3244166000 0 0) 0x7ff5740035b0 con 0x7ff58815de80 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5880d1b00 msgr2=0x7ff5880d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5867fc700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5880d1b00 0x7ff5880d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 >> v1:172.21.15.183:6789/0 conn(0x7ff58815e260 legacy=0x7ff58806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.272+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58806faf0 con 0x7ff58815de80 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7ff570001670 con 0x7ff58815de80 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 >> v1:172.21.15.97:6789/0 conn(0x7ff58815de80 legacy=0x7ff5880cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.278 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff574003a30 0x7ff574003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5880d1b00 0x7ff574004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574004d90 0x7ff574005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58806b580 con 0x7ff574003a30 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57400bbd0 con 0x7ff574004d90 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57400bd90 con 0x7ff5880d1b00 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5877fe700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff574003a30 0x7ff574003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff587fff700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574004d90 0x7ff574005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff586ffd700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5880d1b00 0x7ff574004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5877fe700 1 -- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5880d1b00 msgr2=0x7ff574004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5877fe700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5880d1b00 0x7ff574004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5877fe700 1 -- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574004d90 msgr2=0x7ff574005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5877fe700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574004d90 0x7ff574005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5877fe700 1 -- 172.21.15.97:0/370442857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740035b0 con 0x7ff574003a30 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5877fe700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff574003a30 0x7ff574003e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c240 tx=0x7ff574005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad4792a7b6c22f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570007c10 con 0x7ff574003a30 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57000cde0 con 0x7ff574003a30 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.273+0000 7ff5867fc700 1 -- 172.21.15.97:0/370442857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570005ba0 con 0x7ff574003a30 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.274+0000 7ff58e066700 1 -- 172.21.15.97:0/370442857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff574003a30 msgr2=0x7ff574003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.274+0000 7ff58e066700 1 --2- 172.21.15.97:0/370442857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff574003a30 0x7ff574003e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c240 tx=0x7ff574005880 comp rx=0 tx=0).stop 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.274+0000 7ff587fff700 1 -- 172.21.15.97:0/370442857 reap_dead start 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.274+0000 7ff58e066700 1 -- 172.21.15.97:0/370442857 shutdown_connections 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.274+0000 7ff58e066700 1 -- 172.21.15.97:0/370442857 >> 172.21.15.97:0/370442857 conn(0x7ff58809aa00 msgr2=0x7ff58809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:57.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.274+0000 7ff58e066700 1 -- 172.21.15.97:0/370442857 shutdown_connections 2023-11-24T06:11:57.281 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.274+0000 7ff58e066700 1 -- 172.21.15.97:0/370442857 wait complete. 2023-11-24T06:11:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.275+0000 7ff58e066700 1 Processor -- start 2023-11-24T06:11:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.275+0000 7ff58e066700 1 -- start start 2023-11-24T06:11:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff58e066700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574003a30 0x7ff5880cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff58e066700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff588146bb0 0x7ff5881f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff58e066700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5881606f0 0x7ff588160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff58e066700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5881610b0 con 0x7ff5881606f0 2023-11-24T06:11:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff58e066700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58815fcc0 con 0x7ff574003a30 2023-11-24T06:11:57.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff58e066700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58815f140 con 0x7ff588146bb0 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff587fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5881606f0 0x7ff588160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff587fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5881606f0 0x7ff588160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50774/0 (socket says 172.21.15.97:50774) 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff587fff700 1 -- 172.21.15.97:0/3784468351 learned_addr learned my addr 172.21.15.97:0/3784468351 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff5877fe700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574003a30 0x7ff5880cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.276+0000 7ff586ffd700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff588146bb0 0x7ff5881f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff587fff700 1 -- 172.21.15.97:0/3784468351 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff588146bb0 msgr2=0x7ff5881f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff587fff700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff588146bb0 0x7ff5881f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff587fff700 1 -- 172.21.15.97:0/3784468351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574003a30 msgr2=0x7ff5880cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff587fff700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574003a30 0x7ff5880cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff587fff700 1 -- 172.21.15.97:0/3784468351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58815f390 con 0x7ff5881606f0 2023-11-24T06:11:57.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff587fff700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5881606f0 0x7ff588160b70 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff58806e660 tx=0x7ff580011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff57ffff700 1 -- 172.21.15.97:0/3784468351 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000e040 con 0x7ff5881606f0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff57ffff700 1 -- 172.21.15.97:0/3784468351 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58000f040 con 0x7ff5881606f0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58815f670 con 0x7ff5881606f0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff57ffff700 1 -- 172.21.15.97:0/3784468351 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000fc70 con 0x7ff5881606f0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.277+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5880cba70 con 0x7ff5881606f0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.278+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5880cd220 con 0x7ff5881606f0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.278+0000 7ff57ffff700 1 -- 172.21.15.97:0/3784468351 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff580022460 con 0x7ff5881606f0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.279+0000 7ff57ffff700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff56806b470 0x7ff56806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.279+0000 7ff57ffff700 1 -- 172.21.15.97:0/3784468351 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ff58008a560 con 0x7ff5881606f0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.279+0000 7ff5877fe700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff56806b470 0x7ff56806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.285 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.279+0000 7ff5877fe700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff56806b470 0x7ff56806d930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff570015040 tx=0x7ff570007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.286 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.283+0000 7ff57ffff700 1 -- 172.21.15.97:0/3784468351 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff580054c00 con 0x7ff5881606f0 2023-11-24T06:11:57.480 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.477+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff58815fe70 con 0x7ff5881606f0 2023-11-24T06:11:57.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.513+0000 7ff57ffff700 1 -- 172.21.15.97:0/3784468351 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff58815fe70 con 0x7ff5881606f0 2023-11-24T06:11:57.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.516+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff56806b470 msgr2=0x7ff56806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.516+0000 7ff58e066700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff56806b470 0x7ff56806d930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff570015040 tx=0x7ff570007000 comp rx=0 tx=0).stop 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.516+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5881606f0 msgr2=0x7ff588160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.516+0000 7ff58e066700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5881606f0 0x7ff588160b70 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7ff58806e660 tx=0x7ff580011ed0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.517+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 shutdown_connections 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.517+0000 7ff58e066700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5881606f0 0x7ff588160b70 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.517+0000 7ff58e066700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff574003a30 0x7ff5880cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.517+0000 7ff58e066700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff56806b470 0x7ff56806d930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.517+0000 7ff58e066700 1 --2- 172.21.15.97:0/3784468351 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff588146bb0 0x7ff5881f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.517+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 >> 172.21.15.97:0/3784468351 conn(0x7ff58809aa00 msgr2=0x7ff588154170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.517+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 shutdown_connections 2023-11-24T06:11:57.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.517+0000 7ff58e066700 1 -- 172.21.15.97:0/3784468351 wait complete. 2023-11-24T06:11:57.594 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:11:57.594 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:11:57.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.716+0000 7f2f9867b700 1 Processor -- start 2023-11-24T06:11:57.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.716+0000 7f2f9867b700 1 -- start start 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f9867b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f900b01e0 0x7f2f9006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f9867b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 0x7f2f900b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f9867b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f900b5e40 con 0x7f2f90073f90 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f9867b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f901470a0 con 0x7f2f900b01e0 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f9867b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f900b05c0 con 0x7f2f900b59c0 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f96417700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2f900b01e0 0x7f2f9006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f96c18700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2f90073f90 0x7f2f9006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43236/0 (socket says 172.21.15.97:43236) 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f96c18700 1 -- 172.21.15.97:0/3506362957 learned_addr learned my addr 172.21.15.97:0/3506362957 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f96417700 1 -- 172.21.15.97:0/3506362957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 msgr2=0x7f2f900b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:57.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f96417700 1 --2- 172.21.15.97:0/3506362957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 0x7f2f900b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f96417700 1 -- 172.21.15.97:0/3506362957 >> v1:172.21.15.97:6789/0 conn(0x7f2f90073f90 legacy=0x7f2f9006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f96417700 1 -- 172.21.15.97:0/3506362957 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f900b4250 con 0x7f2f900b01e0 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f96417700 1 --2- 172.21.15.97:0/3506362957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9006e9c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009840 tx=0x7f2f8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6e8aa63ce85d08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f95415700 1 -- 172.21.15.97:0/3506362957 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c011830 con 0x7f2f900b01e0 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f95415700 1 -- 172.21.15.97:0/3506362957 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c011990 con 0x7f2f900b01e0 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.717+0000 7f2f95415700 1 -- 172.21.15.97:0/3506362957 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c011d20 con 0x7f2f900b01e0 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.718+0000 7f2f9867b700 1 -- 172.21.15.97:0/3506362957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 msgr2=0x7f2f9006e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.718+0000 7f2f9867b700 1 --2- 172.21.15.97:0/3506362957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9006e9c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009840 tx=0x7f2f8c009b50 comp rx=0 tx=0).stop 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.718+0000 7f2f9867b700 1 -- 172.21.15.97:0/3506362957 shutdown_connections 2023-11-24T06:11:57.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.718+0000 7f2f9867b700 1 --2- 172.21.15.97:0/3506362957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9006e9c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.718+0000 7f2f9867b700 1 --2- 172.21.15.97:0/3506362957 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 0x7f2f900b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.718+0000 7f2f9867b700 1 -- 172.21.15.97:0/3506362957 >> 172.21.15.97:0/3506362957 conn(0x7f2f900d0b40 msgr2=0x7f2f900d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.718+0000 7f2f9867b700 1 -- 172.21.15.97:0/3506362957 shutdown_connections 2023-11-24T06:11:57.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.718+0000 7f2f9867b700 1 -- 172.21.15.97:0/3506362957 wait complete. 2023-11-24T06:11:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.719+0000 7f2f9867b700 1 Processor -- start 2023-11-24T06:11:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.719+0000 7f2f9867b700 1 -- start start 2023-11-24T06:11:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f9867b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f90073f90 0x7f2f901f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f9867b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9013e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f9867b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 0x7f2f9013e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f9867b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f901e92d0 con 0x7f2f900b01e0 2023-11-24T06:11:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f9867b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f901e86c0 con 0x7f2f90073f90 2023-11-24T06:11:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f9867b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f901e89a0 con 0x7f2f900b59c0 2023-11-24T06:11:57.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f95c16700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9013e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f95c16700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9013e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50788/0 (socket says 172.21.15.97:50788) 2023-11-24T06:11:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f95c16700 1 -- 172.21.15.97:0/645665053 learned_addr learned my addr 172.21.15.97:0/645665053 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f96417700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f90073f90 0x7f2f901f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f96c18700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 0x7f2f9013e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f95c16700 1 -- 172.21.15.97:0/645665053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 msgr2=0x7f2f9013e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f95c16700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 0x7f2f9013e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f95c16700 1 -- 172.21.15.97:0/645665053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f90073f90 msgr2=0x7f2f901f3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f95c16700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f90073f90 0x7f2f901f3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.720+0000 7f2f95c16700 1 -- 172.21.15.97:0/645665053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c0094a0 con 0x7f2f900b01e0 2023-11-24T06:11:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.721+0000 7f2f95c16700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9013e340 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000b530 tx=0x7f2f8000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.721+0000 7f2f86ffd700 1 -- 172.21.15.97:0/645665053 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80011830 con 0x7f2f900b01e0 2023-11-24T06:11:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.721+0000 7f2f86ffd700 1 -- 172.21.15.97:0/645665053 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f80011990 con 0x7f2f900b01e0 2023-11-24T06:11:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.721+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f901e8c80 con 0x7f2f900b01e0 2023-11-24T06:11:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.721+0000 7f2f86ffd700 1 -- 172.21.15.97:0/645665053 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80011d40 con 0x7f2f900b01e0 2023-11-24T06:11:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.721+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f90160f90 con 0x7f2f900b01e0 2023-11-24T06:11:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.722+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f9006e9c0 con 0x7f2f900b01e0 2023-11-24T06:11:57.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.722+0000 7f2f86ffd700 1 -- 172.21.15.97:0/645665053 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2f8001a400 con 0x7f2f900b01e0 2023-11-24T06:11:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.722+0000 7f2f86ffd700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f7406b470 0x7f2f7406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.722+0000 7f2f86ffd700 1 -- 172.21.15.97:0/645665053 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5413+0+0 (secure 0 0 0) 0x7f2f80089f50 con 0x7f2f900b01e0 2023-11-24T06:11:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.722+0000 7f2f96417700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f7406b470 0x7f2f7406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:57.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.723+0000 7f2f96417700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f7406b470 0x7f2f7406d930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c000c00 tx=0x7f2f8c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:57.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.727+0000 7f2f86ffd700 1 -- 172.21.15.97:0/645665053 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f800548e0 con 0x7f2f900b01e0 2023-11-24T06:11:57.921 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:57.919+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2f90067f60 con 0x7f2f900b01e0 2023-11-24T06:11:58.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.542+0000 7f2f86ffd700 1 -- 172.21.15.97:0/645665053 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2f80015070 con 0x7f2f900b01e0 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f7406b470 msgr2=0x7f2f7406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f7406b470 0x7f2f7406d930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c000c00 tx=0x7f2f8c008030 comp rx=0 tx=0).stop 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 msgr2=0x7f2f9013e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9013e340 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000b530 tx=0x7f2f8000b840 comp rx=0 tx=0).stop 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 shutdown_connections 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2f7406b470 0x7f2f7406d930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2f900b59c0 0x7f2f9013e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f90073f90 0x7f2f901f3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 --2- 172.21.15.97:0/645665053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f900b01e0 0x7f2f9013e340 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 >> 172.21.15.97:0/645665053 conn(0x7f2f900d0b40 msgr2=0x7f2f900b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 shutdown_connections 2023-11-24T06:11:58.549 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.545+0000 7f2f9867b700 1 -- 172.21.15.97:0/645665053 wait complete. 2023-11-24T06:11:58.549 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-11-24T06:11:58.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:11:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.683+0000 7f581a13e700 1 Processor -- start 2023-11-24T06:11:58.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f581a13e700 1 -- start start 2023-11-24T06:11:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f581a13e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f581415b370 0x7f581415b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f581a13e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58140b3c60 0x7f58140b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f581a13e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58140ae5e0 con 0x7f58141587f0 2023-11-24T06:11:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f581a13e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58140b4630 con 0x7f58140b3c60 2023-11-24T06:11:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f581a13e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581415be30 con 0x7f581415b370 2023-11-24T06:11:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f5813fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58140b3c60 0x7f58140b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f5813fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58140b3c60 0x7f58140b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50798/0 (socket says 172.21.15.97:50798) 2023-11-24T06:11:58.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.684+0000 7f5813fff700 1 -- 172.21.15.97:0/2972257516 learned_addr learned my addr 172.21.15.97:0/2972257516 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f5812ffd700 1 --2- 172.21.15.97:0/2972257516 >> v2:172.21.15.183:3300/0 conn(0x7f581415b370 0x7f581415b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f5813fff700 1 -- 172.21.15.97:0/2972257516 >> v2:172.21.15.183:3300/0 conn(0x7f581415b370 msgr2=0x7f581415b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f5813fff700 1 --2- 172.21.15.97:0/2972257516 >> v2:172.21.15.183:3300/0 conn(0x7f581415b370 0x7f581415b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f5813fff700 1 -- 172.21.15.97:0/2972257516 >> v1:172.21.15.183:6789/0 conn(0x7f58141587f0 legacy=0x7f581406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f5813fff700 1 -- 172.21.15.97:0/2972257516 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581415c110 con 0x7f58140b3c60 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f5812ffd700 1 --2- 172.21.15.97:0/2972257516 >> v2:172.21.15.183:3300/0 conn(0x7f581415b370 0x7f581415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f5813fff700 1 --2- 172.21.15.97:0/2972257516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140b3c60 0x7f58140b40f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f580c00d150 tx=0x7f580c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2cdcd70ffca855 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f58127fc700 1 -- 172.21.15.97:0/2972257516 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c010070 con 0x7f58140b3c60 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f58127fc700 1 -- 172.21.15.97:0/2972257516 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00dd40 con 0x7f58140b3c60 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.685+0000 7f58127fc700 1 -- 172.21.15.97:0/2972257516 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00b310 con 0x7f58140b3c60 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.686+0000 7f581a13e700 1 -- 172.21.15.97:0/2972257516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140b3c60 msgr2=0x7f58140b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.686+0000 7f581a13e700 1 --2- 172.21.15.97:0/2972257516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140b3c60 0x7f58140b40f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f580c00d150 tx=0x7f580c00d460 comp rx=0 tx=0).stop 2023-11-24T06:11:58.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.686+0000 7f581a13e700 1 -- 172.21.15.97:0/2972257516 shutdown_connections 2023-11-24T06:11:58.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.686+0000 7f581a13e700 1 --2- 172.21.15.97:0/2972257516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58140b3c60 0x7f58140b40f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.686+0000 7f581a13e700 1 --2- 172.21.15.97:0/2972257516 >> v2:172.21.15.183:3300/0 conn(0x7f581415b370 0x7f581415b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.686+0000 7f581a13e700 1 -- 172.21.15.97:0/2972257516 >> 172.21.15.97:0/2972257516 conn(0x7f58140d0bf0 msgr2=0x7f58140d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:58.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.686+0000 7f581a13e700 1 -- 172.21.15.97:0/2972257516 shutdown_connections 2023-11-24T06:11:58.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.686+0000 7f581a13e700 1 -- 172.21.15.97:0/2972257516 wait complete. 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.687+0000 7f581a13e700 1 Processor -- start 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.687+0000 7f581a13e700 1 -- start start 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f581a13e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58140b3c60 0x7f58141543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f581a13e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58141587f0 0x7f5814152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f581a13e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f581415b370 0x7f58141527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f581a13e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58141f3970 con 0x7f581415b370 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f581a13e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58141f0410 con 0x7f58141587f0 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f581a13e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58141f2bb0 con 0x7f58140b3c60 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f581415b370 0x7f58141527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f581415b370 0x7f58141527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50804/0 (socket says 172.21.15.97:50804) 2023-11-24T06:11:58.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 -- 172.21.15.97:0/1031378280 learned_addr learned my addr 172.21.15.97:0/1031378280 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5812ffd700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58141587f0 0x7f5814152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f58137fe700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58140b3c60 0x7f58141543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 -- 172.21.15.97:0/1031378280 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58140b3c60 msgr2=0x7f58141543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58140b3c60 0x7f58141543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 -- 172.21.15.97:0/1031378280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58141587f0 msgr2=0x7f5814152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58141587f0 0x7f5814152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 -- 172.21.15.97:0/1031378280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c00bd40 con 0x7f581415b370 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f5813fff700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f581415b370 0x7f58141527a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f58140d1730 tx=0x7f580c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f57fbfff700 1 -- 172.21.15.97:0/1031378280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00ab20 con 0x7f581415b370 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f57fbfff700 1 -- 172.21.15.97:0/1031378280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00ac80 con 0x7f581415b370 2023-11-24T06:11:58.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58141f2e90 con 0x7f581415b370 2023-11-24T06:11:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.688+0000 7f57fbfff700 1 -- 172.21.15.97:0/1031378280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c004030 con 0x7f581415b370 2023-11-24T06:11:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.689+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58141f3450 con 0x7f581415b370 2023-11-24T06:11:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.689+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58141f35e0 con 0x7f581415b370 2023-11-24T06:11:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.690+0000 7f57fbfff700 1 -- 172.21.15.97:0/1031378280 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f580c004190 con 0x7f581415b370 2023-11-24T06:11:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.690+0000 7f57fbfff700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f580006b470 0x7f580006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.690+0000 7f57fbfff700 1 -- 172.21.15.97:0/1031378280 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f580c08dab0 con 0x7f581415b370 2023-11-24T06:11:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.690+0000 7f58137fe700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f580006b470 0x7f580006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:58.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.691+0000 7f58137fe700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f580006b470 0x7f580006d930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f581406c700 tx=0x7f5808009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:58.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.695+0000 7f57fbfff700 1 -- 172.21.15.97:0/1031378280 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f580c058490 con 0x7f581415b370 2023-11-24T06:11:58.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.892+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5814068fb0 con 0x7f581415b370 2023-11-24T06:11:58.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.896+0000 7f57fbfff700 1 -- 172.21.15.97:0/1031378280 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f580c05bab0 con 0x7f581415b370 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f580006b470 msgr2=0x7f580006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f580006b470 0x7f580006d930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f581406c700 tx=0x7f5808009040 comp rx=0 tx=0).stop 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f581415b370 msgr2=0x7f58141527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f581415b370 0x7f58141527a0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f58140d1730 tx=0x7f580c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 shutdown_connections 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f581415b370 0x7f58141527a0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f580006b470 0x7f580006d930 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f58140b3c60 0x7f58141543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 --2- 172.21.15.97:0/1031378280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58141587f0 0x7f5814152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:58.901 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 >> 172.21.15.97:0/1031378280 conn(0x7f58140d0bf0 msgr2=0x7f581406f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:58.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 shutdown_connections 2023-11-24T06:11:58.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:58.898+0000 7f581a13e700 1 -- 172.21.15.97:0/1031378280 wait complete. 2023-11-24T06:11:58.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T06:11:59.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.030+0000 7f27ec5e9700 1 Processor -- start 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.030+0000 7f27ec5e9700 1 -- start start 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27ec5e9700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f27e415e080 0x7f27e40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27ec5e9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f27e406c710 0x7f27e40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27ec5e9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e40ae430 con 0x7f27e40af4d0 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27ec5e9700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e40af0b0 con 0x7f27e406c710 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27ec5e9700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e40b05a0 con 0x7f27e415e080 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27eab86700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f27e406c710 0x7f27e40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27e9b84700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f27e40af4d0 0x7f27e406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43262/0 (socket says 172.21.15.97:43262) 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27e9b84700 1 -- 172.21.15.97:0/3221685388 learned_addr learned my addr 172.21.15.97:0/3221685388 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27eab86700 1 -- 172.21.15.97:0/3221685388 >> v2:172.21.15.183:3300/0 conn(0x7f27e415e080 msgr2=0x7f27e40b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:59.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27eab86700 1 --2- 172.21.15.97:0/3221685388 >> v2:172.21.15.183:3300/0 conn(0x7f27e415e080 0x7f27e40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27eab86700 1 -- 172.21.15.97:0/3221685388 >> v1:172.21.15.97:6789/0 conn(0x7f27e40af4d0 legacy=0x7f27e406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27eab86700 1 -- 172.21.15.97:0/3221685388 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40ae710 con 0x7f27e406c710 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.031+0000 7f27e9383700 1 -- 172.21.15.97:0/3221685388 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33033111 0 0) 0x7f27e40ae430 con 0x7f27e40af4d0 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27eab86700 1 --2- 172.21.15.97:0/3221685388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e40afb90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f27e4072a90 tx=0x7f27dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c51389e9f75d4ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27e9383700 1 -- 172.21.15.97:0/3221685388 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00e070 con 0x7f27e406c710 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27e9383700 1 -- 172.21.15.97:0/3221685388 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27dc00bb90 con 0x7f27e406c710 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27e9383700 1 -- 172.21.15.97:0/3221685388 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc012430 con 0x7f27e406c710 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27ec5e9700 1 -- 172.21.15.97:0/3221685388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 msgr2=0x7f27e40afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/3221685388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e40afb90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f27e4072a90 tx=0x7f27dc00b330 comp rx=0 tx=0).stop 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27ec5e9700 1 -- 172.21.15.97:0/3221685388 shutdown_connections 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/3221685388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e40afb90 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/3221685388 >> v2:172.21.15.183:3300/0 conn(0x7f27e415e080 0x7f27e40b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27ec5e9700 1 -- 172.21.15.97:0/3221685388 >> 172.21.15.97:0/3221685388 conn(0x7f27e4071b90 msgr2=0x7f27e4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:59.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27ec5e9700 1 -- 172.21.15.97:0/3221685388 shutdown_connections 2023-11-24T06:11:59.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.032+0000 7f27ec5e9700 1 -- 172.21.15.97:0/3221685388 wait complete. 2023-11-24T06:11:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ec5e9700 1 Processor -- start 2023-11-24T06:11:59.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ec5e9700 1 -- start start 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ec5e9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e414f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ec5e9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e40af4d0 0x7f27e41f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ec5e9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27e415e080 0x7f27e414a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ec5e9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41f7da0 con 0x7f27e406c710 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ec5e9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e414ff10 con 0x7f27e40af4d0 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ec5e9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41501f0 con 0x7f27e415e080 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ea385700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e414f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ea385700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e414f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50814/0 (socket says 172.21.15.97:50814) 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.034+0000 7f27ea385700 1 -- 172.21.15.97:0/594188972 learned_addr learned my addr 172.21.15.97:0/594188972 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27e9b84700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e40af4d0 0x7f27e41f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27eab86700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27e415e080 0x7f27e414a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27ea385700 1 -- 172.21.15.97:0/594188972 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27e415e080 msgr2=0x7f27e414a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27ea385700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27e415e080 0x7f27e414a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27ea385700 1 -- 172.21.15.97:0/594188972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e40af4d0 msgr2=0x7f27e41f7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27ea385700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e40af4d0 0x7f27e41f7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27ea385700 1 -- 172.21.15.97:0/594188972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc009a70 con 0x7f27e406c710 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27ea385700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e414f870 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b4a0 tx=0x7f27d400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27daffd700 1 -- 172.21.15.97:0/594188972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4011830 con 0x7f27e406c710 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27daffd700 1 -- 172.21.15.97:0/594188972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d4011990 con 0x7f27e406c710 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e41504d0 con 0x7f27e406c710 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27daffd700 1 -- 172.21.15.97:0/594188972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4011d40 con 0x7f27e406c710 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.035+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e4150a90 con 0x7f27e406c710 2023-11-24T06:11:59.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.036+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e40753f0 con 0x7f27e406c710 2023-11-24T06:11:59.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.036+0000 7f27daffd700 1 -- 172.21.15.97:0/594188972 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f27d401a400 con 0x7f27e406c710 2023-11-24T06:11:59.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.037+0000 7f27daffd700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f27c806b470 0x7f27c806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.037+0000 7f27daffd700 1 -- 172.21.15.97:0/594188972 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f27d4089d60 con 0x7f27e406c710 2023-11-24T06:11:59.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.037+0000 7f27e9b84700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f27c806b470 0x7f27c806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.037+0000 7f27e9b84700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f27c806b470 0x7f27c806d930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f27e40b0530 tx=0x7f27e0007c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:59.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.041+0000 7f27daffd700 1 -- 172.21.15.97:0/594188972 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27d4054890 con 0x7f27e406c710 2023-11-24T06:11:59.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.309+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f27e4150c80 con 0x7f27e406c710 2023-11-24T06:11:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.543+0000 7f27daffd700 1 -- 172.21.15.97:0/594188972 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v196) v1 ==== 166+0+0 (secure 0 0 0) 0x7f27e4150c80 con 0x7f27e406c710 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f27c806b470 msgr2=0x7f27c806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f27c806b470 0x7f27c806d930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f27e40b0530 tx=0x7f27e0007c00 comp rx=0 tx=0).stop 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 msgr2=0x7f27e414f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e414f870 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b4a0 tx=0x7f27d400b860 comp rx=0 tx=0).stop 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 shutdown_connections 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f27c806b470 0x7f27c806d930 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f27e415e080 0x7f27e414a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27e406c710 0x7f27e414f870 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 --2- 172.21.15.97:0/594188972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27e40af4d0 0x7f27e41f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.550 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 >> 172.21.15.97:0/594188972 conn(0x7f27e4071b90 msgr2=0x7f27e40b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:59.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 shutdown_connections 2023-11-24T06:11:59.551 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.547+0000 7f27ec5e9700 1 -- 172.21.15.97:0/594188972 wait complete. 2023-11-24T06:11:59.562 INFO:tasks.cephfs_test_runner:tests the scenario where 'fs subvolume snapshot info' command ... ok 2023-11-24T06:11:59.562 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:11:59.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:11:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.680+0000 7f9df7411700 1 Processor -- start 2023-11-24T06:11:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.680+0000 7f9df7411700 1 -- start start 2023-11-24T06:11:59.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df7411700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9df00b59f0 0x7f9df006db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df7411700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df00af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df7411700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df00b0290 con 0x7f9df006e0b0 2023-11-24T06:11:59.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df7411700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df00b4840 con 0x7f9df00b59f0 2023-11-24T06:11:59.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df7411700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00b4b00 con 0x7f9df00b4350 2023-11-24T06:11:59.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df51ad700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9df00b59f0 0x7f9df006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df49ac700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9df006e0b0 0x7f9df00b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43276/0 (socket says 172.21.15.97:43276) 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df49ac700 1 -- 172.21.15.97:0/2055947330 learned_addr learned my addr 172.21.15.97:0/2055947330 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df59ae700 1 --2- 172.21.15.97:0/2055947330 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df00af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df51ad700 1 -- 172.21.15.97:0/2055947330 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 msgr2=0x7f9df00af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df51ad700 1 --2- 172.21.15.97:0/2055947330 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df00af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df51ad700 1 -- 172.21.15.97:0/2055947330 >> v1:172.21.15.97:6789/0 conn(0x7f9df006e0b0 legacy=0x7f9df00b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df51ad700 1 -- 172.21.15.97:0/2055947330 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df00ae9b0 con 0x7f9df00b59f0 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9de7fff700 1 -- 172.21.15.97:0/2055947330 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027338363 0 0) 0x7f9df00b0290 con 0x7f9df006e0b0 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9df51ad700 1 --2- 172.21.15.97:0/2055947330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df00b59f0 0x7f9df006db70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009870 tx=0x7f9dec009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e2b4022ba7c4e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.681+0000 7f9de7fff700 1 -- 172.21.15.97:0/2055947330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec011830 con 0x7f9df00b59f0 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9de7fff700 1 -- 172.21.15.97:0/2055947330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec011990 con 0x7f9df00b59f0 2023-11-24T06:11:59.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9de7fff700 1 -- 172.21.15.97:0/2055947330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec011d20 con 0x7f9df00b59f0 2023-11-24T06:11:59.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9df7411700 1 -- 172.21.15.97:0/2055947330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df00b59f0 msgr2=0x7f9df006db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9df7411700 1 --2- 172.21.15.97:0/2055947330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df00b59f0 0x7f9df006db70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009870 tx=0x7f9dec009b80 comp rx=0 tx=0).stop 2023-11-24T06:11:59.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9df7411700 1 -- 172.21.15.97:0/2055947330 shutdown_connections 2023-11-24T06:11:59.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9df7411700 1 --2- 172.21.15.97:0/2055947330 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df00af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9df7411700 1 --2- 172.21.15.97:0/2055947330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df00b59f0 0x7f9df006db70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9df7411700 1 -- 172.21.15.97:0/2055947330 >> 172.21.15.97:0/2055947330 conn(0x7f9df0072990 msgr2=0x7f9df006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:59.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9df7411700 1 -- 172.21.15.97:0/2055947330 shutdown_connections 2023-11-24T06:11:59.687 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.682+0000 7f9df7411700 1 -- 172.21.15.97:0/2055947330 wait complete. 2023-11-24T06:11:59.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.683+0000 7f9df7411700 1 Processor -- start 2023-11-24T06:11:59.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.683+0000 7f9df7411700 1 -- start start 2023-11-24T06:11:59.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df7411700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e0b0 0x7f9df0151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df7411700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df015c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df7411700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00cf960 0x7f9df00ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df7411700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0151730 con 0x7f9df006e0b0 2023-11-24T06:11:59.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df7411700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0150ab0 con 0x7f9df00cf960 2023-11-24T06:11:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df7411700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0150d90 con 0x7f9df00b4350 2023-11-24T06:11:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e0b0 0x7f9df0151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e0b0 0x7f9df0151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50828/0 (socket says 172.21.15.97:50828) 2023-11-24T06:11:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 -- 172.21.15.97:0/406857229 learned_addr learned my addr 172.21.15.97:0/406857229 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:11:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df49ac700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df015c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df59ae700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00cf960 0x7f9df00ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 -- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 msgr2=0x7f9df015c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df015c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 -- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00cf960 msgr2=0x7f9df00ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00cf960 0x7f9df00ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 -- 172.21.15.97:0/406857229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0094a0 con 0x7f9df006e0b0 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df49ac700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df015c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df59ae700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00cf960 0x7f9df00ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.684+0000 7f9df51ad700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e0b0 0x7f9df0151b50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9dec000c00 tx=0x7f9dec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.685+0000 7f9de5ffb700 1 -- 172.21.15.97:0/406857229 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0045d0 con 0x7f9df006e0b0 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.685+0000 7f9de5ffb700 1 -- 172.21.15.97:0/406857229 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec01f070 con 0x7f9df006e0b0 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.685+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df00aed40 con 0x7f9df006e0b0 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.685+0000 7f9de5ffb700 1 -- 172.21.15.97:0/406857229 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0038a0 con 0x7f9df006e0b0 2023-11-24T06:11:59.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.685+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df0151350 con 0x7f9df006e0b0 2023-11-24T06:11:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.686+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df01519a0 con 0x7f9df006e0b0 2023-11-24T06:11:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.686+0000 7f9de5ffb700 1 -- 172.21.15.97:0/406857229 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f9dec003a00 con 0x7f9df006e0b0 2023-11-24T06:11:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.687+0000 7f9de5ffb700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd406b470 0x7f9dd406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:11:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.687+0000 7f9de5ffb700 1 -- 172.21.15.97:0/406857229 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f9dec021070 con 0x7f9df006e0b0 2023-11-24T06:11:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.687+0000 7f9df49ac700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd406b470 0x7f9dd406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:11:59.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.687+0000 7f9df49ac700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd406b470 0x7f9dd406d930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f60 tx=0x7f9de0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:11:59.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.691+0000 7f9de5ffb700 1 -- 172.21.15.97:0/406857229 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9dec0560a0 con 0x7f9df006e0b0 2023-11-24T06:11:59.888 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.884+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9df0067f60 con 0x7f9df006e0b0 2023-11-24T06:11:59.889 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.885+0000 7f9de5ffb700 1 -- 172.21.15.97:0/406857229 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+13044 (secure 0 0 0) 0x7f9dec003cb0 con 0x7f9df006e0b0 2023-11-24T06:11:59.889 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:11:59.889 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":76,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:11:58.538412+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.183:6835/3452869588":"2023-11-25T06:11:56.118677+0000","172.21.15.183:6833/3452869588":"2023-11-25T06:11:56.118677+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.887+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd406b470 msgr2=0x7f9dd406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd406b470 0x7f9dd406d930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f60 tx=0x7f9de0009040 comp rx=0 tx=0).stop 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e0b0 msgr2=0x7f9df0151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e0b0 0x7f9df0151b50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9dec000c00 tx=0x7f9dec004750 comp rx=0 tx=0).stop 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 shutdown_connections 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9df00cf960 0x7f9df00ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9df006e0b0 0x7f9df0151b50 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f9dd406b470 0x7f9dd406d930 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 --2- 172.21.15.97:0/406857229 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9df00b4350 0x7f9df015c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:11:59.891 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 >> 172.21.15.97:0/406857229 conn(0x7f9df0072990 msgr2=0x7f9df00b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:11:59.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 shutdown_connections 2023-11-24T06:11:59.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:11:59.888+0000 7f9df7411700 1 -- 172.21.15.97:0/406857229 wait complete. 2023-11-24T06:11:59.902 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:11:59.903 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:00.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.022+0000 7f5b69524700 1 Processor -- start 2023-11-24T06:12:00.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.022+0000 7f5b69524700 1 -- start start 2023-11-24T06:12:00.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b69524700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b6415e140 0x7f5b640d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b69524700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b640b80e0 con 0x7f5b640b98c0 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b69524700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b640b67d0 con 0x7f5b640d39d0 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b69524700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6415e520 con 0x7f5b6415e140 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b637fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5b640b98c0 0x7f5b640b60c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43292/0 (socket says 172.21.15.97:43292) 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b637fe700 1 -- 172.21.15.97:0/3413227939 learned_addr learned my addr 172.21.15.97:0/3413227939 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25315527 0 0) 0x7f5b5c001490 con 0x7f5b640b98c0 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b62ffd700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b6415e140 0x7f5b640d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b4c0035b0 con 0x7f5b640b98c0 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1996608799 0 0) 0x7f5b4c0035b0 con 0x7f5b640b98c0 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b6415e140 msgr2=0x7f5b640d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b6415e140 0x7f5b640d3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 >> v1:172.21.15.183:6789/0 conn(0x7f5b640d39d0 legacy=0x7f5b640b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b640d2930 con 0x7f5b640b98c0 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f5b5c001b80 con 0x7f5b640b98c0 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.023+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 >> v1:172.21.15.97:6789/0 conn(0x7f5b640b98c0 legacy=0x7f5b640b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b4c003a30 0x7f5b4c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b6415e140 0x7f5b4c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c004d90 0x7f5b4c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6415e520 con 0x7f5b4c004d90 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c00bac0 con 0x7f5b4c003a30 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c00bc80 con 0x7f5b6415e140 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b637fe700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c004d90 0x7f5b4c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b62ffd700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b4c003a30 0x7f5b4c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b627fc700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b6415e140 0x7f5b4c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b637fe700 1 -- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b6415e140 msgr2=0x7f5b4c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b637fe700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b6415e140 0x7f5b4c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b637fe700 1 -- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b4c003a30 msgr2=0x7f5b4c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b637fe700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b4c003a30 0x7f5b4c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b637fe700 1 -- 172.21.15.97:0/3413227939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0035b0 con 0x7f5b4c004d90 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b637fe700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c004d90 0x7f5b4c005210 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5b640b9d70 tx=0x7f5b4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61828af9fcdbc688 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00a490 con 0x7f5b4c004d90 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c012600 con 0x7f5b4c004d90 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.024+0000 7f5b61ffb700 1 -- 172.21.15.97:0/3413227939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c012990 con 0x7f5b4c004d90 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.025+0000 7f5b69524700 1 -- 172.21.15.97:0/3413227939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c004d90 msgr2=0x7f5b4c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.025+0000 7f5b69524700 1 --2- 172.21.15.97:0/3413227939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b4c004d90 0x7f5b4c005210 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5b640b9d70 tx=0x7f5b4c005880 comp rx=0 tx=0).stop 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.025+0000 7f5b637fe700 1 -- 172.21.15.97:0/3413227939 reap_dead start 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.025+0000 7f5b69524700 1 -- 172.21.15.97:0/3413227939 shutdown_connections 2023-11-24T06:12:00.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.025+0000 7f5b69524700 1 -- 172.21.15.97:0/3413227939 >> 172.21.15.97:0/3413227939 conn(0x7f5b6406b410 msgr2=0x7f5b6406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.025+0000 7f5b69524700 1 -- 172.21.15.97:0/3413227939 shutdown_connections 2023-11-24T06:12:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.025+0000 7f5b69524700 1 -- 172.21.15.97:0/3413227939 wait complete. 2023-11-24T06:12:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.026+0000 7f5b69524700 1 Processor -- start 2023-11-24T06:12:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.026+0000 7f5b69524700 1 -- start start 2023-11-24T06:12:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b69524700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b641f3da0 0x7f5b64149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b69524700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b641570b0 0x7f5b640c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b69524700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b640c7cc0 0x7f5b641586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b69524700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640c9c00 con 0x7f5b640c7cc0 2023-11-24T06:12:00.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b69524700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b640ca0a0 con 0x7f5b641570b0 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b69524700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b641499a0 con 0x7f5b641f3da0 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b640c7cc0 0x7f5b641586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b640c7cc0 0x7f5b641586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50852/0 (socket says 172.21.15.97:50852) 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 -- 172.21.15.97:0/2757255120 learned_addr learned my addr 172.21.15.97:0/2757255120 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b627fc700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b641570b0 0x7f5b640c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b62ffd700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b641f3da0 0x7f5b64149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 -- 172.21.15.97:0/2757255120 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b641f3da0 msgr2=0x7f5b64149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b641f3da0 0x7f5b64149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 -- 172.21.15.97:0/2757255120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b641570b0 msgr2=0x7f5b640c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b641570b0 0x7f5b640c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 -- 172.21.15.97:0/2757255120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64157600 con 0x7f5b640c7cc0 2023-11-24T06:12:00.035 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.027+0000 7f5b637fe700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b640c7cc0 0x7f5b641586a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c012000 tx=0x7f5b5c001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.028+0000 7f5b4b7fe700 1 -- 172.21.15.97:0/2757255120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c00a490 con 0x7f5b640c7cc0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.028+0000 7f5b4b7fe700 1 -- 172.21.15.97:0/2757255120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5c001490 con 0x7f5b640c7cc0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.028+0000 7f5b4b7fe700 1 -- 172.21.15.97:0/2757255120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c001b50 con 0x7f5b640c7cc0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.028+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b641578e0 con 0x7f5b640c7cc0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.028+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b64157ea0 con 0x7f5b640c7cc0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.029+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b64068fb0 con 0x7f5b640c7cc0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.029+0000 7f5b4b7fe700 1 -- 172.21.15.97:0/2757255120 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5b5c02b460 con 0x7f5b640c7cc0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.029+0000 7f5b4b7fe700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5b5006b470 0x7f5b5006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.029+0000 7f5b4b7fe700 1 -- 172.21.15.97:0/2757255120 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f5b5c08b980 con 0x7f5b640c7cc0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.030+0000 7f5b62ffd700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5b5006b470 0x7f5b5006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.030+0000 7f5b62ffd700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5b5006b470 0x7f5b5006d930 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5b640ce610 tx=0x7f5b54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.034+0000 7f5b4b7fe700 1 -- 172.21.15.97:0/2757255120 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b5c056560 con 0x7f5b640c7cc0 2023-11-24T06:12:00.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.225+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b640b80e0 con 0x7f5b640c7cc0 2023-11-24T06:12:00.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.226+0000 7f5b4b7fe700 1 -- 172.21.15.97:0/2757255120 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b640b80e0 con 0x7f5b640c7cc0 2023-11-24T06:12:00.229 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:12:00.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5b5006b470 msgr2=0x7f5b5006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5b5006b470 0x7f5b5006d930 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5b640ce610 tx=0x7f5b54008040 comp rx=0 tx=0).stop 2023-11-24T06:12:00.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b640c7cc0 msgr2=0x7f5b641586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b640c7cc0 0x7f5b641586a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c012000 tx=0x7f5b5c001f10 comp rx=0 tx=0).stop 2023-11-24T06:12:00.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 shutdown_connections 2023-11-24T06:12:00.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b640c7cc0 0x7f5b641586a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5b5006b470 0x7f5b5006d930 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.231 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5b641f3da0 0x7f5b64149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 --2- 172.21.15.97:0/2757255120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b641570b0 0x7f5b640c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 >> 172.21.15.97:0/2757255120 conn(0x7f5b6406b410 msgr2=0x7f5b640d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:00.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 shutdown_connections 2023-11-24T06:12:00.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.228+0000 7f5b69524700 1 -- 172.21.15.97:0/2757255120 wait complete. 2023-11-24T06:12:00.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T06:12:00.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.360+0000 7fdb38df0700 1 Processor -- start 2023-11-24T06:12:00.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.360+0000 7fdb38df0700 1 -- start start 2023-11-24T06:12:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb38df0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb340ceb70 0x7fdb34166ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb38df0700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3415fb30 con 0x7fdb340746a0 2023-11-24T06:12:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb38df0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3415fd80 con 0x7fdb34145510 2023-11-24T06:12:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb38df0700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb34160060 con 0x7fdb340ceb70 2023-11-24T06:12:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb31d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fdb340746a0 0x7fdb340ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43310/0 (socket says 172.21.15.97:43310) 2023-11-24T06:12:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb32d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdb340ceb70 0x7fdb34166ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb31d9b700 1 -- 172.21.15.97:0/3694350742 learned_addr learned my addr 172.21.15.97:0/3694350742 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb32d9d700 1 -- 172.21.15.97:0/3694350742 >> v1:172.21.15.183:6789/0 conn(0x7fdb34145510 legacy=0x7fdb34073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb32d9d700 1 -- 172.21.15.97:0/3694350742 >> v1:172.21.15.97:6789/0 conn(0x7fdb340746a0 legacy=0x7fdb340ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb32d9d700 1 -- 172.21.15.97:0/3694350742 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb34160340 con 0x7fdb340ceb70 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.361+0000 7fdb3159a700 1 -- 172.21.15.97:0/3694350742 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174259378 0 0) 0x7fdb3415fb30 con 0x7fdb340746a0 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb32d9d700 1 --2- 172.21.15.97:0/3694350742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340ceb70 0x7fdb34166ec0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009840 tx=0x7fdb24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0af3ff5e0667399 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb3159a700 1 -- 172.21.15.97:0/3694350742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24011830 con 0x7fdb340ceb70 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb3159a700 1 -- 172.21.15.97:0/3694350742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb24011990 con 0x7fdb340ceb70 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb3159a700 1 -- 172.21.15.97:0/3694350742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24011d20 con 0x7fdb340ceb70 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb38df0700 1 -- 172.21.15.97:0/3694350742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340ceb70 msgr2=0x7fdb34166ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb38df0700 1 --2- 172.21.15.97:0/3694350742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340ceb70 0x7fdb34166ec0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009840 tx=0x7fdb24009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb38df0700 1 -- 172.21.15.97:0/3694350742 shutdown_connections 2023-11-24T06:12:00.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb38df0700 1 --2- 172.21.15.97:0/3694350742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340ceb70 0x7fdb34166ec0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb38df0700 1 -- 172.21.15.97:0/3694350742 >> 172.21.15.97:0/3694350742 conn(0x7fdb34072990 msgr2=0x7fdb3406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:00.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb38df0700 1 -- 172.21.15.97:0/3694350742 shutdown_connections 2023-11-24T06:12:00.367 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.362+0000 7fdb38df0700 1 -- 172.21.15.97:0/3694350742 wait complete. 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.364+0000 7fdb38df0700 1 Processor -- start 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.364+0000 7fdb38df0700 1 -- start start 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb38df0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340746a0 0x7fdb341557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb38df0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb340ceb70 0x7fdb341f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb38df0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb34145510 0x7fdb341f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb38df0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3412ed60 con 0x7fdb340746a0 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb38df0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3412e0e0 con 0x7fdb34145510 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb38df0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3412e3c0 con 0x7fdb340ceb70 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb3259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340746a0 0x7fdb341557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb3259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340746a0 0x7fdb341557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50862/0 (socket says 172.21.15.97:50862) 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb3259c700 1 -- 172.21.15.97:0/952525190 learned_addr learned my addr 172.21.15.97:0/952525190 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb31d9b700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb340ceb70 0x7fdb341f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb32d9d700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb34145510 0x7fdb341f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb3259c700 1 -- 172.21.15.97:0/952525190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb340ceb70 msgr2=0x7fdb341f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb3259c700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb340ceb70 0x7fdb341f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb3259c700 1 -- 172.21.15.97:0/952525190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb34145510 msgr2=0x7fdb341f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb3259c700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb34145510 0x7fdb341f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.365+0000 7fdb3259c700 1 -- 172.21.15.97:0/952525190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240094a0 con 0x7fdb340746a0 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.366+0000 7fdb3259c700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340746a0 0x7fdb341557d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c001ed0 tx=0x7fdb2c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.366+0000 7fdb22ffd700 1 -- 172.21.15.97:0/952525190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c00ade0 con 0x7fdb340746a0 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.366+0000 7fdb22ffd700 1 -- 172.21.15.97:0/952525190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0020d0 con 0x7fdb340746a0 2023-11-24T06:12:00.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.366+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3412e6a0 con 0x7fdb340746a0 2023-11-24T06:12:00.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.366+0000 7fdb22ffd700 1 -- 172.21.15.97:0/952525190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c003c20 con 0x7fdb340746a0 2023-11-24T06:12:00.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.366+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb34155d10 con 0x7fdb340746a0 2023-11-24T06:12:00.371 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.367+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb34067f60 con 0x7fdb340746a0 2023-11-24T06:12:00.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.367+0000 7fdb22ffd700 1 -- 172.21.15.97:0/952525190 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fdb2c0050d0 con 0x7fdb340746a0 2023-11-24T06:12:00.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.367+0000 7fdb22ffd700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdb1406b470 0x7fdb1406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:00.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.368+0000 7fdb22ffd700 1 -- 172.21.15.97:0/952525190 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fdb2c089eb0 con 0x7fdb340746a0 2023-11-24T06:12:00.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.368+0000 7fdb31d9b700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdb1406b470 0x7fdb1406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:00.372 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.368+0000 7fdb31d9b700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdb1406b470 0x7fdb1406d930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007fd0 tx=0x7fdb28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:00.375 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.372+0000 7fdb22ffd700 1 -- 172.21.15.97:0/952525190 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb2c054a20 con 0x7fdb340746a0 2023-11-24T06:12:00.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:00.645+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7fdb3412eea0 con 0x7fdb340746a0 2023-11-24T06:12:01.554 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.552+0000 7fdb22ffd700 1 -- 172.21.15.97:0/952525190 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v198) v1 ==== 167+0+0 (secure 0 0 0) 0x7fdb2c058040 con 0x7fdb340746a0 2023-11-24T06:12:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdb1406b470 msgr2=0x7fdb1406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdb1406b470 0x7fdb1406d930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb28007fd0 tx=0x7fdb28009040 comp rx=0 tx=0).stop 2023-11-24T06:12:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340746a0 msgr2=0x7fdb341557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340746a0 0x7fdb341557d0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c001ed0 tx=0x7fdb2c00a340 comp rx=0 tx=0).stop 2023-11-24T06:12:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 shutdown_connections 2023-11-24T06:12:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb34145510 0x7fdb341f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb340746a0 0x7fdb341557d0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.558 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdb1406b470 0x7fdb1406d930 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 --2- 172.21.15.97:0/952525190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdb340ceb70 0x7fdb341f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 >> 172.21.15.97:0/952525190 conn(0x7fdb34072990 msgr2=0x7fdb3415c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:01.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 shutdown_connections 2023-11-24T06:12:01.559 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.555+0000 7fdb38df0700 1 -- 172.21.15.97:0/952525190 wait complete. 2023-11-24T06:12:01.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:12:01.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.698+0000 7f372cf2c700 1 Processor -- start 2023-11-24T06:12:01.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.698+0000 7f372cf2c700 1 -- start start 2023-11-24T06:12:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372cf2c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f37280746a0 0x7f37280b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372cf2c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37280afc90 0x7f372806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372cf2c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37280b3c60 con 0x7f372815e160 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372cf2c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37280b4500 con 0x7f37280746a0 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372cf2c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37280b58c0 con 0x7f37280afc90 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372659c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f372815e160 0x7f3728073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43322/0 (socket says 172.21.15.97:43322) 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372659c700 1 -- 172.21.15.97:0/3055436190 learned_addr learned my addr 172.21.15.97:0/3055436190 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706337423 0 0) 0x7f37280b3c60 con 0x7f372815e160 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f3726d9d700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37280afc90 0x7f372806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37100035b0 con 0x7f372815e160 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f3725d9b700 1 --2- 172.21.15.97:0/3055436190 >> v2:172.21.15.183:3300/0 conn(0x7f37280746a0 0x7f37280b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2899911571 0 0) 0x7f37100035b0 con 0x7f372815e160 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37280afc90 msgr2=0x7f372806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372559a700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37280afc90 0x7f372806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 >> v2:172.21.15.183:3300/0 conn(0x7f37280746a0 msgr2=0x7f37280b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372559a700 1 --2- 172.21.15.97:0/3055436190 >> v2:172.21.15.183:3300/0 conn(0x7f37280746a0 0x7f37280b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.699+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37280b0120 con 0x7f372815e160 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f371c001670 con 0x7f372815e160 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 >> v1:172.21.15.97:6789/0 conn(0x7f372815e160 legacy=0x7f3728073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3710003a00 0x7f3710003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3710004850 0x7f3710004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37280afc90 0x7f3710005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37280b4500 con 0x7f3710004850 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37280b58c0 con 0x7f3710003a00 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371000ba10 con 0x7f37280afc90 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f3725d9b700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3710004850 0x7f3710004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372659c700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3710003a00 0x7f3710003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f3726d9d700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37280afc90 0x7f3710005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f3725d9b700 1 -- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37280afc90 msgr2=0x7f3710005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f3725d9b700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37280afc90 0x7f3710005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f3725d9b700 1 -- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3710003a00 msgr2=0x7f3710003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f3725d9b700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3710003a00 0x7f3710003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f3725d9b700 1 -- 172.21.15.97:0/3055436190 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37100035b0 con 0x7f3710004850 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372659c700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3710003a00 0x7f3710003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f3725d9b700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3710004850 0x7f3710004cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3718009b60 tx=0x7f3710005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad7df1519919ba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371800e6c0 con 0x7f3710004850 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371800e820 con 0x7f3710004850 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.700+0000 7f372559a700 1 -- 172.21.15.97:0/3055436190 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f371800eb90 con 0x7f3710004850 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.701+0000 7f372cf2c700 1 -- 172.21.15.97:0/3055436190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3710004850 msgr2=0x7f3710004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.701+0000 7f372cf2c700 1 --2- 172.21.15.97:0/3055436190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3710004850 0x7f3710004cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3718009b60 tx=0x7f3710005880 comp rx=0 tx=0).stop 2023-11-24T06:12:01.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.701+0000 7f3726d9d700 1 -- 172.21.15.97:0/3055436190 reap_dead start 2023-11-24T06:12:01.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.701+0000 7f372cf2c700 1 -- 172.21.15.97:0/3055436190 shutdown_connections 2023-11-24T06:12:01.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.701+0000 7f372cf2c700 1 -- 172.21.15.97:0/3055436190 >> 172.21.15.97:0/3055436190 conn(0x7f37280d0bf0 msgr2=0x7f37280d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:01.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.701+0000 7f372cf2c700 1 -- 172.21.15.97:0/3055436190 shutdown_connections 2023-11-24T06:12:01.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.701+0000 7f372cf2c700 1 -- 172.21.15.97:0/3055436190 wait complete. 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.702+0000 7f372cf2c700 1 Processor -- start 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.703+0000 7f372cf2c700 1 -- start start 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.703+0000 7f372cf2c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3710004850 0x7f37280b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.703+0000 7f372cf2c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281f3710 0x7f3728151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.703+0000 7f372cf2c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280b17c0 0x7f3728151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.703+0000 7f372cf2c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3728155740 con 0x7f37280b17c0 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.703+0000 7f372cf2c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37280b1440 con 0x7f37281f3710 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.703+0000 7f372cf2c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372815efc0 con 0x7f3710004850 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280b17c0 0x7f3728151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280b17c0 0x7f3728151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50886/0 (socket says 172.21.15.97:50886) 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 -- 172.21.15.97:0/1126352584 learned_addr learned my addr 172.21.15.97:0/1126352584 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:01.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3725d9b700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281f3710 0x7f3728151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f372659c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3710004850 0x7f37280b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 -- 172.21.15.97:0/1126352584 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3710004850 msgr2=0x7f37280b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3710004850 0x7f37280b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 -- 172.21.15.97:0/1126352584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281f3710 msgr2=0x7f3728151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281f3710 0x7f3728151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 -- 172.21.15.97:0/1126352584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37280b0ba0 con 0x7f37280b17c0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3726d9d700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280b17c0 0x7f3728151d40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f37280d16e0 tx=0x7f37200114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3716ffd700 1 -- 172.21.15.97:0/1126352584 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372000e070 con 0x7f37280b17c0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3716ffd700 1 -- 172.21.15.97:0/1126352584 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3720003cf0 con 0x7f37280b17c0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37280b0e80 con 0x7f37280b17c0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f3716ffd700 1 -- 172.21.15.97:0/1126352584 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720003ea0 con 0x7f37280b17c0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.704+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372815f260 con 0x7f37280b17c0 2023-11-24T06:12:01.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.705+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372815f5d0 con 0x7f37280b17c0 2023-11-24T06:12:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.706+0000 7f3716ffd700 1 -- 172.21.15.97:0/1126352584 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f3720011a40 con 0x7f37280b17c0 2023-11-24T06:12:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.706+0000 7f3716ffd700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370806b470 0x7f370806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.706+0000 7f372659c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370806b470 0x7f370806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.706+0000 7f3716ffd700 1 -- 172.21.15.97:0/1126352584 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f372008a280 con 0x7f37280b17c0 2023-11-24T06:12:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.707+0000 7f372659c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370806b470 0x7f370806d930 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f371c000fb0 tx=0x7f371c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:01.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.711+0000 7f3716ffd700 1 -- 172.21.15.97:0/1126352584 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3720054c90 con 0x7f37280b17c0 2023-11-24T06:12:01.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.902+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f372815f3f0 con 0x7f37280b17c0 2023-11-24T06:12:01.907 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.905+0000 7f3716ffd700 1 -- 172.21.15.97:0/1126352584 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3720011cf0 con 0x7f37280b17c0 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370806b470 msgr2=0x7f370806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370806b470 0x7f370806d930 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f371c000fb0 tx=0x7f371c009040 comp rx=0 tx=0).stop 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280b17c0 msgr2=0x7f3728151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280b17c0 0x7f3728151d40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f37280d16e0 tx=0x7f37200114b0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 shutdown_connections 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37280b17c0 0x7f3728151d40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f370806b470 0x7f370806d930 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3710004850 0x7f37280b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.910 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 --2- 172.21.15.97:0/1126352584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37281f3710 0x7f3728151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 >> 172.21.15.97:0/1126352584 conn(0x7f37280d0bf0 msgr2=0x7f372806ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 shutdown_connections 2023-11-24T06:12:01.911 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:01.907+0000 7f372cf2c700 1 -- 172.21.15.97:0/1126352584 wait complete. 2023-11-24T06:12:01.922 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:12:01.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:01.966 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:02.017 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:12:02.018 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:12:02.018 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:02.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:12:02.035 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:12:02.035 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:02.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:12:02.054 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:12:02.054 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:02.092 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:12:02.092 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:12:02.092 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:02.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:12:02.113 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:12:02.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.249+0000 7f828b98c700 1 Processor -- start 2023-11-24T06:12:02.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.250+0000 7f828b98c700 1 -- start start 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.250+0000 7f828b98c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f828415e080 0x7f82840b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f828b98c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f828406c710 0x7f82840afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f828b98c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82840ae430 con 0x7f82840af4d0 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f828b98c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82840af0b0 con 0x7f828406c710 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f828b98c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82840b05a0 con 0x7f828415e080 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289f29700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f828406c710 0x7f82840afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289f29700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f828406c710 0x7f82840afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50894/0 (socket says 172.21.15.97:50894) 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289f29700 1 -- 172.21.15.97:0/3646069380 learned_addr learned my addr 172.21.15.97:0/3646069380 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289728700 1 --2- 172.21.15.97:0/3646069380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f828415e080 0x7f82840b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289f29700 1 -- 172.21.15.97:0/3646069380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f828415e080 msgr2=0x7f82840b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289f29700 1 --2- 172.21.15.97:0/3646069380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f828415e080 0x7f82840b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289f29700 1 -- 172.21.15.97:0/3646069380 >> v1:172.21.15.183:6789/0 conn(0x7f82840af4d0 legacy=0x7f828406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289f29700 1 -- 172.21.15.97:0/3646069380 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82840ae710 con 0x7f828406c710 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8289f29700 1 --2- 172.21.15.97:0/3646069380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828406c710 0x7f82840afb90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f827c009e80 tx=0x7f827c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c17cdb82d2c0e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8277fff700 1 -- 172.21.15.97:0/3646069380 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00e070 con 0x7f828406c710 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8277fff700 1 -- 172.21.15.97:0/3646069380 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c00b980 con 0x7f828406c710 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.251+0000 7f8277fff700 1 -- 172.21.15.97:0/3646069380 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00bd30 con 0x7f828406c710 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.252+0000 7f828b98c700 1 -- 172.21.15.97:0/3646069380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828406c710 msgr2=0x7f82840afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.252+0000 7f828b98c700 1 --2- 172.21.15.97:0/3646069380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828406c710 0x7f82840afb90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f827c009e80 tx=0x7f827c00b330 comp rx=0 tx=0).stop 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.252+0000 7f828b98c700 1 -- 172.21.15.97:0/3646069380 shutdown_connections 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.252+0000 7f828b98c700 1 --2- 172.21.15.97:0/3646069380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828406c710 0x7f82840afb90 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.252+0000 7f828b98c700 1 --2- 172.21.15.97:0/3646069380 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f828415e080 0x7f82840b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.252+0000 7f828b98c700 1 -- 172.21.15.97:0/3646069380 >> 172.21.15.97:0/3646069380 conn(0x7f8284071b90 msgr2=0x7f8284071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.252+0000 7f828b98c700 1 -- 172.21.15.97:0/3646069380 shutdown_connections 2023-11-24T06:12:02.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.252+0000 7f828b98c700 1 -- 172.21.15.97:0/3646069380 wait complete. 2023-11-24T06:12:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.253+0000 7f828b98c700 1 Processor -- start 2023-11-24T06:12:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.253+0000 7f828b98c700 1 -- start start 2023-11-24T06:12:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f828b98c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f828406c710 0x7f8284150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f828b98c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82840af4d0 0x7f82840b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f828b98c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828415e080 0x7f82840b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f828b98c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8284156da0 con 0x7f828415e080 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f828b98c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8284155d00 con 0x7f828406c710 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f828b98c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8284155fe0 con 0x7f82840af4d0 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828415e080 0x7f82840b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828415e080 0x7f82840b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50900/0 (socket says 172.21.15.97:50900) 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 -- 172.21.15.97:0/3383365776 learned_addr learned my addr 172.21.15.97:0/3383365776 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289728700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f828406c710 0x7f8284150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8288f27700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82840af4d0 0x7f82840b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 -- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82840af4d0 msgr2=0x7f82840b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82840af4d0 0x7f82840b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 -- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f828406c710 msgr2=0x7f8284150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f828406c710 0x7f8284150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 -- 172.21.15.97:0/3383365776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c009b30 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289728700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f828406c710 0x7f8284150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.254+0000 7f8289f29700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828415e080 0x7f82840b1cd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f828409f040 tx=0x7f827c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.255+0000 7f8275ffb700 1 -- 172.21.15.97:0/3383365776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c0048b0 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.255+0000 7f8275ffb700 1 -- 172.21.15.97:0/3383365776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c0099b0 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.255+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82841562c0 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.255+0000 7f8275ffb700 1 -- 172.21.15.97:0/3383365776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c01c680 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.255+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8284156880 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.256+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8284156bf0 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.256+0000 7f8275ffb700 1 -- 172.21.15.97:0/3383365776 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f827c01c7e0 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.257+0000 7f8275ffb700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f826806b470 0x7f826806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.257+0000 7f8275ffb700 1 -- 172.21.15.97:0/3383365776 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f827c08b720 con 0x7f828415e080 2023-11-24T06:12:02.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.257+0000 7f8289728700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f826806b470 0x7f826806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.257+0000 7f8289728700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f826806b470 0x7f826806d930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f82840b13a0 tx=0x7f8278008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.261+0000 7f8275ffb700 1 -- 172.21.15.97:0/3383365776 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f827c056180 con 0x7f828415e080 2023-11-24T06:12:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.492+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82840753f0 con 0x7f828415e080 2023-11-24T06:12:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.493+0000 7f8275ffb700 1 -- 172.21.15.97:0/3383365776 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f827c0597a0 con 0x7f828415e080 2023-11-24T06:12:02.496 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:02.496 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5597,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2882287048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":2882287048},{"type":"v1","addr":"172.21.15.183:6835","nonce":2882287048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5863,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/2765643379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":2765643379},{"type":"v1","addr":"172.21.15.183:6834","nonce":2765643379}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41},{"gid":6459,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1367625334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1367625334},{"type":"v1","addr":"172.21.15.97:6836","nonce":1367625334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6462,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/807372372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":807372372},{"type":"v1","addr":"172.21.15.97:6837","nonce":807372372}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[]} 2023-11-24T06:12:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f826806b470 msgr2=0x7f826806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f826806b470 0x7f826806d930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f82840b13a0 tx=0x7f8278008040 comp rx=0 tx=0).stop 2023-11-24T06:12:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828415e080 msgr2=0x7f82840b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828415e080 0x7f82840b1cd0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f828409f040 tx=0x7f827c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 shutdown_connections 2023-11-24T06:12:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f828415e080 0x7f82840b1cd0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f828406c710 0x7f8284150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f826806b470 0x7f826806d930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 --2- 172.21.15.97:0/3383365776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82840af4d0 0x7f82840b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 >> 172.21.15.97:0/3383365776 conn(0x7f8284071b90 msgr2=0x7f828406be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 shutdown_connections 2023-11-24T06:12:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.495+0000 7f828b98c700 1 -- 172.21.15.97:0/3383365776 wait complete. 2023-11-24T06:12:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 49 2023-11-24T06:12:02.510 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:12:02.510 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:12:02.510 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:12:02.510 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:02.513 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:12:02.513 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:12:02.514 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:12:02.514 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:02.514 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:12:02.514 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:12:02.514 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:12:02.514 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:02.514 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:12:02.515 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:12:02.515 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:12:02.515 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:02.519 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:12:02.520 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:12:02.523 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:12:02.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:12:02.526 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:12:02.526 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:12:02.527 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:12:02.527 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:12:02.567 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:12:02.567 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:02.568 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:12:02.568 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:02.569 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:12:02.569 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:02.570 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:12:02.570 DEBUG:teuthology.parallel:result is None 2023-11-24T06:12:02.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:12:02.645 INFO:tasks.ceph.mds.a.smithi097.stdout:starting mds.a at 2023-11-24T06:12:02.645 INFO:tasks.ceph.mds.c.smithi097.stdout:starting mds.c at 2023-11-24T06:12:02.653 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-11-24T06:12:02.653 INFO:tasks.ceph.mds.b.smithi183.stdout:starting mds.b at 2023-11-24T06:12:02.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.694+0000 7fbc0cd30700 1 Processor -- start 2023-11-24T06:12:02.696 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.694+0000 7fbc0cd30700 1 -- start start 2023-11-24T06:12:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0cd30700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc0815dfc0 0x7fbc080d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0cd30700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbc080d44e0 0x7fbc080d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0cd30700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc080d2d10 con 0x7fbc080b2fc0 2023-11-24T06:12:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0cd30700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc080cefd0 con 0x7fbc0815dfc0 2023-11-24T06:12:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0cd30700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc080cf2b0 con 0x7fbc080d44e0 2023-11-24T06:12:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc0815dfc0 0x7fbc080d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0659c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbc0815dfc0 0x7fbc080d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50928/0 (socket says 172.21.15.97:50928) 2023-11-24T06:12:02.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0659c700 1 -- 172.21.15.97:0/2781785938 learned_addr learned my addr 172.21.15.97:0/2781785938 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc06d9d700 1 --2- 172.21.15.97:0/2781785938 >> v2:172.21.15.183:3300/0 conn(0x7fbc080d44e0 0x7fbc080d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0659c700 1 -- 172.21.15.97:0/2781785938 >> v2:172.21.15.183:3300/0 conn(0x7fbc080d44e0 msgr2=0x7fbc080d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0659c700 1 --2- 172.21.15.97:0/2781785938 >> v2:172.21.15.183:3300/0 conn(0x7fbc080d44e0 0x7fbc080d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0659c700 1 -- 172.21.15.97:0/2781785938 >> v1:172.21.15.183:6789/0 conn(0x7fbc080b2fc0 legacy=0x7fbc08161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0659c700 1 -- 172.21.15.97:0/2781785938 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc080d2fb0 con 0x7fbc0815dfc0 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0659c700 1 --2- 172.21.15.97:0/2781785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc080d0e60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbc080caf20 tx=0x7fbbf8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e98d8396ed7d4e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.695+0000 7fbc0559a700 1 -- 172.21.15.97:0/2781785938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8004230 con 0x7fbc0815dfc0 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0559a700 1 -- 172.21.15.97:0/2781785938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf8004390 con 0x7fbc0815dfc0 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0559a700 1 -- 172.21.15.97:0/2781785938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf80036a0 con 0x7fbc0815dfc0 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0cd30700 1 -- 172.21.15.97:0/2781785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 msgr2=0x7fbc080d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/2781785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc080d0e60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbc080caf20 tx=0x7fbbf8009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0cd30700 1 -- 172.21.15.97:0/2781785938 shutdown_connections 2023-11-24T06:12:02.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/2781785938 >> v2:172.21.15.183:3300/0 conn(0x7fbc080d44e0 0x7fbc080d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/2781785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc080d0e60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0cd30700 1 -- 172.21.15.97:0/2781785938 >> 172.21.15.97:0/2781785938 conn(0x7fbc0806b410 msgr2=0x7fbc0806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:02.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0cd30700 1 -- 172.21.15.97:0/2781785938 shutdown_connections 2023-11-24T06:12:02.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.696+0000 7fbc0cd30700 1 -- 172.21.15.97:0/2781785938 wait complete. 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.697+0000 7fbc0cd30700 1 Processor -- start 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc0cd30700 1 -- start start 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc0cd30700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc080b2fc0 0x7fbc081f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc0cd30700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc080d44e0 0x7fbc080cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc0cd30700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc08152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc0cd30700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc081f8190 con 0x7fbc0815dfc0 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc0cd30700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080c8e20 con 0x7fbc080b2fc0 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc0cd30700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080c82a0 con 0x7fbc080d44e0 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc06d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc08152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc06d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc08152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50938/0 (socket says 172.21.15.97:50938) 2023-11-24T06:12:02.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc06d9d700 1 -- 172.21.15.97:0/4064840507 learned_addr learned my addr 172.21.15.97:0/4064840507 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc0659c700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc080b2fc0 0x7fbc081f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.698+0000 7fbc05d9b700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc080d44e0 0x7fbc080cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbc06d9d700 1 -- 172.21.15.97:0/4064840507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc080d44e0 msgr2=0x7fbc080cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbc06d9d700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc080d44e0 0x7fbc080cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbc06d9d700 1 -- 172.21.15.97:0/4064840507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc080b2fc0 msgr2=0x7fbc081f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbc06d9d700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc080b2fc0 0x7fbc081f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbc06d9d700 1 -- 172.21.15.97:0/4064840507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8009550 con 0x7fbc0815dfc0 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbc06d9d700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc08152fc0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007e30 tx=0x7fbc0000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbbf6ffd700 1 -- 172.21.15.97:0/4064840507 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0000ed30 con 0x7fbc0815dfc0 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbbf6ffd700 1 -- 172.21.15.97:0/4064840507 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0000ee90 con 0x7fbc0815dfc0 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc080c8580 con 0x7fbc0815dfc0 2023-11-24T06:12:02.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbbf6ffd700 1 -- 172.21.15.97:0/4064840507 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00014660 con 0x7fbc0815dfc0 2023-11-24T06:12:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.699+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc080c8b40 con 0x7fbc0815dfc0 2023-11-24T06:12:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.700+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0815f6a0 con 0x7fbc0815dfc0 2023-11-24T06:12:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.700+0000 7fbbf6ffd700 1 -- 172.21.15.97:0/4064840507 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fbc000147c0 con 0x7fbc0815dfc0 2023-11-24T06:12:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.701+0000 7fbbf6ffd700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbbe806b470 0x7fbbe806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.701+0000 7fbbf6ffd700 1 -- 172.21.15.97:0/4064840507 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fbc0008a310 con 0x7fbc0815dfc0 2023-11-24T06:12:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.701+0000 7fbc0659c700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbbe806b470 0x7fbbe806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:02.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.701+0000 7fbc0659c700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbbe806b470 0x7fbbe806d930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009130 tx=0x7fbbf8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:02.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.706+0000 7fbbf6ffd700 1 -- 172.21.15.97:0/4064840507 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc00054f30 con 0x7fbc0815dfc0 2023-11-24T06:12:02.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:02.914+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbc08067f60 con 0x7fbc0815dfc0 2023-11-24T06:12:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.930+0000 7fbbf6ffd700 1 -- 172.21.15.97:0/4064840507 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbc00058550 con 0x7fbc0815dfc0 2023-11-24T06:12:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbbe806b470 msgr2=0x7fbbe806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbbe806b470 0x7fbbe806d930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009130 tx=0x7fbbf8008030 comp rx=0 tx=0).stop 2023-11-24T06:12:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 msgr2=0x7fbc08152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc08152fc0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007e30 tx=0x7fbc0000e3c0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 shutdown_connections 2023-11-24T06:12:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc0815dfc0 0x7fbc08152fc0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbc080b2fc0 0x7fbc081f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbbe806b470 0x7fbbe806d930 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 --2- 172.21.15.97:0/4064840507 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbc080d44e0 0x7fbc080cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 >> 172.21.15.97:0/4064840507 conn(0x7fbc0806b410 msgr2=0x7fbc080d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 shutdown_connections 2023-11-24T06:12:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:03.933+0000 7fbc0cd30700 1 -- 172.21.15.97:0/4064840507 wait complete. 2023-11-24T06:12:03.938 INFO:teuthology.orchestra.run.smithi097.stderr: removed all blocklist entries 2023-11-24T06:12:03.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:12:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.073+0000 7f4360ff1700 1 Processor -- start 2023-11-24T06:12:04.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.073+0000 7f4360ff1700 1 -- start start 2023-11-24T06:12:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f4360ff1700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f435c15ad00 0x7f435c06ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f4360ff1700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f435c0b3d20 0x7f435c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f4360ff1700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c06f240 con 0x7f435c0b2050 2023-11-24T06:12:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f4360ff1700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c15b0e0 con 0x7f435c15ad00 2023-11-24T06:12:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f4360ff1700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c06dc60 con 0x7f435c0b3d20 2023-11-24T06:12:04.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f435a59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f435c15ad00 0x7f435c06ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f435a59c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f435c15ad00 0x7f435c06ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50946/0 (socket says 172.21.15.97:50946) 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f435a59c700 1 -- 172.21.15.97:0/105425486 learned_addr learned my addr 172.21.15.97:0/105425486 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f4359d9b700 1 --2- 172.21.15.97:0/105425486 >> v2:172.21.15.183:3300/0 conn(0x7f435c0b3d20 0x7f435c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f435a59c700 1 -- 172.21.15.97:0/105425486 >> v2:172.21.15.183:3300/0 conn(0x7f435c0b3d20 msgr2=0x7f435c06d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f435a59c700 1 --2- 172.21.15.97:0/105425486 >> v2:172.21.15.183:3300/0 conn(0x7f435c0b3d20 0x7f435c06d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f435a59c700 1 -- 172.21.15.97:0/105425486 >> v1:172.21.15.183:6789/0 conn(0x7f435c0b2050 legacy=0x7f435c0b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.074+0000 7f435a59c700 1 -- 172.21.15.97:0/105425486 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c06df40 con 0x7f435c15ad00 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f435a59c700 1 --2- 172.21.15.97:0/105425486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c06ed00 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f435c0b2bf0 tx=0x7f434c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336737ecad74e7e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f435959a700 1 -- 172.21.15.97:0/105425486 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c004230 con 0x7f435c15ad00 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f435959a700 1 -- 172.21.15.97:0/105425486 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c004390 con 0x7f435c15ad00 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f435959a700 1 -- 172.21.15.97:0/105425486 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c0036a0 con 0x7f435c15ad00 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f4360ff1700 1 -- 172.21.15.97:0/105425486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 msgr2=0x7f435c06ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f4360ff1700 1 --2- 172.21.15.97:0/105425486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c06ed00 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f435c0b2bf0 tx=0x7f434c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f4360ff1700 1 -- 172.21.15.97:0/105425486 shutdown_connections 2023-11-24T06:12:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f4360ff1700 1 --2- 172.21.15.97:0/105425486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c06ed00 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f4360ff1700 1 --2- 172.21.15.97:0/105425486 >> v2:172.21.15.183:3300/0 conn(0x7f435c0b3d20 0x7f435c06d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f4360ff1700 1 -- 172.21.15.97:0/105425486 >> 172.21.15.97:0/105425486 conn(0x7f435c0743b0 msgr2=0x7f435c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f4360ff1700 1 -- 172.21.15.97:0/105425486 shutdown_connections 2023-11-24T06:12:04.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.075+0000 7f4360ff1700 1 -- 172.21.15.97:0/105425486 wait complete. 2023-11-24T06:12:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.077+0000 7f4360ff1700 1 Processor -- start 2023-11-24T06:12:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.077+0000 7f4360ff1700 1 -- start start 2023-11-24T06:12:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.077+0000 7f4360ff1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f435c0b2050 0x7f435c14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.077+0000 7f4360ff1700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f435c0b3d20 0x7f435c0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.077+0000 7f4360ff1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c14d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.077+0000 7f4360ff1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1501f0 con 0x7f435c15ad00 2023-11-24T06:12:04.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.077+0000 7f4360ff1700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c15c6e0 con 0x7f435c0b2050 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.077+0000 7f4360ff1700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c15c9c0 con 0x7f435c0b3d20 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c14d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c14d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50956/0 (socket says 172.21.15.97:50956) 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 -- 172.21.15.97:0/2372312599 learned_addr learned my addr 172.21.15.97:0/2372312599 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f4359d9b700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f435c0b3d20 0x7f435c0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435a59c700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f435c0b2050 0x7f435c14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 -- 172.21.15.97:0/2372312599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f435c0b3d20 msgr2=0x7f435c0a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f435c0b3d20 0x7f435c0a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 -- 172.21.15.97:0/2372312599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f435c0b2050 msgr2=0x7f435c14d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f435c0b2050 0x7f435c14d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 -- 172.21.15.97:0/2372312599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c009550 con 0x7f435c15ad00 2023-11-24T06:12:04.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f435ad9d700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c14d970 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f435400cc10 tx=0x7f435400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f434affd700 1 -- 172.21.15.97:0/2372312599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400aa60 con 0x7f435c15ad00 2023-11-24T06:12:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f434affd700 1 -- 172.21.15.97:0/2372312599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435400abc0 con 0x7f435c15ad00 2023-11-24T06:12:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c15cca0 con 0x7f435c15ad00 2023-11-24T06:12:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f434affd700 1 -- 172.21.15.97:0/2372312599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354012420 con 0x7f435c15ad00 2023-11-24T06:12:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.078+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c15d260 con 0x7f435c15ad00 2023-11-24T06:12:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.079+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c15d5d0 con 0x7f435c15ad00 2023-11-24T06:12:04.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.080+0000 7f434affd700 1 -- 172.21.15.97:0/2372312599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4354012580 con 0x7f435c15ad00 2023-11-24T06:12:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.080+0000 7f434affd700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f433c06b470 0x7f433c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.080+0000 7f434affd700 1 -- 172.21.15.97:0/2372312599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f4354089f10 con 0x7f435c15ad00 2023-11-24T06:12:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.080+0000 7f435a59c700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f433c06b470 0x7f433c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.080+0000 7f435a59c700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f433c06b470 0x7f433c06d930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f434c009130 tx=0x7f434c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.084+0000 7f434affd700 1 -- 172.21.15.97:0/2372312599 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4354054b10 con 0x7f435c15ad00 2023-11-24T06:12:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.354+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f435c067f60 con 0x7f435c15ad00 2023-11-24T06:12:04.357 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.355+0000 7f434affd700 1 -- 172.21.15.97:0/2372312599 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4354058130 con 0x7f435c15ad00 2023-11-24T06:12:04.358 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:04.358 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:04.358 INFO:teuthology.orchestra.run.smithi097.stdout: "auth_dump": [ 2023-11-24T06:12:04.358 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.358 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.a", 2023-11-24T06:12:04.358 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlijU/IxAAp5/+zEd6ER/JFqum0ptk0Q==", 2023-11-24T06:12:04.358 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.358 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.b", 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLftpLxAAH+Ew2GQv73YtKLzZJjSBew==", 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.361 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.c", 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlGtxpKhAAiQm5ore3YYETyG+sIKzEcw==", 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:12:04.362 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mds.d", 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBl05iyNhAAHZYZ5pGghTwe55n84iXXQw==", 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.363 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.0", 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDAPWBlwipJFBAAGLCD8wGcYoSy4yM4RXb0/Q==", 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.364 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.1", 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDDPWBlgW7fExAAQuUSGJn+ZfHSMwSi5r1pQg==", 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.2", 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDGPWBl3E8uExAAJKjzbRFyYie6dmnRrP7VVw==", 2023-11-24T06:12:04.365 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.3", 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDJPWBl5gnRExAAbKbgF6DcdTSv8M84GXyONw==", 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:04.366 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.4", 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDPPWBlmCfaFRAAsIQWHUnlgDQcRXLndlUZKg==", 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.5", 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDSPWBlPxfqFRAARyMzgivfTfgmFxZRjp54JQ==", 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.6", 2023-11-24T06:12:04.368 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDVPWBl8cjTFBAAsNfkNY4gGVTJwh+mMtADgg==", 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "osd.7", 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQDYPWBljh8EFBAA4nVPoyEXigIDTUB6enXsXA==", 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow profile osd", 2023-11-24T06:12:04.369 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile osd", 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.0", 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA==", 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow r", 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.1", 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlOXAsBhAABOT6CW2RVj7Iv5XMPvQTgQ==", 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.2", 2023-11-24T06:12:04.371 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlJRe4AhAAGQVWmqQr/urD7AefOfmoZQ==", 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.3", 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC9PWBlwX2YCRAAIM9eB39aohf1xUdEvHr/tA==", 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow", 2023-11-24T06:12:04.372 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow r", 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow rw", 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow rwx" 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "client.admin", 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlVBcHBhAAbu6IOCKCLUA+5APAYwbRUw==", 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr": "allow *", 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow *", 2023-11-24T06:12:04.373 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.x", 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBlOezbHxAAlNbpLHDzRHvk2SXFvQoTXQ==", 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:04.374 INFO:teuthology.orchestra.run.smithi097.stdout: "entity": "mgr.y", 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout: "key": "AQC8PWBla7myGxAAHEPHxXHfBzCeqi2nLjhK6g==", 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": { 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": "allow *", 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout: "mon": "allow profile mgr", 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout: "osd": "allow *" 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-11-24T06:12:04.375 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:04.376 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f433c06b470 msgr2=0x7f433c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f433c06b470 0x7f433c06d930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f434c009130 tx=0x7f434c008030 comp rx=0 tx=0).stop 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 msgr2=0x7f435c14d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c14d970 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f435400cc10 tx=0x7f435400cf20 comp rx=0 tx=0).stop 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 shutdown_connections 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f435c15ad00 0x7f435c14d970 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f435c0b2050 0x7f435c14d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f433c06b470 0x7f433c06d930 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 --2- 172.21.15.97:0/2372312599 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f435c0b3d20 0x7f435c0a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 >> 172.21.15.97:0/2372312599 conn(0x7f435c0743b0 msgr2=0x7f435c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 shutdown_connections 2023-11-24T06:12:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.357+0000 7f4360ff1700 1 -- 172.21.15.97:0/2372312599 wait complete. 2023-11-24T06:12:04.378 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.495+0000 7fa6769c6700 1 Processor -- start 2023-11-24T06:12:04.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.495+0000 7fa6769c6700 1 -- start start 2023-11-24T06:12:04.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa6769c6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa67006c710 0x7fa670165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa6769c6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700af950 con 0x7fa6700adf00 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa6769c6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700afc30 con 0x7fa6701446a0 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa6769c6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa670144a80 con 0x7fa67006c710 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa6700adf00 0x7fa67006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43414/0 (socket says 172.21.15.97:43414) 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66f7fe700 1 -- 172.21.15.97:0/861169285 learned_addr learned my addr 172.21.15.97:0/861169285 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa674f63700 1 --2- 172.21.15.97:0/861169285 >> v2:172.21.15.183:3300/0 conn(0x7fa67006c710 0x7fa670165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311353612 0 0) 0x7fa6700af950 con 0x7fa6700adf00 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa65c0035b0 con 0x7fa6700adf00 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3585977808 0 0) 0x7fa65c0035b0 con 0x7fa6700adf00 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 >> v2:172.21.15.183:3300/0 conn(0x7fa67006c710 msgr2=0x7fa670165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 --2- 172.21.15.97:0/861169285 >> v2:172.21.15.183:3300/0 conn(0x7fa67006c710 0x7fa670165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 >> v1:172.21.15.183:6789/0 conn(0x7fa6701446a0 legacy=0x7fa6700b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700aff10 con 0x7fa6700adf00 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fa658001670 con 0x7fa6700adf00 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa674f63700 1 --2- 172.21.15.97:0/861169285 >> v2:172.21.15.183:3300/0 conn(0x7fa67006c710 0x7fa670165b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 >> v1:172.21.15.97:6789/0 conn(0x7fa6700adf00 legacy=0x7fa67006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.496+0000 7fa66effd700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c003a00 0x7fa65c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66effd700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65c004850 0x7fa65c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66effd700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa65c005210 0x7fa65c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa670144a80 con 0x7fa65c004850 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c00c1b0 con 0x7fa65c005210 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c00c370 con 0x7fa65c003a00 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66f7fe700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65c004850 0x7fa65c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66ffff700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c003a00 0x7fa65c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa674f63700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa65c005210 0x7fa65c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66f7fe700 1 -- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c003a00 msgr2=0x7fa65c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66f7fe700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c003a00 0x7fa65c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66f7fe700 1 -- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa65c005210 msgr2=0x7fa65c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66f7fe700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa65c005210 0x7fa65c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66f7fe700 1 -- 172.21.15.97:0/861169285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0035b0 con 0x7fa65c004850 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa674f63700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa65c005210 0x7fa65c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa674f63700 1 -- 172.21.15.97:0/861169285 reap_dead start 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66f7fe700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65c004850 0x7fa65c004cd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa67006f1d0 tx=0x7fa65c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e86de751723fa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65800cdd0 con 0x7fa65c004850 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa658005960 con 0x7fa65c004850 2023-11-24T06:12:04.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.497+0000 7fa66effd700 1 -- 172.21.15.97:0/861169285 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658005cf0 con 0x7fa65c004850 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.498+0000 7fa6769c6700 1 -- 172.21.15.97:0/861169285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65c004850 msgr2=0x7fa65c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.498+0000 7fa6769c6700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65c004850 0x7fa65c004cd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa67006f1d0 tx=0x7fa65c005880 comp rx=0 tx=0).stop 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.498+0000 7fa6769c6700 1 -- 172.21.15.97:0/861169285 shutdown_connections 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.498+0000 7fa6769c6700 1 --2- 172.21.15.97:0/861169285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa65c004850 0x7fa65c004cd0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.498+0000 7fa6769c6700 1 -- 172.21.15.97:0/861169285 >> 172.21.15.97:0/861169285 conn(0x7fa670071b90 msgr2=0x7fa670071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.498+0000 7fa6769c6700 1 -- 172.21.15.97:0/861169285 shutdown_connections 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.498+0000 7fa6769c6700 1 -- 172.21.15.97:0/861169285 wait complete. 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.499+0000 7fa6769c6700 1 Processor -- start 2023-11-24T06:12:04.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.499+0000 7fa6769c6700 1 -- start start 2023-11-24T06:12:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa6769c6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c004850 0x7fa6700cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa6769c6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa67015a430 0x7fa670074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa6769c6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700c5e00 0x7fa6700c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa6769c6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700cfc30 con 0x7fa6700c5e00 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa6769c6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700ccbf0 con 0x7fa67015a430 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa6769c6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67014b4c0 con 0x7fa65c004850 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700c5e00 0x7fa6700c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700c5e00 0x7fa6700c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50976/0 (socket says 172.21.15.97:50976) 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 -- 172.21.15.97:0/2794490572 learned_addr learned my addr 172.21.15.97:0/2794490572 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa66f7fe700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa67015a430 0x7fa670074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa66ffff700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c004850 0x7fa6700cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 -- 172.21.15.97:0/2794490572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c004850 msgr2=0x7fa6700cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c004850 0x7fa6700cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 -- 172.21.15.97:0/2794490572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa67015a430 msgr2=0x7fa670074200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa67015a430 0x7fa670074200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 -- 172.21.15.97:0/2794490572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700cf390 con 0x7fa6700c5e00 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa674f63700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700c5e00 0x7fa6700c6250 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa6700ae860 tx=0x7fa6680114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.500+0000 7fa66cff9700 1 -- 172.21.15.97:0/2794490572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800e070 con 0x7fa6700c5e00 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.501+0000 7fa66cff9700 1 -- 172.21.15.97:0/2794490572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668003c30 con 0x7fa6700c5e00 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.501+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700cf670 con 0x7fa6700c5e00 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.501+0000 7fa66cff9700 1 -- 172.21.15.97:0/2794490572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668003da0 con 0x7fa6700c5e00 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.501+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6700cef80 con 0x7fa6700c5e00 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.502+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6700753f0 con 0x7fa6700c5e00 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.502+0000 7fa66cff9700 1 -- 172.21.15.97:0/2794490572 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fa6680119b0 con 0x7fa6700c5e00 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.502+0000 7fa66cff9700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa65006b470 0x7fa65006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.502+0000 7fa66ffff700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa65006b470 0x7fa65006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.502+0000 7fa66cff9700 1 -- 172.21.15.97:0/2794490572 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fa66808a680 con 0x7fa6700c5e00 2023-11-24T06:12:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.503+0000 7fa66ffff700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa65006b470 0x7fa65006d930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa660007fd0 tx=0x7fa660009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.507+0000 7fa66cff9700 1 -- 172.21.15.97:0/2794490572 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa668055050 con 0x7fa6700c5e00 2023-11-24T06:12:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.699+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa67014b290 con 0x7fa6700c5e00 2023-11-24T06:12:04.702 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.700+0000 7fa66cff9700 1 -- 172.21.15.97:0/2794490572 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+12913 (secure 0 0 0) 0x7fa67014b290 con 0x7fa6700c5e00 2023-11-24T06:12:04.703 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:04.703 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":81,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:03.929621+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa65006b470 msgr2=0x7fa65006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa65006b470 0x7fa65006d930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa660007fd0 tx=0x7fa660009040 comp rx=0 tx=0).stop 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700c5e00 msgr2=0x7fa6700c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700c5e00 0x7fa6700c6250 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa6700ae860 tx=0x7fa6680114b0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 shutdown_connections 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6700c5e00 0x7fa6700c6250 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa65006b470 0x7fa65006d930 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa65c004850 0x7fa6700cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 --2- 172.21.15.97:0/2794490572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa67015a430 0x7fa670074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 >> 172.21.15.97:0/2794490572 conn(0x7fa670071b90 msgr2=0x7fa67015dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 shutdown_connections 2023-11-24T06:12:04.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.702+0000 7fa6769c6700 1 -- 172.21.15.97:0/2794490572 wait complete. 2023-11-24T06:12:04.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:04.717 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.838+0000 7f1d1a745700 1 Processor -- start 2023-11-24T06:12:04.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.838+0000 7f1d1a745700 1 -- start start 2023-11-24T06:12:04.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d1a745700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d140746a0 0x7f1d140b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d1a745700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140afc90 0x7f1d1406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d1a745700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d140b3c60 con 0x7f1d1415e160 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d1a745700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d140b4500 con 0x7f1d140746a0 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d1a745700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d140b58c0 con 0x7f1d140afc90 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d137fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d140746a0 0x7f1d140b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d137fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1d140746a0 0x7f1d140b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50982/0 (socket says 172.21.15.97:50982) 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d137fe700 1 -- 172.21.15.97:0/2545793550 learned_addr learned my addr 172.21.15.97:0/2545793550 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d18ce2700 1 --2- 172.21.15.97:0/2545793550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140afc90 0x7f1d1406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d137fe700 1 -- 172.21.15.97:0/2545793550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140afc90 msgr2=0x7f1d1406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d137fe700 1 --2- 172.21.15.97:0/2545793550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140afc90 0x7f1d1406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d137fe700 1 -- 172.21.15.97:0/2545793550 >> v1:172.21.15.183:6789/0 conn(0x7f1d1415e160 legacy=0x7f1d14073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.839+0000 7f1d137fe700 1 -- 172.21.15.97:0/2545793550 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d140b0120 con 0x7f1d140746a0 2023-11-24T06:12:04.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d137fe700 1 --2- 172.21.15.97:0/2545793550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d140746a0 0x7f1d140b5da0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1d040098f0 tx=0x7f1d04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47a7b17986805bb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d12ffd700 1 -- 172.21.15.97:0/2545793550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04011830 con 0x7f1d140746a0 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d12ffd700 1 -- 172.21.15.97:0/2545793550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d04011990 con 0x7f1d140746a0 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d12ffd700 1 -- 172.21.15.97:0/2545793550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04011d20 con 0x7f1d140746a0 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d1a745700 1 -- 172.21.15.97:0/2545793550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d140746a0 msgr2=0x7f1d140b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2545793550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d140746a0 0x7f1d140b5da0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f1d040098f0 tx=0x7f1d04009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d1a745700 1 -- 172.21.15.97:0/2545793550 shutdown_connections 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2545793550 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140afc90 0x7f1d1406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2545793550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d140746a0 0x7f1d140b5da0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.840+0000 7f1d1a745700 1 -- 172.21.15.97:0/2545793550 >> 172.21.15.97:0/2545793550 conn(0x7f1d140d0bf0 msgr2=0x7f1d140d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.841+0000 7f1d1a745700 1 -- 172.21.15.97:0/2545793550 shutdown_connections 2023-11-24T06:12:04.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.841+0000 7f1d1a745700 1 -- 172.21.15.97:0/2545793550 wait complete. 2023-11-24T06:12:04.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.842+0000 7f1d1a745700 1 Processor -- start 2023-11-24T06:12:04.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.842+0000 7f1d1a745700 1 -- start start 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d1a745700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140746a0 0x7f1d14148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d1a745700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d140afc90 0x7f1d14160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d1a745700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d1415e160 0x7f1d140ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d1a745700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14157010 con 0x7f1d1415e160 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d1a745700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d141606c0 con 0x7f1d140afc90 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d1a745700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d14154480 con 0x7f1d140746a0 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d1415e160 0x7f1d140ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d1415e160 0x7f1d140ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50990/0 (socket says 172.21.15.97:50990) 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 -- 172.21.15.97:0/2850963949 learned_addr learned my addr 172.21.15.97:0/2850963949 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d137fe700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d140afc90 0x7f1d14160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d13fff700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140746a0 0x7f1d14148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 -- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140746a0 msgr2=0x7f1d14148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140746a0 0x7f1d14148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 -- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d140afc90 msgr2=0x7f1d14160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d140afc90 0x7f1d14160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 -- 172.21.15.97:0/2850963949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04009550 con 0x7f1d1415e160 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d137fe700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d140afc90 0x7f1d14160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d18ce2700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d1415e160 0x7f1d140ae5d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00efc0 tx=0x7f1d0c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.843+0000 7f1d10ff9700 1 -- 172.21.15.97:0/2850963949 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00cd40 con 0x7f1d1415e160 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.844+0000 7f1d10ff9700 1 -- 172.21.15.97:0/2850963949 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c00f040 con 0x7f1d1415e160 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.844+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d14153900 con 0x7f1d1415e160 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.844+0000 7f1d10ff9700 1 -- 172.21.15.97:0/2850963949 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c0146c0 con 0x7f1d1415e160 2023-11-24T06:12:04.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.844+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d14153ec0 con 0x7f1d1415e160 2023-11-24T06:12:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.844+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d14062580 con 0x7f1d1415e160 2023-11-24T06:12:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.845+0000 7f1d10ff9700 1 -- 172.21.15.97:0/2850963949 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1d0c014820 con 0x7f1d1415e160 2023-11-24T06:12:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.845+0000 7f1d10ff9700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d0006b470 0x7f1d0006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.845+0000 7f1d10ff9700 1 -- 172.21.15.97:0/2850963949 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f1d0c08a820 con 0x7f1d1415e160 2023-11-24T06:12:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.845+0000 7f1d13fff700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d0006b470 0x7f1d0006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:04.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.846+0000 7f1d13fff700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d0006b470 0x7f1d0006d930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000f60 tx=0x7f1d08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:04.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:04.850+0000 7f1d10ff9700 1 -- 172.21.15.97:0/2850963949 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d0c055240 con 0x7f1d1415e160 2023-11-24T06:12:05.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.040+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d14157150 con 0x7f1d1415e160 2023-11-24T06:12:05.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.040+0000 7f1d10ff9700 1 -- 172.21.15.97:0/2850963949 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d0c00cea0 con 0x7f1d1415e160 2023-11-24T06:12:05.043 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:12:05.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d0006b470 msgr2=0x7f1d0006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d0006b470 0x7f1d0006d930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1d08000f60 tx=0x7f1d08009040 comp rx=0 tx=0).stop 2023-11-24T06:12:05.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d1415e160 msgr2=0x7f1d140ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d1415e160 0x7f1d140ae5d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00efc0 tx=0x7f1d0c00c3a0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 shutdown_connections 2023-11-24T06:12:05.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d1415e160 0x7f1d140ae5d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d0006b470 0x7f1d0006d930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1d140746a0 0x7f1d14148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 --2- 172.21.15.97:0/2850963949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d140afc90 0x7f1d14160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 >> 172.21.15.97:0/2850963949 conn(0x7f1d140d0bf0 msgr2=0x7f1d14073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:05.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 shutdown_connections 2023-11-24T06:12:05.046 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.042+0000 7f1d1a745700 1 -- 172.21.15.97:0/2850963949 wait complete. 2023-11-24T06:12:05.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:12:05.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.173+0000 7f7b80c92700 1 Processor -- start 2023-11-24T06:12:05.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.173+0000 7f7b80c92700 1 -- start start 2023-11-24T06:12:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.174+0000 7f7b80c92700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c0cff10 0x7f7b7c165bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.174+0000 7f7b80c92700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0b4290 con 0x7f7b7c15f180 2023-11-24T06:12:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.174+0000 7f7b80c92700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c0d0380 con 0x7f7b7c13e000 2023-11-24T06:12:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.174+0000 7f7b80c92700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c13e570 con 0x7f7b7c0cff10 2023-11-24T06:12:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.174+0000 7f7b79d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7b7c15f180 0x7f7b7c0cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43448/0 (socket says 172.21.15.97:43448) 2023-11-24T06:12:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.174+0000 7f7b79d9b700 1 -- 172.21.15.97:0/2898200639 learned_addr learned my addr 172.21.15.97:0/2898200639 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7ad9d700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c0cff10 0x7f7b7c165bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667953399 0 0) 0x7f7b7c0b4290 con 0x7f7b7c15f180 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b680035b0 con 0x7f7b7c15f180 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4009546861 0 0) 0x7f7b680035b0 con 0x7f7b7c15f180 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c0cff10 msgr2=0x7f7b7c165bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c0cff10 0x7f7b7c165bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 >> v1:172.21.15.183:6789/0 conn(0x7f7b7c13e000 legacy=0x7f7b7c15ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0d05d0 con 0x7f7b7c15f180 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f7b6c001670 con 0x7f7b7c15f180 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 >> v1:172.21.15.97:6789/0 conn(0x7f7b7c15f180 legacy=0x7f7b7c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b68003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c0cff10 0x7f7b68004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b68004d90 0x7f7b68005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c13e570 con 0x7f7b68003a30 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6800bb70 con 0x7f7b68004d90 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6800bd30 con 0x7f7b7c0cff10 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7a59c700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b68003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7ad9d700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b68004d90 0x7f7b68005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b79d9b700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c0cff10 0x7f7b68004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7a59c700 1 -- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c0cff10 msgr2=0x7f7b68004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7a59c700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c0cff10 0x7f7b68004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7a59c700 1 -- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b68004d90 msgr2=0x7f7b68005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7a59c700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b68004d90 0x7f7b68005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.175+0000 7f7b7a59c700 1 -- 172.21.15.97:0/2898200639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680035b0 con 0x7f7b68003a30 2023-11-24T06:12:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b7a59c700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b68003e50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7b640017c0 tx=0x7f7b68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865c58d398283d00 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b640052d0 con 0x7f7b68003a30 2023-11-24T06:12:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b64005430 con 0x7f7b68003a30 2023-11-24T06:12:05.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b7959a700 1 -- 172.21.15.97:0/2898200639 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b640038c0 con 0x7f7b68003a30 2023-11-24T06:12:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b80c92700 1 -- 172.21.15.97:0/2898200639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 msgr2=0x7f7b68003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b80c92700 1 --2- 172.21.15.97:0/2898200639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b68003e50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7b640017c0 tx=0x7f7b68005880 comp rx=0 tx=0).stop 2023-11-24T06:12:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b7ad9d700 1 -- 172.21.15.97:0/2898200639 reap_dead start 2023-11-24T06:12:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b80c92700 1 -- 172.21.15.97:0/2898200639 shutdown_connections 2023-11-24T06:12:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b80c92700 1 -- 172.21.15.97:0/2898200639 >> 172.21.15.97:0/2898200639 conn(0x7f7b7c072990 msgr2=0x7f7b7c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b80c92700 1 -- 172.21.15.97:0/2898200639 shutdown_connections 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.176+0000 7f7b80c92700 1 -- 172.21.15.97:0/2898200639 wait complete. 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b80c92700 1 Processor -- start 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b80c92700 1 -- start start 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b80c92700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b7c15b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b80c92700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c06e800 0x7f7b7c06ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b80c92700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b7c159fe0 0x7f7b7c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b80c92700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c142080 con 0x7f7b68003a30 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b80c92700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c142320 con 0x7f7b7c159fe0 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b80c92700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c142600 con 0x7f7b7c06e800 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b7a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b7c15b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b7a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b7c15b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51006/0 (socket says 172.21.15.97:51006) 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.178+0000 7f7b7a59c700 1 -- 172.21.15.97:0/3389891350 learned_addr learned my addr 172.21.15.97:0/3389891350 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b7ad9d700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b7c159fe0 0x7f7b7c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b79d9b700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c06e800 0x7f7b7c06ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b7a59c700 1 -- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c06e800 msgr2=0x7f7b7c06ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b7a59c700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c06e800 0x7f7b7c06ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b7a59c700 1 -- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b7c159fe0 msgr2=0x7f7b7c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b7a59c700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b7c159fe0 0x7f7b7c15a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b7a59c700 1 -- 172.21.15.97:0/3389891350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c1428e0 con 0x7f7b68003a30 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b7ad9d700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b7c159fe0 0x7f7b7c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b7a59c700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b7c15b5f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000c00 tx=0x7f7b64001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b72ffd700 1 -- 172.21.15.97:0/3389891350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64007450 con 0x7f7b68003a30 2023-11-24T06:12:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b72ffd700 1 -- 172.21.15.97:0/3389891350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b640075b0 con 0x7f7b68003a30 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7c142bc0 con 0x7f7b68003a30 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b72ffd700 1 -- 172.21.15.97:0/3389891350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b640038c0 con 0x7f7b68003a30 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.179+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c1430e0 con 0x7f7b68003a30 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.180+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7c067f60 con 0x7f7b68003a30 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.180+0000 7f7b72ffd700 1 -- 172.21.15.97:0/3389891350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f7b64003a20 con 0x7f7b68003a30 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.181+0000 7f7b72ffd700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b5c06b470 0x7f7b5c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.181+0000 7f7b72ffd700 1 -- 172.21.15.97:0/3389891350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f7b6408a890 con 0x7f7b68003a30 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.181+0000 7f7b79d9b700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b5c06b470 0x7f7b5c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.181+0000 7f7b79d9b700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b5c06b470 0x7f7b5c06d930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c144aa0 tx=0x7f7b6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:05.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.185+0000 7f7b72ffd700 1 -- 172.21.15.97:0/3389891350 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b64055490 con 0x7f7b68003a30 2023-11-24T06:12:05.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.378+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7b7c06bd40 con 0x7f7b68003a30 2023-11-24T06:12:05.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.378+0000 7f7b72ffd700 1 -- 172.21.15.97:0/3389891350 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7b64058ab0 con 0x7f7b68003a30 2023-11-24T06:12:05.381 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:05.381 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:12:05.381 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:05.381 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2023-11-24T06:12:05.381 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2023-11-24T06:12:05.381 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:05.381 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:12:05.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b5c06b470 msgr2=0x7f7b5c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.385 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b5c06b470 0x7f7b5c06d930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c144aa0 tx=0x7f7b6c009040 comp rx=0 tx=0).stop 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 msgr2=0x7f7b7c15b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b7c15b5f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7b64000c00 tx=0x7f7b64001120 comp rx=0 tx=0).stop 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 shutdown_connections 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b7c159fe0 0x7f7b7c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b68003a30 0x7f7b7c15b5f0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7b5c06b470 0x7f7b5c06d930 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 --2- 172.21.15.97:0/3389891350 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7b7c06e800 0x7f7b7c06ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 >> 172.21.15.97:0/3389891350 conn(0x7f7b7c072990 msgr2=0x7f7b7c140960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 shutdown_connections 2023-11-24T06:12:05.386 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.380+0000 7f7b80c92700 1 -- 172.21.15.97:0/3389891350 wait complete. 2023-11-24T06:12:05.395 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:12:05.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:12:05.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.517+0000 7f99ec59e700 1 Processor -- start 2023-11-24T06:12:05.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.517+0000 7f99ec59e700 1 -- start start 2023-11-24T06:12:05.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99ec59e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40b3b50 0x7f99e406e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99ec59e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e40b4410 con 0x7f99e4073f90 2023-11-24T06:12:05.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99ec59e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e406dd80 con 0x7f99e40b5850 2023-11-24T06:12:05.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99ec59e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40b4130 con 0x7f99e40b3b50 2023-11-24T06:12:05.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99ea33a700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f99e4073f90 0x7f99e4074370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43466/0 (socket says 172.21.15.97:43466) 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99ea33a700 1 -- 172.21.15.97:0/3475505223 learned_addr learned my addr 172.21.15.97:0/3475505223 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128645467 0 0) 0x7f99e40b4410 con 0x7f99e4073f90 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99d00035b0 con 0x7f99e4073f90 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.518+0000 7f99e9b39700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40b3b50 0x7f99e406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1068467576 0 0) 0x7f99d00035b0 con 0x7f99e4073f90 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40b3b50 msgr2=0x7f99e406e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40b3b50 0x7f99e406e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 >> v1:172.21.15.183:6789/0 conn(0x7f99e40b5850 legacy=0x7f99e406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e406e040 con 0x7f99e4073f90 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f99d4001670 con 0x7f99e4073f90 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 >> v1:172.21.15.97:6789/0 conn(0x7f99e4073f90 legacy=0x7f99e4074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40b3b50 0x7f99d0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004060 0x7f99d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99d0004d90 0x7f99d0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40b4130 con 0x7f99d0004d90 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d000bc50 con 0x7f99d0004060 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d000be10 con 0x7f99e40b3b50 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99eab3b700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99d0004d90 0x7f99d0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99ea33a700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40b3b50 0x7f99d0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9b39700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004060 0x7f99d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99eab3b700 1 -- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40b3b50 msgr2=0x7f99d0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99eab3b700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40b3b50 0x7f99d0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.525 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99eab3b700 1 -- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004060 msgr2=0x7f99d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99eab3b700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004060 0x7f99d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99eab3b700 1 -- 172.21.15.97:0/3475505223 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00035b0 con 0x7f99d0004d90 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99eab3b700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99d0004d90 0x7f99d0005210 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00c190 tx=0x7f99d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e4f96a5dd1f379 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc001db0 con 0x7f99d0004d90 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc00ccd0 con 0x7f99d0004d90 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.519+0000 7f99e9338700 1 -- 172.21.15.97:0/3475505223 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc003c40 con 0x7f99d0004d90 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.520+0000 7f99ec59e700 1 -- 172.21.15.97:0/3475505223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99d0004d90 msgr2=0x7f99d0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.520+0000 7f99ec59e700 1 --2- 172.21.15.97:0/3475505223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99d0004d90 0x7f99d0005210 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00c190 tx=0x7f99d0005880 comp rx=0 tx=0).stop 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.520+0000 7f99eab3b700 1 -- 172.21.15.97:0/3475505223 reap_dead start 2023-11-24T06:12:05.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.520+0000 7f99ec59e700 1 -- 172.21.15.97:0/3475505223 shutdown_connections 2023-11-24T06:12:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.520+0000 7f99ec59e700 1 -- 172.21.15.97:0/3475505223 >> 172.21.15.97:0/3475505223 conn(0x7f99e40d0b40 msgr2=0x7f99e40d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.520+0000 7f99ec59e700 1 -- 172.21.15.97:0/3475505223 shutdown_connections 2023-11-24T06:12:05.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.520+0000 7f99ec59e700 1 -- 172.21.15.97:0/3475505223 wait complete. 2023-11-24T06:12:05.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.521+0000 7f99ec59e700 1 Processor -- start 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.521+0000 7f99ec59e700 1 -- start start 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99ec59e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004d90 0x7f99e41453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99ec59e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e40cce90 0x7f99e415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99ec59e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40c7f40 0x7f99e40c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99ec59e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40c5e10 con 0x7f99e40cce90 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99ec59e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40c8930 con 0x7f99d0004d90 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99ec59e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e4145930 con 0x7f99e40c7f40 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99e9b39700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e40cce90 0x7f99e415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99e9b39700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e40cce90 0x7f99e415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51028/0 (socket says 172.21.15.97:51028) 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99e9b39700 1 -- 172.21.15.97:0/3229337086 learned_addr learned my addr 172.21.15.97:0/3229337086 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:05.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99ea33a700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004d90 0x7f99e41453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99eab3b700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40c7f40 0x7f99e40c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99e9b39700 1 -- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40c7f40 msgr2=0x7f99e40c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99e9b39700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40c7f40 0x7f99e40c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99e9b39700 1 -- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004d90 msgr2=0x7f99e41453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99e9b39700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004d90 0x7f99e41453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99e9b39700 1 -- 172.21.15.97:0/3229337086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e40c5ac0 con 0x7f99e40cce90 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.522+0000 7f99eab3b700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40c7f40 0x7f99e40c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.523+0000 7f99e9b39700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e40cce90 0x7f99e415ef40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f99e0011720 tx=0x7f99e0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.523+0000 7f99daffd700 1 -- 172.21.15.97:0/3229337086 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e000c070 con 0x7f99e40cce90 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.523+0000 7f99daffd700 1 -- 172.21.15.97:0/3229337086 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0015870 con 0x7f99e40cce90 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.523+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e415f840 con 0x7f99e40cce90 2023-11-24T06:12:05.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.523+0000 7f99daffd700 1 -- 172.21.15.97:0/3229337086 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0015c20 con 0x7f99e40cce90 2023-11-24T06:12:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.523+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e415fe00 con 0x7f99e40cce90 2023-11-24T06:12:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.524+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e40c5f50 con 0x7f99e40cce90 2023-11-24T06:12:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.524+0000 7f99daffd700 1 -- 172.21.15.97:0/3229337086 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f99e00143e0 con 0x7f99e40cce90 2023-11-24T06:12:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.525+0000 7f99daffd700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99c806b4c0 0x7f99c806d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.525+0000 7f99daffd700 1 -- 172.21.15.97:0/3229337086 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f99e0089e40 con 0x7f99e40cce90 2023-11-24T06:12:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.525+0000 7f99ea33a700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99c806b4c0 0x7f99c806d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.525+0000 7f99ea33a700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99c806b4c0 0x7f99c806d980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f99d40011e0 tx=0x7f99d4001170 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:05.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.529+0000 7f99daffd700 1 -- 172.21.15.97:0/3229337086 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99e40c5f50 con 0x7f99e40cce90 2023-11-24T06:12:05.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:05.720+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f99e40c5f50 con 0x7f99e40cce90 2023-11-24T06:12:06.568 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.564+0000 7f99daffd700 1 -- 172.21.15.97:0/3229337086 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f99e40c5f50 con 0x7f99e40cce90 2023-11-24T06:12:06.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.567+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99c806b4c0 msgr2=0x7f99c806d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:06.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.567+0000 7f99ec59e700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99c806b4c0 0x7f99c806d980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f99d40011e0 tx=0x7f99d4001170 comp rx=0 tx=0).stop 2023-11-24T06:12:06.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.567+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e40cce90 msgr2=0x7f99e415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:06.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.567+0000 7f99ec59e700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e40cce90 0x7f99e415ef40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f99e0011720 tx=0x7f99e0011ae0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.572 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.568+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 shutdown_connections 2023-11-24T06:12:06.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.568+0000 7f99ec59e700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99c806b4c0 0x7f99c806d980 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.568+0000 7f99ec59e700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f99e40c7f40 0x7f99e40c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.568+0000 7f99ec59e700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99d0004d90 0x7f99e41453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.568+0000 7f99ec59e700 1 --2- 172.21.15.97:0/3229337086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e40cce90 0x7f99e415ef40 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.568+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 >> 172.21.15.97:0/3229337086 conn(0x7f99e40d0b40 msgr2=0x7f99e4074370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:06.573 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.569+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 shutdown_connections 2023-11-24T06:12:06.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.569+0000 7f99ec59e700 1 -- 172.21.15.97:0/3229337086 wait complete. 2023-11-24T06:12:06.574 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2023-11-24T06:12:06.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:12:06.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.712+0000 7f42c397a700 1 Processor -- start 2023-11-24T06:12:06.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.712+0000 7f42c397a700 1 -- start start 2023-11-24T06:12:06.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c397a700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f42bc06c000 0x7f42bc06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c397a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 0x7f42bc06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c397a700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc0b3fb0 con 0x7f42bc15e3e0 2023-11-24T06:12:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c397a700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc0b56d0 con 0x7f42bc06c000 2023-11-24T06:12:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c397a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc06c920 con 0x7f42bc0b5ee0 2023-11-24T06:12:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c0f15700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f42bc15e3e0 0x7f42bc0b4fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43486/0 (socket says 172.21.15.97:43486) 2023-11-24T06:12:06.720 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c0f15700 1 -- 172.21.15.97:0/2705144071 learned_addr learned my addr 172.21.15.97:0/2705144071 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c1716700 1 --2- 172.21.15.97:0/2705144071 >> v2:172.21.15.183:3300/0 conn(0x7f42bc06c000 0x7f42bc06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42c1f17700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 0x7f42bc06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519661046 0 0) 0x7f42bc0b3fb0 con 0x7f42bc15e3e0 2023-11-24T06:12:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a80035b0 con 0x7f42bc15e3e0 2023-11-24T06:12:06.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1205241126 0 0) 0x7f42a80035b0 con 0x7f42bc15e3e0 2023-11-24T06:12:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 msgr2=0x7f42bc06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 0x7f42bc06e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 >> v2:172.21.15.183:3300/0 conn(0x7f42bc06c000 msgr2=0x7f42bc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 --2- 172.21.15.97:0/2705144071 >> v2:172.21.15.183:3300/0 conn(0x7f42bc06c000 0x7f42bc06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc0b4200 con 0x7f42bc15e3e0 2023-11-24T06:12:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.713+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f42b8001670 con 0x7f42bc15e3e0 2023-11-24T06:12:06.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 >> v1:172.21.15.97:6789/0 conn(0x7f42bc15e3e0 legacy=0x7f42bc0b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a8003a00 0x7f42a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8004850 0x7f42a8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 0x7f42a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0b56d0 con 0x7f42a8003a00 2023-11-24T06:12:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc06c920 con 0x7f42a8004850 2023-11-24T06:12:06.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a800ba10 con 0x7f42bc0b5ee0 2023-11-24T06:12:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1716700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a8003a00 0x7f42a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c0f15700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8004850 0x7f42a8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1f17700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 0x7f42a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1716700 1 -- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 msgr2=0x7f42a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1716700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 0x7f42a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1716700 1 -- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8004850 msgr2=0x7f42a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:06.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1716700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8004850 0x7f42a8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1716700 1 -- 172.21.15.97:0/2705144071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80035b0 con 0x7f42a8003a00 2023-11-24T06:12:06.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1f17700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc0b5ee0 0x7f42a8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:06.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42c1716700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a8003a00 0x7f42a8003e20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009b90 tx=0x7f42a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c1fa118d0ba2b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:06.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00e6c0 con 0x7f42a8003a00 2023-11-24T06:12:06.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac00e820 con 0x7f42a8003a00 2023-11-24T06:12:06.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.714+0000 7f42b3fff700 1 -- 172.21.15.97:0/2705144071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00eb90 con 0x7f42a8003a00 2023-11-24T06:12:06.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.715+0000 7f42c397a700 1 -- 172.21.15.97:0/2705144071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a8003a00 msgr2=0x7f42a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:06.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.715+0000 7f42c397a700 1 --2- 172.21.15.97:0/2705144071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42a8003a00 0x7f42a8003e20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009b90 tx=0x7f42a8005880 comp rx=0 tx=0).stop 2023-11-24T06:12:06.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.715+0000 7f42c1f17700 1 -- 172.21.15.97:0/2705144071 reap_dead start 2023-11-24T06:12:06.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.715+0000 7f42c397a700 1 -- 172.21.15.97:0/2705144071 shutdown_connections 2023-11-24T06:12:06.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.715+0000 7f42c397a700 1 -- 172.21.15.97:0/2705144071 >> 172.21.15.97:0/2705144071 conn(0x7f42bc071ac0 msgr2=0x7f42bc071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.715+0000 7f42c397a700 1 -- 172.21.15.97:0/2705144071 shutdown_connections 2023-11-24T06:12:06.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.715+0000 7f42c397a700 1 -- 172.21.15.97:0/2705144071 wait complete. 2023-11-24T06:12:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.716+0000 7f42c397a700 1 Processor -- start 2023-11-24T06:12:06.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.717+0000 7f42c397a700 1 -- start start 2023-11-24T06:12:06.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.717+0000 7f42c397a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8003a00 0x7f42bc1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.717+0000 7f42c397a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42bc156d30 0x7f42bc1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.717+0000 7f42c397a700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc15e9b0 0x7f42bc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.717+0000 7f42c397a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc1576f0 con 0x7f42bc156d30 2023-11-24T06:12:06.730 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.717+0000 7f42c397a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc160780 con 0x7f42a8003a00 2023-11-24T06:12:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.717+0000 7f42c397a700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc154810 con 0x7f42bc15e9b0 2023-11-24T06:12:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42bc156d30 0x7f42bc1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42bc156d30 0x7f42bc1571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51048/0 (socket says 172.21.15.97:51048) 2023-11-24T06:12:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 -- 172.21.15.97:0/3952933703 learned_addr learned my addr 172.21.15.97:0/3952933703 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c1716700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8003a00 0x7f42bc1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.731 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c1f17700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc15e9b0 0x7f42bc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 -- 172.21.15.97:0/3952933703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc15e9b0 msgr2=0x7f42bc15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:06.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc15e9b0 0x7f42bc15ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 -- 172.21.15.97:0/3952933703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8003a00 msgr2=0x7f42bc1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:06.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8003a00 0x7f42bc1542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:06.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 -- 172.21.15.97:0/3952933703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc15f6e0 con 0x7f42bc156d30 2023-11-24T06:12:06.732 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c0f15700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42bc156d30 0x7f42bc1571b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f42b8007090 tx=0x7f42b8007450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42b1ffb700 1 -- 172.21.15.97:0/3952933703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b800f5e0 con 0x7f42bc156d30 2023-11-24T06:12:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42b1ffb700 1 -- 172.21.15.97:0/3952933703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b800f760 con 0x7f42bc156d30 2023-11-24T06:12:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc15f9c0 con 0x7f42bc156d30 2023-11-24T06:12:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42b1ffb700 1 -- 172.21.15.97:0/3952933703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b800fb10 con 0x7f42bc156d30 2023-11-24T06:12:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.718+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc15ff80 con 0x7f42bc156d30 2023-11-24T06:12:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.719+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc068f10 con 0x7f42bc156d30 2023-11-24T06:12:06.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.720+0000 7f42b1ffb700 1 -- 172.21.15.97:0/3952933703 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f42b800fc70 con 0x7f42bc156d30 2023-11-24T06:12:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.720+0000 7f42b1ffb700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42a006b470 0x7f42a006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.721+0000 7f42b1ffb700 1 -- 172.21.15.97:0/3952933703 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f42b8089df0 con 0x7f42bc156d30 2023-11-24T06:12:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.721+0000 7f42c1716700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42a006b470 0x7f42a006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.721+0000 7f42c1716700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42a006b470 0x7f42a006d930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009b30 tx=0x7f42ac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:06.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.725+0000 7f42b1ffb700 1 -- 172.21.15.97:0/3952933703 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42b8054840 con 0x7f42bc156d30 2023-11-24T06:12:06.918 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:06.915+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f42bc0adcf0 con 0x7f42bc156d30 2023-11-24T06:12:07.570 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.567+0000 7f42b1ffb700 1 -- 172.21.15.97:0/3952933703 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f42b8057e60 con 0x7f42bc156d30 2023-11-24T06:12:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.570+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42a006b470 msgr2=0x7f42a006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.570+0000 7f42c397a700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42a006b470 0x7f42a006d930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009b30 tx=0x7f42ac008030 comp rx=0 tx=0).stop 2023-11-24T06:12:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.570+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42bc156d30 msgr2=0x7f42bc1571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.574 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.570+0000 7f42c397a700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42bc156d30 0x7f42bc1571b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f42b8007090 tx=0x7f42b8007450 comp rx=0 tx=0).stop 2023-11-24T06:12:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.571+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 shutdown_connections 2023-11-24T06:12:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.571+0000 7f42c397a700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f42a006b470 0x7f42a006d930 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.571+0000 7f42c397a700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f42bc15e9b0 0x7f42bc15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.575 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.571+0000 7f42c397a700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42a8003a00 0x7f42bc1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.571+0000 7f42c397a700 1 --2- 172.21.15.97:0/3952933703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42bc156d30 0x7f42bc1571b0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.571+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 >> 172.21.15.97:0/3952933703 conn(0x7f42bc071ac0 msgr2=0x7f42bc0b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:07.576 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.571+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 shutdown_connections 2023-11-24T06:12:07.577 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.571+0000 7f42c397a700 1 -- 172.21.15.97:0/3952933703 wait complete. 2023-11-24T06:12:07.579 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2023-11-24T06:12:07.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:12:07.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.719+0000 7f95eca24700 1 Processor -- start 2023-11-24T06:12:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.719+0000 7f95eca24700 1 -- start start 2023-11-24T06:12:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.720+0000 7f95eca24700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95e80d2a20 0x7f95e8168ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.720+0000 7f95eca24700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e80cf590 con 0x7f95e815e1c0 2023-11-24T06:12:07.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.720+0000 7f95eca24700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e80d2eb0 con 0x7f95e806aff0 2023-11-24T06:12:07.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.720+0000 7f95eca24700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e80d3190 con 0x7f95e80d2a20 2023-11-24T06:12:07.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.720+0000 7f95e6d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f95e80d2a20 0x7f95e8168ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.720+0000 7f95e5d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f95e815e1c0 0x7f95e80cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43506/0 (socket says 172.21.15.97:43506) 2023-11-24T06:12:07.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.720+0000 7f95e5d9b700 1 -- 172.21.15.97:0/645752065 learned_addr learned my addr 172.21.15.97:0/645752065 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:07.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.721+0000 7f95e6d9d700 1 -- 172.21.15.97:0/645752065 >> v1:172.21.15.183:6789/0 conn(0x7f95e806aff0 legacy=0x7f95e806b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:07.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.721+0000 7f95e6d9d700 1 -- 172.21.15.97:0/645752065 >> v1:172.21.15.97:6789/0 conn(0x7f95e815e1c0 legacy=0x7f95e80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.721+0000 7f95e6d9d700 1 -- 172.21.15.97:0/645752065 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e80d3470 con 0x7f95e80d2a20 2023-11-24T06:12:07.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.721+0000 7f95e6d9d700 1 --2- 172.21.15.97:0/645752065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e80d2a20 0x7f95e8168ae0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009d90 tx=0x7f95e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98207e2d99f8293d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.721+0000 7f95e559a700 1 -- 172.21.15.97:0/645752065 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e000e070 con 0x7f95e80d2a20 2023-11-24T06:12:07.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.721+0000 7f95e559a700 1 -- 172.21.15.97:0/645752065 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e000bb90 con 0x7f95e80d2a20 2023-11-24T06:12:07.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.721+0000 7f95e559a700 1 -- 172.21.15.97:0/645752065 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0003a20 con 0x7f95e80d2a20 2023-11-24T06:12:07.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.722+0000 7f95eca24700 1 -- 172.21.15.97:0/645752065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e80d2a20 msgr2=0x7f95e8168ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.722+0000 7f95eca24700 1 --2- 172.21.15.97:0/645752065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e80d2a20 0x7f95e8168ae0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009d90 tx=0x7f95e000b330 comp rx=0 tx=0).stop 2023-11-24T06:12:07.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.722+0000 7f95eca24700 1 -- 172.21.15.97:0/645752065 shutdown_connections 2023-11-24T06:12:07.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.722+0000 7f95eca24700 1 --2- 172.21.15.97:0/645752065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e80d2a20 0x7f95e8168ae0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.722+0000 7f95eca24700 1 -- 172.21.15.97:0/645752065 >> 172.21.15.97:0/645752065 conn(0x7f95e80d3e40 msgr2=0x7f95e80d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:07.728 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.722+0000 7f95eca24700 1 -- 172.21.15.97:0/645752065 shutdown_connections 2023-11-24T06:12:07.729 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.722+0000 7f95eca24700 1 -- 172.21.15.97:0/645752065 wait complete. 2023-11-24T06:12:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.724+0000 7f95eca24700 1 Processor -- start 2023-11-24T06:12:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.724+0000 7f95eca24700 1 -- start start 2023-11-24T06:12:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95eca24700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95e806aff0 0x7f95e81e7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.734 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95eca24700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e815e1c0 0x7f95e80cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.736 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95eca24700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8159930 0x7f95e8159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95eca24700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e815a300 con 0x7f95e815e1c0 2023-11-24T06:12:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95eca24700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e81e6590 con 0x7f95e8159930 2023-11-24T06:12:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95eca24700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e81e6870 con 0x7f95e806aff0 2023-11-24T06:12:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95e5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e815e1c0 0x7f95e80cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95e5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e815e1c0 0x7f95e80cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51060/0 (socket says 172.21.15.97:51060) 2023-11-24T06:12:07.738 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.725+0000 7f95e5d9b700 1 -- 172.21.15.97:0/3859628230 learned_addr learned my addr 172.21.15.97:0/3859628230 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e659c700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95e806aff0 0x7f95e81e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e6d9d700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8159930 0x7f95e8159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e5d9b700 1 -- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95e806aff0 msgr2=0x7f95e81e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e5d9b700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95e806aff0 0x7f95e81e7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e5d9b700 1 -- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8159930 msgr2=0x7f95e8159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e5d9b700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8159930 0x7f95e8159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e5d9b700 1 -- 172.21.15.97:0/3859628230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0009a70 con 0x7f95e815e1c0 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e659c700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95e806aff0 0x7f95e81e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95e5d9b700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e815e1c0 0x7f95e80cd6b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c430 tx=0x7f95dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95d6ffd700 1 -- 172.21.15.97:0/3859628230 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc005d70 con 0x7f95e815e1c0 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95d6ffd700 1 -- 172.21.15.97:0/3859628230 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc0070d0 con 0x7f95e815e1c0 2023-11-24T06:12:07.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e81e6b50 con 0x7f95e815e1c0 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95d6ffd700 1 -- 172.21.15.97:0/3859628230 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc007460 con 0x7f95e815e1c0 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.726+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e81e7890 con 0x7f95e815e1c0 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.727+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e8068fb0 con 0x7f95e815e1c0 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.728+0000 7f95d6ffd700 1 -- 172.21.15.97:0/3859628230 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f95dc0075c0 con 0x7f95e815e1c0 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.728+0000 7f95d6ffd700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f95c806b4a0 0x7f95c806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.728+0000 7f95d6ffd700 1 -- 172.21.15.97:0/3859628230 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f95dc0893c0 con 0x7f95e815e1c0 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.728+0000 7f95e659c700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f95c806b4a0 0x7f95c806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.728+0000 7f95e659c700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f95c806b4a0 0x7f95c806d960 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f95d8001750 tx=0x7f95d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:07.740 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.733+0000 7f95d6ffd700 1 -- 172.21.15.97:0/3859628230 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95dc053cd0 con 0x7f95e815e1c0 2023-11-24T06:12:07.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:07.975+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f95e815d8e0 con 0x7f95e815e1c0 2023-11-24T06:12:08.592 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.588+0000 7f95d6ffd700 1 -- 172.21.15.97:0/3859628230 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v51) v1 ==== 174+0+0 (secure 0 0 0) 0x7f95dc0572f0 con 0x7f95e815e1c0 2023-11-24T06:12:08.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f95c806b4a0 msgr2=0x7f95c806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f95c806b4a0 0x7f95c806d960 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f95d8001750 tx=0x7f95d8009040 comp rx=0 tx=0).stop 2023-11-24T06:12:08.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e815e1c0 msgr2=0x7f95e80cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e815e1c0 0x7f95e80cd6b0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c430 tx=0x7f95dc00c7f0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 shutdown_connections 2023-11-24T06:12:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95e8159930 0x7f95e8159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f95c806b4a0 0x7f95c806d960 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f95e806aff0 0x7f95e81e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 --2- 172.21.15.97:0/3859628230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95e815e1c0 0x7f95e80cd6b0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.597+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 >> 172.21.15.97:0/3859628230 conn(0x7f95e80d3e40 msgr2=0x7f95e815e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.598+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 shutdown_connections 2023-11-24T06:12:08.602 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.598+0000 7f95eca24700 1 -- 172.21.15.97:0/3859628230 wait complete. 2023-11-24T06:12:08.610 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 12 and data pool 13 2023-11-24T06:12:08.622 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.740+0000 7fd59eb64700 1 Processor -- start 2023-11-24T06:12:08.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.740+0000 7fd59eb64700 1 -- start start 2023-11-24T06:12:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.741+0000 7fd59eb64700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd59806c710 0x7fd5980b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.741+0000 7fd59eb64700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd5980af4d0 0x7fd5980afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.741+0000 7fd59eb64700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5980ae430 con 0x7fd59815e080 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.741+0000 7fd59eb64700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980af0b0 con 0x7fd59806c710 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.741+0000 7fd59eb64700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980b05a0 con 0x7fd5980af4d0 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.741+0000 7fd597fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd59806c710 0x7fd5980b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.741+0000 7fd59c900700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd59815e080 0x7fd59806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43518/0 (socket says 172.21.15.97:43518) 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.741+0000 7fd59c900700 1 -- 172.21.15.97:0/2377615641 learned_addr learned my addr 172.21.15.97:0/2377615641 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd59d101700 1 --2- 172.21.15.97:0/2377615641 >> v2:172.21.15.183:3300/0 conn(0x7fd5980af4d0 0x7fd5980afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd597fff700 1 -- 172.21.15.97:0/2377615641 >> v2:172.21.15.183:3300/0 conn(0x7fd5980af4d0 msgr2=0x7fd5980afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd597fff700 1 --2- 172.21.15.97:0/2377615641 >> v2:172.21.15.183:3300/0 conn(0x7fd5980af4d0 0x7fd5980afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd597fff700 1 -- 172.21.15.97:0/2377615641 >> v1:172.21.15.97:6789/0 conn(0x7fd59815e080 legacy=0x7fd59806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd5977fe700 1 -- 172.21.15.97:0/2377615641 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585466621 0 0) 0x7fd5980ae430 con 0x7fd59815e080 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd597fff700 1 -- 172.21.15.97:0/2377615641 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980ae710 con 0x7fd59806c710 2023-11-24T06:12:08.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd597fff700 1 --2- 172.21.15.97:0/2377615641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd5980b5050 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd59809eea0 tx=0x7fd584009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df9788852cb115b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd5977fe700 1 -- 172.21.15.97:0/2377615641 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584011830 con 0x7fd59806c710 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd5977fe700 1 -- 172.21.15.97:0/2377615641 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd584011990 con 0x7fd59806c710 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd5977fe700 1 -- 172.21.15.97:0/2377615641 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584011d20 con 0x7fd59806c710 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd59eb64700 1 -- 172.21.15.97:0/2377615641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 msgr2=0x7fd5980b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd59eb64700 1 --2- 172.21.15.97:0/2377615641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd5980b5050 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd59809eea0 tx=0x7fd584009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd59eb64700 1 -- 172.21.15.97:0/2377615641 shutdown_connections 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd59eb64700 1 --2- 172.21.15.97:0/2377615641 >> v2:172.21.15.183:3300/0 conn(0x7fd5980af4d0 0x7fd5980afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd59eb64700 1 --2- 172.21.15.97:0/2377615641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd5980b5050 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.742+0000 7fd59eb64700 1 -- 172.21.15.97:0/2377615641 >> 172.21.15.97:0/2377615641 conn(0x7fd598071b90 msgr2=0x7fd598071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.743+0000 7fd59eb64700 1 -- 172.21.15.97:0/2377615641 shutdown_connections 2023-11-24T06:12:08.748 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.743+0000 7fd59eb64700 1 -- 172.21.15.97:0/2377615641 wait complete. 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.744+0000 7fd59eb64700 1 Processor -- start 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.744+0000 7fd59eb64700 1 -- start start 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59eb64700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd598152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59eb64700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5980af4d0 0x7fd5981544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59eb64700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd59815e080 0x7fd5981549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59eb64700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981f5670 con 0x7fd59806c710 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59eb64700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981f4a60 con 0x7fd5980af4d0 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59eb64700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981f4d40 con 0x7fd59815e080 2023-11-24T06:12:08.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59c900700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd598152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59c900700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd598152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51072/0 (socket says 172.21.15.97:51072) 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59c900700 1 -- 172.21.15.97:0/4166383540 learned_addr learned my addr 172.21.15.97:0/4166383540 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd59d101700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd59815e080 0x7fd5981549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.745+0000 7fd597fff700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5980af4d0 0x7fd5981544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd59c900700 1 -- 172.21.15.97:0/4166383540 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd59815e080 msgr2=0x7fd5981549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd59c900700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd59815e080 0x7fd5981549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd59c900700 1 -- 172.21.15.97:0/4166383540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5980af4d0 msgr2=0x7fd5981544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd59c900700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5980af4d0 0x7fd5981544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd59c900700 1 -- 172.21.15.97:0/4166383540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840094a0 con 0x7fd59806c710 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd59c900700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd598152a60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00c550 tx=0x7fd58c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd5957fa700 1 -- 172.21.15.97:0/4166383540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c004c80 con 0x7fd59806c710 2023-11-24T06:12:08.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd5957fa700 1 -- 172.21.15.97:0/4166383540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c004e00 con 0x7fd59806c710 2023-11-24T06:12:08.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5981f5020 con 0x7fd59806c710 2023-11-24T06:12:08.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd5957fa700 1 -- 172.21.15.97:0/4166383540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c007310 con 0x7fd59806c710 2023-11-24T06:12:08.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.746+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5981570a0 con 0x7fd59806c710 2023-11-24T06:12:08.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.747+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5980753f0 con 0x7fd59806c710 2023-11-24T06:12:08.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.747+0000 7fd5957fa700 1 -- 172.21.15.97:0/4166383540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd58c007470 con 0x7fd59806c710 2023-11-24T06:12:08.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.748+0000 7fd5957fa700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd57c06b470 0x7fd57c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:08.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.748+0000 7fd5957fa700 1 -- 172.21.15.97:0/4166383540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd58c089df0 con 0x7fd59806c710 2023-11-24T06:12:08.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.748+0000 7fd597fff700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd57c06b470 0x7fd57c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:08.754 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.748+0000 7fd597fff700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd57c06b470 0x7fd57c06d930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd5840097c0 tx=0x7fd584008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:08.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.752+0000 7fd5957fa700 1 -- 172.21.15.97:0/4166383540 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd58c0545d0 con 0x7fd59806c710 2023-11-24T06:12:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.943+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd598072cc0 con 0x7fd59806c710 2023-11-24T06:12:08.948 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.944+0000 7fd5957fa700 1 -- 172.21.15.97:0/4166383540 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16628 (secure 0 0 0) 0x7fd58c057bf0 con 0x7fd59806c710 2023-11-24T06:12:08.948 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:08.948 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:07.978023+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:05.739467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:06.934719+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.946+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd57c06b470 msgr2=0x7fd57c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.946+0000 7fd59eb64700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd57c06b470 0x7fd57c06d930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd5840097c0 tx=0x7fd584008030 comp rx=0 tx=0).stop 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.946+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 msgr2=0x7fd598152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.946+0000 7fd59eb64700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd598152a60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00c550 tx=0x7fd58c00c910 comp rx=0 tx=0).stop 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.947+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 shutdown_connections 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.947+0000 7fd59eb64700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd57c06b470 0x7fd57c06d930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.947+0000 7fd59eb64700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd59815e080 0x7fd5981549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.947+0000 7fd59eb64700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd59806c710 0x7fd598152a60 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.947+0000 7fd59eb64700 1 --2- 172.21.15.97:0/4166383540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5980af4d0 0x7fd5981544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.947+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 >> 172.21.15.97:0/4166383540 conn(0x7fd598071b90 msgr2=0x7fd59806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.947+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 shutdown_connections 2023-11-24T06:12:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:08.947+0000 7fd59eb64700 1 -- 172.21.15.97:0/4166383540 wait complete. 2023-11-24T06:12:08.962 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.078+0000 7fea03f2b700 1 Processor -- start 2023-11-24T06:12:09.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.078+0000 7fea03f2b700 1 -- start start 2023-11-24T06:12:09.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.079+0000 7fea03f2b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9fc0b49a0 0x7fe9fc0d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.079+0000 7fea03f2b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc15c4b0 con 0x7fe9fc06c710 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.079+0000 7fea03f2b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc15c790 con 0x7fe9fc152cc0 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.079+0000 7fea03f2b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9fc15ca70 con 0x7fe9fc0b49a0 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.079+0000 7fea024c8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9fc0b49a0 0x7fe9fc0d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.079+0000 7fea024c8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9fc0b49a0 0x7fe9fc0d0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51084/0 (socket says 172.21.15.97:51084) 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.079+0000 7fea024c8700 1 -- 172.21.15.97:0/1407961444 learned_addr learned my addr 172.21.15.97:0/1407961444 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.079+0000 7fea014c6700 1 --1- 172.21.15.97:0/1407961444 >> v1:172.21.15.97:6789/0 conn(0x7fe9fc06c710 0x7fe9fc0cff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43534/0 (socket says 172.21.15.97:43534) 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea024c8700 1 -- 172.21.15.97:0/1407961444 >> v1:172.21.15.183:6789/0 conn(0x7fe9fc152cc0 legacy=0x7fe9fc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea024c8700 1 -- 172.21.15.97:0/1407961444 >> v1:172.21.15.97:6789/0 conn(0x7fe9fc06c710 legacy=0x7fe9fc0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea024c8700 1 -- 172.21.15.97:0/1407961444 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc15cd50 con 0x7fe9fc0b49a0 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea024c8700 1 --2- 172.21.15.97:0/1407961444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc0b49a0 0x7fe9fc0d0630 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009e50 tx=0x7fe9f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d068364592020a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea00cc5700 1 -- 172.21.15.97:0/1407961444 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400e070 con 0x7fe9fc0b49a0 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea00cc5700 1 -- 172.21.15.97:0/1407961444 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4003a20 con 0x7fe9fc0b49a0 2023-11-24T06:12:09.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea00cc5700 1 -- 172.21.15.97:0/1407961444 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4003dd0 con 0x7fe9fc0b49a0 2023-11-24T06:12:09.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea03f2b700 1 -- 172.21.15.97:0/1407961444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc0b49a0 msgr2=0x7fe9fc0d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.087 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea03f2b700 1 --2- 172.21.15.97:0/1407961444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc0b49a0 0x7fe9fc0d0630 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009e50 tx=0x7fe9f400b330 comp rx=0 tx=0).stop 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea03f2b700 1 -- 172.21.15.97:0/1407961444 shutdown_connections 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea03f2b700 1 --2- 172.21.15.97:0/1407961444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc0b49a0 0x7fe9fc0d0630 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.080+0000 7fea03f2b700 1 -- 172.21.15.97:0/1407961444 >> 172.21.15.97:0/1407961444 conn(0x7fe9fc071b90 msgr2=0x7fe9fc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.081+0000 7fea03f2b700 1 -- 172.21.15.97:0/1407961444 shutdown_connections 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.081+0000 7fea03f2b700 1 -- 172.21.15.97:0/1407961444 wait complete. 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea03f2b700 1 Processor -- start 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea03f2b700 1 -- start start 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea03f2b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc06c710 0x7fe9fc138b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea03f2b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9fc0b49a0 0x7fe9fc1e7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea03f2b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc152cc0 0x7fe9fc14ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea03f2b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc14c830 con 0x7fe9fc06c710 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea03f2b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1e68a0 con 0x7fe9fc152cc0 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea03f2b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1e6b80 con 0x7fe9fc0b49a0 2023-11-24T06:12:09.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.082+0000 7fea01cc7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc06c710 0x7fe9fc138b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea01cc7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc06c710 0x7fe9fc138b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51086/0 (socket says 172.21.15.97:51086) 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea01cc7700 1 -- 172.21.15.97:0/4242892992 learned_addr learned my addr 172.21.15.97:0/4242892992 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea01cc7700 1 -- 172.21.15.97:0/4242892992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9fc0b49a0 msgr2=0x7fe9fc1e7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea01cc7700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9fc0b49a0 0x7fe9fc1e7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea01cc7700 1 -- 172.21.15.97:0/4242892992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc152cc0 msgr2=0x7fe9fc14ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea024c8700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc152cc0 0x7fe9fc14ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea01cc7700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc152cc0 0x7fe9fc14ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea01cc7700 1 -- 172.21.15.97:0/4242892992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f4009b30 con 0x7fe9fc06c710 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea01cc7700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc06c710 0x7fe9fc138b20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00b4a0 tx=0x7fe9ec00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fe9f27fc700 1 -- 172.21.15.97:0/4242892992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec011830 con 0x7fe9fc06c710 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fe9f27fc700 1 -- 172.21.15.97:0/4242892992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec011990 con 0x7fe9fc06c710 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc1e6e60 con 0x7fe9fc06c710 2023-11-24T06:12:09.089 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fe9f27fc700 1 -- 172.21.15.97:0/4242892992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec011d40 con 0x7fe9fc06c710 2023-11-24T06:12:09.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.083+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc1e7420 con 0x7fe9fc06c710 2023-11-24T06:12:09.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.084+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc1e75e0 con 0x7fe9fc06c710 2023-11-24T06:12:09.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.084+0000 7fe9f27fc700 1 -- 172.21.15.97:0/4242892992 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe9ec01a400 con 0x7fe9fc06c710 2023-11-24T06:12:09.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.085+0000 7fe9f27fc700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9e006b470 0x7fe9e006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.090 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.085+0000 7fe9f27fc700 1 -- 172.21.15.97:0/4242892992 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe9ec089d20 con 0x7fe9fc06c710 2023-11-24T06:12:09.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.085+0000 7fea014c6700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9e006b470 0x7fe9e006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.091 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.085+0000 7fea014c6700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9e006b470 0x7fe9e006d930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8000f60 tx=0x7fe9f8007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.093 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.089+0000 7fe9f27fc700 1 -- 172.21.15.97:0/4242892992 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9ec0545d0 con 0x7fe9fc06c710 2023-11-24T06:12:09.283 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.280+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9fc0753f0 con 0x7fe9fc06c710 2023-11-24T06:12:09.284 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.281+0000 7fe9f27fc700 1 -- 172.21.15.97:0/4242892992 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16628 (secure 0 0 0) 0x7fe9ec015090 con 0x7fe9fc06c710 2023-11-24T06:12:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:09.285 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":84,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:07.978023+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:05.739467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:06.934719+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9e006b470 msgr2=0x7fe9e006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9e006b470 0x7fe9e006d930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8000f60 tx=0x7fe9f8007a90 comp rx=0 tx=0).stop 2023-11-24T06:12:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc06c710 msgr2=0x7fe9fc138b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc06c710 0x7fe9fc138b20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00b4a0 tx=0x7fe9ec00b860 comp rx=0 tx=0).stop 2023-11-24T06:12:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 shutdown_connections 2023-11-24T06:12:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc152cc0 0x7fe9fc14ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.287 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9fc06c710 0x7fe9fc138b20 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fe9e006b470 0x7fe9e006d930 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 --2- 172.21.15.97:0/4242892992 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe9fc0b49a0 0x7fe9fc1e7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 >> 172.21.15.97:0/4242892992 conn(0x7fe9fc071b90 msgr2=0x7fe9fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 shutdown_connections 2023-11-24T06:12:09.288 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.283+0000 7fea03f2b700 1 -- 172.21.15.97:0/4242892992 wait complete. 2023-11-24T06:12:09.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:12:09.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.419+0000 7fd08e0b8700 1 Processor -- start 2023-11-24T06:12:09.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.419+0000 7fd08e0b8700 1 -- start start 2023-11-24T06:12:09.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.420+0000 7fd08e0b8700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd08815bf50 0x7fd0880ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.420+0000 7fd08e0b8700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd0880ce830 0x7fd0880cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.420+0000 7fd08e0b8700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0880d0140 con 0x7fd0880cf1f0 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.420+0000 7fd08e0b8700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08815c330 con 0x7fd08815bf50 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.420+0000 7fd08e0b8700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0880d03e0 con 0x7fd0880ce830 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.420+0000 7fd0877fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd08815bf50 0x7fd0880ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.420+0000 7fd0877fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd08815bf50 0x7fd0880ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51094/0 (socket says 172.21.15.97:51094) 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.420+0000 7fd0877fe700 1 -- 172.21.15.97:0/2799564644 learned_addr learned my addr 172.21.15.97:0/2799564644 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd086ffd700 1 --2- 172.21.15.97:0/2799564644 >> v2:172.21.15.183:3300/0 conn(0x7fd0880ce830 0x7fd0880cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd0877fe700 1 -- 172.21.15.97:0/2799564644 >> v2:172.21.15.183:3300/0 conn(0x7fd0880ce830 msgr2=0x7fd0880cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd0877fe700 1 --2- 172.21.15.97:0/2799564644 >> v2:172.21.15.183:3300/0 conn(0x7fd0880ce830 0x7fd0880cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd0877fe700 1 -- 172.21.15.97:0/2799564644 >> v1:172.21.15.183:6789/0 conn(0x7fd0880cf1f0 legacy=0x7fd0880c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd0877fe700 1 -- 172.21.15.97:0/2799564644 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880d06c0 con 0x7fd08815bf50 2023-11-24T06:12:09.426 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd0877fe700 1 --2- 172.21.15.97:0/2799564644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0880ce2f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd088072420 tx=0x7fd070009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8599c72a4f6064b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd0867fc700 1 -- 172.21.15.97:0/2799564644 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070004230 con 0x7fd08815bf50 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd0867fc700 1 -- 172.21.15.97:0/2799564644 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd070004390 con 0x7fd08815bf50 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd0867fc700 1 -- 172.21.15.97:0/2799564644 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0700036a0 con 0x7fd08815bf50 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd08e0b8700 1 -- 172.21.15.97:0/2799564644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 msgr2=0x7fd0880ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/2799564644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0880ce2f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd088072420 tx=0x7fd070009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.421+0000 7fd08e0b8700 1 -- 172.21.15.97:0/2799564644 shutdown_connections 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.422+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/2799564644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0880ce2f0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.422+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/2799564644 >> v2:172.21.15.183:3300/0 conn(0x7fd0880ce830 0x7fd0880cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.422+0000 7fd08e0b8700 1 -- 172.21.15.97:0/2799564644 >> 172.21.15.97:0/2799564644 conn(0x7fd0880b3780 msgr2=0x7fd0880b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.422+0000 7fd08e0b8700 1 -- 172.21.15.97:0/2799564644 shutdown_connections 2023-11-24T06:12:09.427 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.422+0000 7fd08e0b8700 1 -- 172.21.15.97:0/2799564644 wait complete. 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.423+0000 7fd08e0b8700 1 Processor -- start 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.423+0000 7fd08e0b8700 1 -- start start 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd08e0b8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0880ce830 0x7fd0880c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd08e0b8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0880cf1f0 0x7fd088153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd08e0b8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0881535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd08e0b8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd088153b00 con 0x7fd08815bf50 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd08e0b8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880b6000 con 0x7fd0880cf1f0 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd08e0b8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0881e6f90 con 0x7fd0880ce830 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0881535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0881535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51102/0 (socket says 172.21.15.97:51102) 2023-11-24T06:12:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 -- 172.21.15.97:0/1312740313 learned_addr learned my addr 172.21.15.97:0/1312740313 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd0877fe700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0880ce830 0x7fd0880c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd086ffd700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0880cf1f0 0x7fd088153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 -- 172.21.15.97:0/1312740313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0880ce830 msgr2=0x7fd0880c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0880ce830 0x7fd0880c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 -- 172.21.15.97:0/1312740313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0880cf1f0 msgr2=0x7fd088153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0880cf1f0 0x7fd088153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 -- 172.21.15.97:0/1312740313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070009550 con 0x7fd08815bf50 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd087fff700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0881535c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd08000cc10 tx=0x7fd08000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd07ffff700 1 -- 172.21.15.97:0/1312740313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080004b90 con 0x7fd08815bf50 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd07ffff700 1 -- 172.21.15.97:0/1312740313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd080004cf0 con 0x7fd08815bf50 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0881e6cb0 con 0x7fd08815bf50 2023-11-24T06:12:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd07ffff700 1 -- 172.21.15.97:0/1312740313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000aae0 con 0x7fd08815bf50 2023-11-24T06:12:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.424+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0880b6580 con 0x7fd08815bf50 2023-11-24T06:12:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.425+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0880b5e60 con 0x7fd08815bf50 2023-11-24T06:12:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.426+0000 7fd07ffff700 1 -- 172.21.15.97:0/1312740313 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd08000ac40 con 0x7fd08815bf50 2023-11-24T06:12:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.426+0000 7fd07ffff700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd06806b470 0x7fd06806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.426+0000 7fd0877fe700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd06806b470 0x7fd06806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.426+0000 7fd07ffff700 1 -- 172.21.15.97:0/1312740313 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd080089900 con 0x7fd08815bf50 2023-11-24T06:12:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.427+0000 7fd0877fe700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd06806b470 0x7fd06806d930 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd070009130 tx=0x7fd070008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:09.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.431+0000 7fd07ffff700 1 -- 172.21.15.97:0/1312740313 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0880b5e60 con 0x7fd08815bf50 2023-11-24T06:12:09.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:09.670+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd0880b5e60 con 0x7fd08815bf50 2023-11-24T06:12:10.612 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.609+0000 7fd07ffff700 1 -- 172.21.15.97:0/1312740313 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd0880b5e60 con 0x7fd08815bf50 2023-11-24T06:12:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd06806b470 msgr2=0x7fd06806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd06806b470 0x7fd06806d930 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd070009130 tx=0x7fd070008030 comp rx=0 tx=0).stop 2023-11-24T06:12:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 msgr2=0x7fd0881535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0881535c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fd08000cc10 tx=0x7fd08000cf20 comp rx=0 tx=0).stop 2023-11-24T06:12:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 shutdown_connections 2023-11-24T06:12:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd08815bf50 0x7fd0881535c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd06806b470 0x7fd06806d930 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd0880ce830 0x7fd0880c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 --2- 172.21.15.97:0/1312740313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0880cf1f0 0x7fd088153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 >> 172.21.15.97:0/1312740313 conn(0x7fd0880b3780 msgr2=0x7fd0880c5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 shutdown_connections 2023-11-24T06:12:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.612+0000 7fd08e0b8700 1 -- 172.21.15.97:0/1312740313 wait complete. 2023-11-24T06:12:10.630 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:12:10.631 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:12:10.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.756+0000 7fd588edf700 1 Processor -- start 2023-11-24T06:12:10.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.756+0000 7fd588edf700 1 -- start start 2023-11-24T06:12:10.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd588edf700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd58415e160 0x7fd5840b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd588edf700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840746a0 0x7fd58406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd588edf700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5840b3c60 con 0x7fd5840afc90 2023-11-24T06:12:10.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd588edf700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5840b4500 con 0x7fd58415e160 2023-11-24T06:12:10.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd588edf700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840b58c0 con 0x7fd5840746a0 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd581d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd5840afc90 0x7fd584073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43558/0 (socket says 172.21.15.97:43558) 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd581d9b700 1 -- 172.21.15.97:0/512310220 learned_addr learned my addr 172.21.15.97:0/512310220 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd58259c700 1 --2- 172.21.15.97:0/512310220 >> v2:172.21.15.183:3300/0 conn(0x7fd58415e160 0x7fd5840b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd582d9d700 1 --2- 172.21.15.97:0/512310220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840746a0 0x7fd58406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd58159a700 1 -- 172.21.15.97:0/512310220 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847397324 0 0) 0x7fd5840b3c60 con 0x7fd5840afc90 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd58159a700 1 -- 172.21.15.97:0/512310220 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd56c0035b0 con 0x7fd5840afc90 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd582d9d700 1 -- 172.21.15.97:0/512310220 >> v2:172.21.15.183:3300/0 conn(0x7fd58415e160 msgr2=0x7fd5840b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd582d9d700 1 --2- 172.21.15.97:0/512310220 >> v2:172.21.15.183:3300/0 conn(0x7fd58415e160 0x7fd5840b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd58159a700 1 -- 172.21.15.97:0/512310220 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4090016902 0 0) 0x7fd56c0035b0 con 0x7fd5840afc90 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd582d9d700 1 -- 172.21.15.97:0/512310220 >> v1:172.21.15.97:6789/0 conn(0x7fd5840afc90 legacy=0x7fd584073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd582d9d700 1 -- 172.21.15.97:0/512310220 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840b0140 con 0x7fd5840746a0 2023-11-24T06:12:10.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.757+0000 7fd58259c700 1 --2- 172.21.15.97:0/512310220 >> v2:172.21.15.183:3300/0 conn(0x7fd58415e160 0x7fd5840b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd582d9d700 1 --2- 172.21.15.97:0/512310220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840746a0 0x7fd58406d930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009d90 tx=0x7fd57c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c746b76ccab3a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd58159a700 1 -- 172.21.15.97:0/512310220 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00e070 con 0x7fd5840746a0 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd58159a700 1 -- 172.21.15.97:0/512310220 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c003a20 con 0x7fd5840746a0 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd58159a700 1 -- 172.21.15.97:0/512310220 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578001670 con 0x7fd5840746a0 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd588edf700 1 -- 172.21.15.97:0/512310220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840746a0 msgr2=0x7fd58406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd588edf700 1 --2- 172.21.15.97:0/512310220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840746a0 0x7fd58406d930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009d90 tx=0x7fd57c00b330 comp rx=0 tx=0).stop 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd588edf700 1 -- 172.21.15.97:0/512310220 shutdown_connections 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd588edf700 1 --2- 172.21.15.97:0/512310220 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840746a0 0x7fd58406d930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd588edf700 1 --2- 172.21.15.97:0/512310220 >> v2:172.21.15.183:3300/0 conn(0x7fd58415e160 0x7fd5840b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd588edf700 1 -- 172.21.15.97:0/512310220 >> 172.21.15.97:0/512310220 conn(0x7fd5840d0bf0 msgr2=0x7fd5840d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd588edf700 1 -- 172.21.15.97:0/512310220 shutdown_connections 2023-11-24T06:12:10.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.758+0000 7fd588edf700 1 -- 172.21.15.97:0/512310220 wait complete. 2023-11-24T06:12:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.760+0000 7fd588edf700 1 Processor -- start 2023-11-24T06:12:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.760+0000 7fd588edf700 1 -- start start 2023-11-24T06:12:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.760+0000 7fd588edf700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5840746a0 0x7fd584152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.760+0000 7fd588edf700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840afc90 0x7fd58406e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.760+0000 7fd588edf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58415e160 0x7fd58406eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.760+0000 7fd588edf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd584157590 con 0x7fd58415e160 2023-11-24T06:12:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.760+0000 7fd588edf700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd584158230 con 0x7fd5840746a0 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.760+0000 7fd588edf700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd584157890 con 0x7fd5840afc90 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58415e160 0x7fd58406eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58415e160 0x7fd58406eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51116/0 (socket says 172.21.15.97:51116) 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 -- 172.21.15.97:0/720905346 learned_addr learned my addr 172.21.15.97:0/720905346 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd581d9b700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840afc90 0x7fd58406e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd58259c700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5840746a0 0x7fd584152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 -- 172.21.15.97:0/720905346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840afc90 msgr2=0x7fd58406e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840afc90 0x7fd58406e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 -- 172.21.15.97:0/720905346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5840746a0 msgr2=0x7fd584152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5840746a0 0x7fd584152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 -- 172.21.15.97:0/720905346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740094a0 con 0x7fd58415e160 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd582d9d700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58415e160 0x7fd58406eec0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd57c004000 tx=0x7fd57c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd572ffd700 1 -- 172.21.15.97:0/720905346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c009410 con 0x7fd58415e160 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd572ffd700 1 -- 172.21.15.97:0/720905346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c004460 con 0x7fd58415e160 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd572ffd700 1 -- 172.21.15.97:0/720905346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c004810 con 0x7fd58415e160 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c009a70 con 0x7fd58415e160 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.761+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd584157e50 con 0x7fd58415e160 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.762+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd584068fb0 con 0x7fd58415e160 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.763+0000 7fd572ffd700 1 -- 172.21.15.97:0/720905346 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd57c012430 con 0x7fd58415e160 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.763+0000 7fd572ffd700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd56406b470 0x7fd56406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.763+0000 7fd572ffd700 1 -- 172.21.15.97:0/720905346 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd57c08bf60 con 0x7fd58415e160 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.763+0000 7fd58259c700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd56406b470 0x7fd56406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.763+0000 7fd58259c700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd56406b470 0x7fd56406d930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd574009cd0 tx=0x7fd574009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:10.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:10.768+0000 7fd572ffd700 1 -- 172.21.15.97:0/720905346 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd57c056700 con 0x7fd58415e160 2023-11-24T06:12:11.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.001+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd58406c140 con 0x7fd58415e160 2023-11-24T06:12:11.614 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.611+0000 7fd572ffd700 1 -- 172.21.15.97:0/720905346 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd57c02a0c0 con 0x7fd58415e160 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd56406b470 msgr2=0x7fd56406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd56406b470 0x7fd56406d930 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd574009cd0 tx=0x7fd574009210 comp rx=0 tx=0).stop 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58415e160 msgr2=0x7fd58406eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58415e160 0x7fd58406eec0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd57c004000 tx=0x7fd57c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 shutdown_connections 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd58415e160 0x7fd58406eec0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5840746a0 0x7fd584152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd56406b470 0x7fd56406d930 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 --2- 172.21.15.97:0/720905346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5840afc90 0x7fd58406e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 >> 172.21.15.97:0/720905346 conn(0x7fd5840d0bf0 msgr2=0x7fd58415d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:11.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 shutdown_connections 2023-11-24T06:12:11.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.614+0000 7fd588edf700 1 -- 172.21.15.97:0/720905346 wait complete. 2023-11-24T06:12:11.633 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:12:11.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:11.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.755+0000 7f0dd962e700 1 Processor -- start 2023-11-24T06:12:11.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.756+0000 7f0dd962e700 1 -- start start 2023-11-24T06:12:11.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.756+0000 7f0dd962e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0dd415deb0 0x7f0dd40afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.756+0000 7f0dd962e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f0dd406ed40 0x7f0dd40b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.756+0000 7f0dd962e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd40b5e70 con 0x7f0dd4073f90 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.756+0000 7f0dd962e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd40affa0 con 0x7f0dd415deb0 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.756+0000 7f0dd962e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd406dca0 con 0x7f0dd406ed40 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd2ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0dd415deb0 0x7f0dd40afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd2ffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0dd415deb0 0x7f0dd40afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51120/0 (socket says 172.21.15.97:51120) 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd2ffd700 1 -- 172.21.15.97:0/672352851 learned_addr learned my addr 172.21.15.97:0/672352851 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd27fc700 1 --2- 172.21.15.97:0/672352851 >> v2:172.21.15.183:3300/0 conn(0x7f0dd406ed40 0x7f0dd40b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd2ffd700 1 -- 172.21.15.97:0/672352851 >> v2:172.21.15.183:3300/0 conn(0x7f0dd406ed40 msgr2=0x7f0dd40b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd2ffd700 1 --2- 172.21.15.97:0/672352851 >> v2:172.21.15.183:3300/0 conn(0x7f0dd406ed40 0x7f0dd40b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd2ffd700 1 -- 172.21.15.97:0/672352851 >> v1:172.21.15.183:6789/0 conn(0x7f0dd4073f90 legacy=0x7f0dd40743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd2ffd700 1 -- 172.21.15.97:0/672352851 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd406df80 con 0x7f0dd415deb0 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd2ffd700 1 --2- 172.21.15.97:0/672352851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd415deb0 0x7f0dd40afa60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40d1a40 tx=0x7f0dc4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bb1fa4bf49f235 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd1ffb700 1 -- 172.21.15.97:0/672352851 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4011830 con 0x7f0dd415deb0 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd1ffb700 1 -- 172.21.15.97:0/672352851 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc4011990 con 0x7f0dd415deb0 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.757+0000 7f0dd1ffb700 1 -- 172.21.15.97:0/672352851 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4011d20 con 0x7f0dd415deb0 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.758+0000 7f0dd962e700 1 -- 172.21.15.97:0/672352851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd415deb0 msgr2=0x7f0dd40afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.758+0000 7f0dd962e700 1 --2- 172.21.15.97:0/672352851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd415deb0 0x7f0dd40afa60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40d1a40 tx=0x7f0dc4009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.758+0000 7f0dd962e700 1 -- 172.21.15.97:0/672352851 shutdown_connections 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.758+0000 7f0dd962e700 1 --2- 172.21.15.97:0/672352851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd415deb0 0x7f0dd40afa60 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.758+0000 7f0dd962e700 1 --2- 172.21.15.97:0/672352851 >> v2:172.21.15.183:3300/0 conn(0x7f0dd406ed40 0x7f0dd40b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.758+0000 7f0dd962e700 1 -- 172.21.15.97:0/672352851 >> 172.21.15.97:0/672352851 conn(0x7f0dd40d0ba0 msgr2=0x7f0dd40d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.758+0000 7f0dd962e700 1 -- 172.21.15.97:0/672352851 shutdown_connections 2023-11-24T06:12:11.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.758+0000 7f0dd962e700 1 -- 172.21.15.97:0/672352851 wait complete. 2023-11-24T06:12:11.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.759+0000 7f0dd962e700 1 Processor -- start 2023-11-24T06:12:11.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.759+0000 7f0dd962e700 1 -- start start 2023-11-24T06:12:11.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd962e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd406ed40 0x7f0dd4160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd962e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd4073f90 0x7f0dd4150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd962e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0dd415deb0 0x7f0dd40b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd962e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4150450 con 0x7f0dd406ed40 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd962e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd41526b0 con 0x7f0dd4073f90 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd962e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd41523d0 con 0x7f0dd415deb0 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd406ed40 0x7f0dd4160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd406ed40 0x7f0dd4160ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51126/0 (socket says 172.21.15.97:51126) 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 -- 172.21.15.97:0/1711249015 learned_addr learned my addr 172.21.15.97:0/1711249015 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd37fe700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0dd415deb0 0x7f0dd40b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd27fc700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd4073f90 0x7f0dd4150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 -- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0dd415deb0 msgr2=0x7f0dd40b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0dd415deb0 0x7f0dd40b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 -- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd4073f90 msgr2=0x7f0dd4150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd4073f90 0x7f0dd4150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:11.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 -- 172.21.15.97:0/1711249015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc4009550 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd27fc700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd4073f90 0x7f0dd4150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd2ffd700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd406ed40 0x7f0dd4160ce0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4000c00 tx=0x7f0dc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dbb7fe700 1 -- 172.21.15.97:0/1711249015 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4004570 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dbb7fe700 1 -- 172.21.15.97:0/1711249015 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc401f070 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dbb7fe700 1 -- 172.21.15.97:0/1711249015 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40038e0 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.760+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd41506f0 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.761+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd4161220 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.761+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd4150d40 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.762+0000 7f0dbb7fe700 1 -- 172.21.15.97:0/1711249015 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0dc4021030 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.762+0000 7f0dbb7fe700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0dc006b470 0x7f0dc006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.762+0000 7f0dbb7fe700 1 -- 172.21.15.97:0/1711249015 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0dc408b520 con 0x7f0dd406ed40 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.762+0000 7f0dd27fc700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0dc006b470 0x7f0dc006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:11.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.763+0000 7f0dd27fc700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0dc006b470 0x7f0dc006d930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0dd415d170 tx=0x7f0dc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:11.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.767+0000 7f0dbb7fe700 1 -- 172.21.15.97:0/1711249015 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0dc4055dd0 con 0x7f0dd406ed40 2023-11-24T06:12:12.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.997+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dd4068480 con 0x7f0dd406ed40 2023-11-24T06:12:12.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.997+0000 7f0dbb7fe700 1 -- 172.21.15.97:0/1711249015 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f0dc40593f0 con 0x7f0dd406ed40 2023-11-24T06:12:12.001 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:12.001 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6903,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3848239046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3848239046},{"type":"v1","addr":"172.21.15.97:6836","nonce":3848239046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3181686967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3181686967},{"type":"v1","addr":"172.21.15.97:6837","nonce":3181686967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/4023096288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4023096288},{"type":"v1","addr":"172.21.15.183:6834","nonce":4023096288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:07.978316+0000","modified":"2023-11-24T06:12:11.003557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6921},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3868240464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3868240464},{"type":"v1","addr":"172.21.15.183:6835","nonce":3868240464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:12:12.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0dc006b470 msgr2=0x7f0dc006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0dc006b470 0x7f0dc006d930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0dd415d170 tx=0x7f0dc8008040 comp rx=0 tx=0).stop 2023-11-24T06:12:12.003 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd406ed40 msgr2=0x7f0dd4160ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd406ed40 0x7f0dd4160ce0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4000c00 tx=0x7f0dc4004750 comp rx=0 tx=0).stop 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 shutdown_connections 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f0dc006b470 0x7f0dc006d930 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0dd415deb0 0x7f0dd40b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dd406ed40 0x7f0dd4160ce0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 --2- 172.21.15.97:0/1711249015 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0dd4073f90 0x7f0dd4150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 >> 172.21.15.97:0/1711249015 conn(0x7f0dd40d0ba0 msgr2=0x7f0dd40ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 shutdown_connections 2023-11-24T06:12:12.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:11.999+0000 7f0dd962e700 1 -- 172.21.15.97:0/1711249015 wait complete. 2023-11-24T06:12:12.005 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 55 2023-11-24T06:12:12.015 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.131+0000 7f75f12e0700 1 Processor -- start 2023-11-24T06:12:12.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75f12e0700 1 -- start start 2023-11-24T06:12:12.137 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75f12e0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75ec158770 0x7f75ec06e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75f12e0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec15b140 0x7f75ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75f12e0700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec0b3000 con 0x7f75ec0b2a40 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75f12e0700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ec0b32e0 con 0x7f75ec158770 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75f12e0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0b35c0 con 0x7f75ec15b140 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75eaffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f75ec158770 0x7f75ec06e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75eb7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f75ec0b2a40 0x7f75ec15e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43590/0 (socket says 172.21.15.97:43590) 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.132+0000 7f75eb7fe700 1 -- 172.21.15.97:0/2714301276 learned_addr learned my addr 172.21.15.97:0/2714301276 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75ea7fc700 1 --2- 172.21.15.97:0/2714301276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec15b140 0x7f75ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75eaffd700 1 -- 172.21.15.97:0/2714301276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec15b140 msgr2=0x7f75ec06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75eaffd700 1 --2- 172.21.15.97:0/2714301276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec15b140 0x7f75ec06e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.138 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75eaffd700 1 -- 172.21.15.97:0/2714301276 >> v1:172.21.15.97:6789/0 conn(0x7f75ec0b2a40 legacy=0x7f75ec15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75eaffd700 1 -- 172.21.15.97:0/2714301276 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec0b38a0 con 0x7f75ec158770 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75e9ffb700 1 -- 172.21.15.97:0/2714301276 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135290422 0 0) 0x7f75ec0b3000 con 0x7f75ec0b2a40 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75ea7fc700 1 --2- 172.21.15.97:0/2714301276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec15b140 0x7f75ec06e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75eaffd700 1 --2- 172.21.15.97:0/2714301276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec06e460 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009840 tx=0x7f75e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bf0028bc4d6af9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75e9ffb700 1 -- 172.21.15.97:0/2714301276 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0011830 con 0x7f75ec158770 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75e9ffb700 1 -- 172.21.15.97:0/2714301276 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e0011990 con 0x7f75ec158770 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.133+0000 7f75e9ffb700 1 -- 172.21.15.97:0/2714301276 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0011d20 con 0x7f75ec158770 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.134+0000 7f75f12e0700 1 -- 172.21.15.97:0/2714301276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 msgr2=0x7f75ec06e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.134+0000 7f75f12e0700 1 --2- 172.21.15.97:0/2714301276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec06e460 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009840 tx=0x7f75e0009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.134+0000 7f75f12e0700 1 -- 172.21.15.97:0/2714301276 shutdown_connections 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.134+0000 7f75f12e0700 1 --2- 172.21.15.97:0/2714301276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec06e460 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.139 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.134+0000 7f75f12e0700 1 --2- 172.21.15.97:0/2714301276 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec15b140 0x7f75ec06e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.134+0000 7f75f12e0700 1 -- 172.21.15.97:0/2714301276 >> 172.21.15.97:0/2714301276 conn(0x7f75ec0b1740 msgr2=0x7f75ec0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.134+0000 7f75f12e0700 1 -- 172.21.15.97:0/2714301276 shutdown_connections 2023-11-24T06:12:12.140 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.134+0000 7f75f12e0700 1 -- 172.21.15.97:0/2714301276 wait complete. 2023-11-24T06:12:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.135+0000 7f75f12e0700 1 Processor -- start 2023-11-24T06:12:12.142 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.135+0000 7f75f12e0700 1 -- start start 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75f12e0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec0b2a40 0x7f75ec0a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75f12e0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec0ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75f12e0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75ec15b140 0x7f75ec14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75f12e0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0aafb0 con 0x7f75ec158770 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75f12e0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec155460 con 0x7f75ec15b140 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75f12e0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec155740 con 0x7f75ec0b2a40 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec0ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec0ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51142/0 (socket says 172.21.15.97:51142) 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 -- 172.21.15.97:0/3501078218 learned_addr learned my addr 172.21.15.97:0/3501078218 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75eaffd700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec0b2a40 0x7f75ec0a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75eb7fe700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75ec15b140 0x7f75ec14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 -- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec0b2a40 msgr2=0x7f75ec0a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.143 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec0b2a40 0x7f75ec0a9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 -- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75ec15b140 msgr2=0x7f75ec14a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75ec15b140 0x7f75ec14a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 -- 172.21.15.97:0/3501078218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00094a0 con 0x7f75ec158770 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75eb7fe700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75ec15b140 0x7f75ec14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75ea7fc700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec0ce1b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c7f0 tx=0x7f75dc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75d37fe700 1 -- 172.21.15.97:0/3501078218 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc011800 con 0x7f75ec158770 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75d37fe700 1 -- 172.21.15.97:0/3501078218 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc011960 con 0x7f75ec158770 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec155a20 con 0x7f75ec158770 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.136+0000 7f75d37fe700 1 -- 172.21.15.97:0/3501078218 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc011d10 con 0x7f75ec158770 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.137+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec155fe0 con 0x7f75ec158770 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.137+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec0751e0 con 0x7f75ec158770 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.138+0000 7f75d37fe700 1 -- 172.21.15.97:0/3501078218 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f75dc018360 con 0x7f75ec158770 2023-11-24T06:12:12.144 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.138+0000 7f75d37fe700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f75d806b470 0x7f75d806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.138+0000 7f75d37fe700 1 -- 172.21.15.97:0/3501078218 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f75dc08a1a0 con 0x7f75ec158770 2023-11-24T06:12:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.138+0000 7f75eaffd700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f75d806b470 0x7f75d806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.145 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.139+0000 7f75eaffd700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f75d806b470 0x7f75d806d930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f75e00090b0 tx=0x7f75e0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.146 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.143+0000 7f75d37fe700 1 -- 172.21.15.97:0/3501078218 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75dc054820 con 0x7f75ec158770 2023-11-24T06:12:12.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.333+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75ec157b40 con 0x7f75ec158770 2023-11-24T06:12:12.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.335+0000 7f75d37fe700 1 -- 172.21.15.97:0/3501078218 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f75dc057e40 con 0x7f75ec158770 2023-11-24T06:12:12.341 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:12.341 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:09.577392+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:05.739467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:06.934719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f75d806b470 msgr2=0x7f75d806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f75d806b470 0x7f75d806d930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f75e00090b0 tx=0x7f75e0008030 comp rx=0 tx=0).stop 2023-11-24T06:12:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 msgr2=0x7f75ec0ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec0ce1b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c7f0 tx=0x7f75dc00cbb0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 shutdown_connections 2023-11-24T06:12:12.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f75ec15b140 0x7f75ec14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f75d806b470 0x7f75d806d930 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f75ec0b2a40 0x7f75ec0a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 --2- 172.21.15.97:0/3501078218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f75ec158770 0x7f75ec0ce1b0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 >> 172.21.15.97:0/3501078218 conn(0x7f75ec0b1740 msgr2=0x7f75ec0cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 shutdown_connections 2023-11-24T06:12:12.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.337+0000 7f75f12e0700 1 -- 172.21.15.97:0/3501078218 wait complete. 2023-11-24T06:12:12.353 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:12:12.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:12.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.470+0000 7f938b553700 1 Processor -- start 2023-11-24T06:12:12.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.470+0000 7f938b553700 1 -- start start 2023-11-24T06:12:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f938b553700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938415e080 0x7f93840b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f938b553700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93840af4d0 0x7f93840afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f938b553700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f938406c000 0x7f938406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f938b553700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93840af0b0 con 0x7f938406c000 2023-11-24T06:12:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f938b553700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938406c9f0 con 0x7f93840af4d0 2023-11-24T06:12:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f938b553700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93840b05a0 con 0x7f938415e080 2023-11-24T06:12:12.476 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f938406c000 0x7f938406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f938406c000 0x7f938406c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51152/0 (socket says 172.21.15.97:51152) 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 -- 172.21.15.97:0/1459259346 learned_addr learned my addr 172.21.15.97:0/1459259346 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9388aee700 1 --2- 172.21.15.97:0/1459259346 >> v2:172.21.15.183:3300/0 conn(0x7f93840af4d0 0x7f93840afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 -- 172.21.15.97:0/1459259346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938415e080 msgr2=0x7f93840b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f93892ef700 1 --2- 172.21.15.97:0/1459259346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938415e080 0x7f93840b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 --2- 172.21.15.97:0/1459259346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938415e080 0x7f93840b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 -- 172.21.15.97:0/1459259346 >> v2:172.21.15.183:3300/0 conn(0x7f93840af4d0 msgr2=0x7f93840afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 --2- 172.21.15.97:0/1459259346 >> v2:172.21.15.183:3300/0 conn(0x7f93840af4d0 0x7f93840afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 -- 172.21.15.97:0/1459259346 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93840ae430 con 0x7f938406c000 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9388aee700 1 --2- 172.21.15.97:0/1459259346 >> v2:172.21.15.183:3300/0 conn(0x7f93840af4d0 0x7f93840afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f9389af0700 1 --2- 172.21.15.97:0/1459259346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938406c000 0x7f938406c4b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f938406d380 tx=0x7f937c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2e4c617479c362 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.477 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.471+0000 7f937bfff700 1 -- 172.21.15.97:0/1459259346 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c00e070 con 0x7f938406c000 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f937bfff700 1 -- 172.21.15.97:0/1459259346 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c00b980 con 0x7f938406c000 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f937bfff700 1 -- 172.21.15.97:0/1459259346 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c00bd30 con 0x7f938406c000 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 -- 172.21.15.97:0/1459259346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938406c000 msgr2=0x7f938406c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 --2- 172.21.15.97:0/1459259346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938406c000 0x7f938406c4b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f938406d380 tx=0x7f937c00b330 comp rx=0 tx=0).stop 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 -- 172.21.15.97:0/1459259346 shutdown_connections 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 --2- 172.21.15.97:0/1459259346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938406c000 0x7f938406c4b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 --2- 172.21.15.97:0/1459259346 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938415e080 0x7f93840b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 --2- 172.21.15.97:0/1459259346 >> v2:172.21.15.183:3300/0 conn(0x7f93840af4d0 0x7f93840afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 -- 172.21.15.97:0/1459259346 >> 172.21.15.97:0/1459259346 conn(0x7f9384071b90 msgr2=0x7f9384071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 -- 172.21.15.97:0/1459259346 shutdown_connections 2023-11-24T06:12:12.478 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.472+0000 7f938b553700 1 -- 172.21.15.97:0/1459259346 wait complete. 2023-11-24T06:12:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.473+0000 7f938b553700 1 Processor -- start 2023-11-24T06:12:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.473+0000 7f938b553700 1 -- start start 2023-11-24T06:12:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f938b553700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938406c000 0x7f93841558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f938b553700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93840af4d0 0x7f93841592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f938b553700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938415e080 0x7f93840cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f938b553700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384155e10 con 0x7f93840af4d0 2023-11-24T06:12:12.481 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f938b553700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93841f3c50 con 0x7f938415e080 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f938b553700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93841f3f30 con 0x7f938406c000 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f9388aee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93840af4d0 0x7f93841592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f9388aee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93840af4d0 0x7f93841592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51156/0 (socket says 172.21.15.97:51156) 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f9388aee700 1 -- 172.21.15.97:0/3746049551 learned_addr learned my addr 172.21.15.97:0/3746049551 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f93892ef700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938406c000 0x7f93841558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f9389af0700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938415e080 0x7f93840cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f9388aee700 1 -- 172.21.15.97:0/3746049551 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938406c000 msgr2=0x7f93841558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f9388aee700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938406c000 0x7f93841558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f9388aee700 1 -- 172.21.15.97:0/3746049551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938415e080 msgr2=0x7f93840cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.474+0000 7f9388aee700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938415e080 0x7f93840cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.482 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.475+0000 7f9388aee700 1 -- 172.21.15.97:0/3746049551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c009b30 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.475+0000 7f9388aee700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93840af4d0 0x7f93841592d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f937400c7c0 tx=0x7f937400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.475+0000 7f9379ffb700 1 -- 172.21.15.97:0/3746049551 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374004220 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.475+0000 7f9379ffb700 1 -- 172.21.15.97:0/3746049551 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9374004380 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.475+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93841f4210 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.475+0000 7f9379ffb700 1 -- 172.21.15.97:0/3746049551 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93740036c0 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.475+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93841f47d0 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.476+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93840753f0 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.476+0000 7f9379ffb700 1 -- 172.21.15.97:0/3746049551 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f9374003820 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.477+0000 7f9379ffb700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f936806b470 0x7f936806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.477+0000 7f9379ffb700 1 -- 172.21.15.97:0/3746049551 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f937408a310 con 0x7f93840af4d0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.477+0000 7f93892ef700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f936806b470 0x7f936806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.477+0000 7f93892ef700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f936806b470 0x7f936806d930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9380006fd0 tx=0x7f9380008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.481+0000 7f9379ffb700 1 -- 172.21.15.97:0/3746049551 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93740549f0 con 0x7f93840af4d0 2023-11-24T06:12:12.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.668+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93840b3aa0 con 0x7f936806b470 2023-11-24T06:12:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.676+0000 7f9379ffb700 1 -- 172.21.15.97:0/3746049551 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289346 (secure 0 0 0) 0x7f93840b3aa0 con 0x7f936806b470 2023-11-24T06:12:12.683 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f936806b470 msgr2=0x7f936806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.685 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f936806b470 0x7f936806d930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9380006fd0 tx=0x7f9380008040 comp rx=0 tx=0).stop 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93840af4d0 msgr2=0x7f93841592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93840af4d0 0x7f93841592d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f937400c7c0 tx=0x7f937400cad0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 shutdown_connections 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f938415e080 0x7f93840cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f936806b470 0x7f936806d930 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f938406c000 0x7f93841558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 --2- 172.21.15.97:0/3746049551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93840af4d0 0x7f93841592d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 >> 172.21.15.97:0/3746049551 conn(0x7f9384071b90 msgr2=0x7f93840b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 shutdown_connections 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.680+0000 7f938b553700 1 -- 172.21.15.97:0/3746049551 wait complete. 2023-11-24T06:12:12.686 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:12:12.696 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2023-11-24T06:12:12.362696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225392,"kb_used_data":9204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749328,"statfs":{"total":773094113280,"available":772863311872,"internally_reserved":0,"allocated":9424896,"data_stored":4690812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.795742"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.611798+0000","last_change":"2023-11-24T06:12:08.606482+0000","last_active":"2023-11-24T06:12:08.611798+0000","last_peered":"2023-11-24T06:12:08.611798+0000","last_clean":"2023-11-24T06:12:08.611798+0000","last_became_active":"2023-11-24T06:12:08.604689+0000","last_became_peered":"2023-11-24T06:12:08.604689+0000","last_unstale":"2023-11-24T06:12:08.611798+0000","last_undegraded":"2023-11-24T06:12:08.611798+0000","last_fullsized":"2023-11-24T06:12:08.611798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.496622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.608282+0000","last_change":"2023-11-24T06:12:08.605860+0000","last_active":"2023-11-24T06:12:08.608282+0000","last_peered":"2023-11-24T06:12:08.608282+0000","last_clean":"2023-11-24T06:12:08.608282+0000","last_became_active":"2023-11-24T06:12:08.605561+0000","last_became_peered":"2023-11-24T06:12:08.605561+0000","last_unstale":"2023-11-24T06:12:08.608282+0000","last_undegraded":"2023-11-24T06:12:08.608282+0000","last_fullsized":"2023-11-24T06:12:08.608282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:07:12.226762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610533+0000","last_change":"2023-11-24T06:12:08.606534+0000","last_active":"2023-11-24T06:12:08.610533+0000","last_peered":"2023-11-24T06:12:08.610533+0000","last_clean":"2023-11-24T06:12:08.610533+0000","last_became_active":"2023-11-24T06:12:08.606262+0000","last_became_peered":"2023-11-24T06:12:08.606262+0000","last_unstale":"2023-11-24T06:12:08.610533+0000","last_undegraded":"2023-11-24T06:12:08.610533+0000","last_fullsized":"2023-11-24T06:12:08.610533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:11.639234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.591333+0000","last_change":"2023-11-24T06:12:08.590098+0000","last_active":"2023-11-24T06:12:08.591333+0000","last_peered":"2023-11-24T06:12:08.591333+0000","last_clean":"2023-11-24T06:12:08.591333+0000","last_became_active":"2023-11-24T06:12:08.589831+0000","last_became_peered":"2023-11-24T06:12:08.589831+0000","last_unstale":"2023-11-24T06:12:08.591333+0000","last_undegraded":"2023-11-24T06:12:08.591333+0000","last_fullsized":"2023-11-24T06:12:08.591333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:16.508157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601151+0000","last_change":"2023-11-24T06:12:08.596231+0000","last_active":"2023-11-24T06:12:08.601151+0000","last_peered":"2023-11-24T06:12:08.601151+0000","last_clean":"2023-11-24T06:12:08.601151+0000","last_became_active":"2023-11-24T06:12:08.595963+0000","last_became_peered":"2023-11-24T06:12:08.595963+0000","last_unstale":"2023-11-24T06:12:08.601151+0000","last_undegraded":"2023-11-24T06:12:08.601151+0000","last_fullsized":"2023-11-24T06:12:08.601151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:15.556683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600001+0000","last_change":"2023-11-24T06:12:08.595454+0000","last_active":"2023-11-24T06:12:08.600001+0000","last_peered":"2023-11-24T06:12:08.600001+0000","last_clean":"2023-11-24T06:12:08.600001+0000","last_became_active":"2023-11-24T06:12:08.595255+0000","last_became_peered":"2023-11-24T06:12:08.595255+0000","last_unstale":"2023-11-24T06:12:08.600001+0000","last_undegraded":"2023-11-24T06:12:08.600001+0000","last_fullsized":"2023-11-24T06:12:08.600001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:54.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.589967+0000","last_change":"2023-11-24T06:12:08.587062+0000","last_active":"2023-11-24T06:12:08.589967+0000","last_peered":"2023-11-24T06:12:08.589967+0000","last_clean":"2023-11-24T06:12:08.589967+0000","last_became_active":"2023-11-24T06:12:08.586573+0000","last_became_peered":"2023-11-24T06:12:08.586573+0000","last_unstale":"2023-11-24T06:12:08.589967+0000","last_undegraded":"2023-11-24T06:12:08.589967+0000","last_fullsized":"2023-11-24T06:12:08.589967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:25.412669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601104+0000","last_change":"2023-11-24T06:12:08.596549+0000","last_active":"2023-11-24T06:12:08.601104+0000","last_peered":"2023-11-24T06:12:08.601104+0000","last_clean":"2023-11-24T06:12:08.601104+0000","last_became_active":"2023-11-24T06:12:08.596263+0000","last_became_peered":"2023-11-24T06:12:08.596263+0000","last_unstale":"2023-11-24T06:12:08.601104+0000","last_undegraded":"2023-11-24T06:12:08.601104+0000","last_fullsized":"2023-11-24T06:12:08.601104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:36.334024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600107+0000","last_change":"2023-11-24T06:12:08.595859+0000","last_active":"2023-11-24T06:12:08.600107+0000","last_peered":"2023-11-24T06:12:08.600107+0000","last_clean":"2023-11-24T06:12:08.600107+0000","last_became_active":"2023-11-24T06:12:08.595403+0000","last_became_peered":"2023-11-24T06:12:08.595403+0000","last_unstale":"2023-11-24T06:12:08.600107+0000","last_undegraded":"2023-11-24T06:12:08.600107+0000","last_fullsized":"2023-11-24T06:12:08.600107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:56.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610067+0000","last_change":"2023-11-24T06:12:08.605059+0000","last_active":"2023-11-24T06:12:08.610067+0000","last_peered":"2023-11-24T06:12:08.610067+0000","last_clean":"2023-11-24T06:12:08.610067+0000","last_became_active":"2023-11-24T06:12:08.604801+0000","last_became_peered":"2023-11-24T06:12:08.604801+0000","last_unstale":"2023-11-24T06:12:08.610067+0000","last_undegraded":"2023-11-24T06:12:08.610067+0000","last_fullsized":"2023-11-24T06:12:08.610067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:15.556683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.595187+0000","last_change":"2023-11-24T06:12:08.590038+0000","last_active":"2023-11-24T06:12:08.595187+0000","last_peered":"2023-11-24T06:12:08.595187+0000","last_clean":"2023-11-24T06:12:08.595187+0000","last_became_active":"2023-11-24T06:12:08.589682+0000","last_became_peered":"2023-11-24T06:12:08.589682+0000","last_unstale":"2023-11-24T06:12:08.595187+0000","last_undegraded":"2023-11-24T06:12:08.595187+0000","last_fullsized":"2023-11-24T06:12:08.595187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:24.568650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.605898+0000","last_change":"2023-11-24T06:12:08.598041+0000","last_active":"2023-11-24T06:12:08.605898+0000","last_peered":"2023-11-24T06:12:08.605898+0000","last_clean":"2023-11-24T06:12:08.605898+0000","last_became_active":"2023-11-24T06:12:08.597746+0000","last_became_peered":"2023-11-24T06:12:08.597746+0000","last_unstale":"2023-11-24T06:12:08.605898+0000","last_undegraded":"2023-11-24T06:12:08.605898+0000","last_fullsized":"2023-11-24T06:12:08.605898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:06.650930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.608735+0000","last_change":"2023-11-24T06:12:08.601118+0000","last_active":"2023-11-24T06:12:08.608735+0000","last_peered":"2023-11-24T06:12:08.608735+0000","last_clean":"2023-11-24T06:12:08.608735+0000","last_became_active":"2023-11-24T06:12:08.600819+0000","last_became_peered":"2023-11-24T06:12:08.600819+0000","last_unstale":"2023-11-24T06:12:08.608735+0000","last_undegraded":"2023-11-24T06:12:08.608735+0000","last_fullsized":"2023-11-24T06:12:08.608735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:24.205371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599075+0000","last_change":"2023-11-24T06:12:08.592525+0000","last_active":"2023-11-24T06:12:08.599075+0000","last_peered":"2023-11-24T06:12:08.599075+0000","last_clean":"2023-11-24T06:12:08.599075+0000","last_became_active":"2023-11-24T06:12:08.591906+0000","last_became_peered":"2023-11-24T06:12:08.591906+0000","last_unstale":"2023-11-24T06:12:08.599075+0000","last_undegraded":"2023-11-24T06:12:08.599075+0000","last_fullsized":"2023-11-24T06:12:08.599075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:57.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597439+0000","last_change":"2023-11-24T06:12:08.593370+0000","last_active":"2023-11-24T06:12:08.597439+0000","last_peered":"2023-11-24T06:12:08.597439+0000","last_clean":"2023-11-24T06:12:08.597439+0000","last_became_active":"2023-11-24T06:12:08.592975+0000","last_became_peered":"2023-11-24T06:12:08.592975+0000","last_unstale":"2023-11-24T06:12:08.597439+0000","last_undegraded":"2023-11-24T06:12:08.597439+0000","last_fullsized":"2023-11-24T06:12:08.597439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:22:04.337279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.609293+0000","last_change":"2023-11-24T06:12:08.605870+0000","last_active":"2023-11-24T06:12:08.609293+0000","last_peered":"2023-11-24T06:12:08.609293+0000","last_clean":"2023-11-24T06:12:08.609293+0000","last_became_active":"2023-11-24T06:12:08.605594+0000","last_became_peered":"2023-11-24T06:12:08.605594+0000","last_unstale":"2023-11-24T06:12:08.609293+0000","last_undegraded":"2023-11-24T06:12:08.609293+0000","last_fullsized":"2023-11-24T06:12:08.609293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:15.801019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601990+0000","last_change":"2023-11-24T06:12:08.594594+0000","last_active":"2023-11-24T06:12:08.601990+0000","last_peered":"2023-11-24T06:12:08.601990+0000","last_clean":"2023-11-24T06:12:08.601990+0000","last_became_active":"2023-11-24T06:12:08.594289+0000","last_became_peered":"2023-11-24T06:12:08.594289+0000","last_unstale":"2023-11-24T06:12:08.601990+0000","last_undegraded":"2023-11-24T06:12:08.601990+0000","last_fullsized":"2023-11-24T06:12:08.601990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:54.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616292+0000","last_change":"2023-11-24T06:12:08.605733+0000","last_active":"2023-11-24T06:12:08.616292+0000","last_peered":"2023-11-24T06:12:08.616292+0000","last_clean":"2023-11-24T06:12:08.616292+0000","last_became_active":"2023-11-24T06:12:08.605448+0000","last_became_peered":"2023-11-24T06:12:08.605448+0000","last_unstale":"2023-11-24T06:12:08.616292+0000","last_undegraded":"2023-11-24T06:12:08.616292+0000","last_fullsized":"2023-11-24T06:12:08.616292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.537897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597888+0000","last_change":"2023-11-24T06:12:08.593304+0000","last_active":"2023-11-24T06:12:08.597888+0000","last_peered":"2023-11-24T06:12:08.597888+0000","last_clean":"2023-11-24T06:12:08.597888+0000","last_became_active":"2023-11-24T06:12:08.593037+0000","last_became_peered":"2023-11-24T06:12:08.593037+0000","last_unstale":"2023-11-24T06:12:08.597888+0000","last_undegraded":"2023-11-24T06:12:08.597888+0000","last_fullsized":"2023-11-24T06:12:08.597888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:00.716649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593637+0000","last_change":"2023-11-24T06:12:08.590634+0000","last_active":"2023-11-24T06:12:08.593637+0000","last_peered":"2023-11-24T06:12:08.593637+0000","last_clean":"2023-11-24T06:12:08.593637+0000","last_became_active":"2023-11-24T06:12:08.590369+0000","last_became_peered":"2023-11-24T06:12:08.590369+0000","last_unstale":"2023-11-24T06:12:08.593637+0000","last_undegraded":"2023-11-24T06:12:08.593637+0000","last_fullsized":"2023-11-24T06:12:08.593637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:52:59.601691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599113+0000","last_change":"2023-11-24T06:12:08.593687+0000","last_active":"2023-11-24T06:12:08.599113+0000","last_peered":"2023-11-24T06:12:08.599113+0000","last_clean":"2023-11-24T06:12:08.599113+0000","last_became_active":"2023-11-24T06:12:08.593415+0000","last_became_peered":"2023-11-24T06:12:08.593415+0000","last_unstale":"2023-11-24T06:12:08.599113+0000","last_undegraded":"2023-11-24T06:12:08.599113+0000","last_fullsized":"2023-11-24T06:12:08.599113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:28.388932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597625+0000","last_change":"2023-11-24T06:12:08.592618+0000","last_active":"2023-11-24T06:12:08.597625+0000","last_peered":"2023-11-24T06:12:08.597625+0000","last_clean":"2023-11-24T06:12:08.597625+0000","last_became_active":"2023-11-24T06:12:08.592361+0000","last_became_peered":"2023-11-24T06:12:08.592361+0000","last_unstale":"2023-11-24T06:12:08.597625+0000","last_undegraded":"2023-11-24T06:12:08.597625+0000","last_fullsized":"2023-11-24T06:12:08.597625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:23.514172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610013+0000","last_change":"2023-11-24T06:12:08.604842+0000","last_active":"2023-11-24T06:12:08.610013+0000","last_peered":"2023-11-24T06:12:08.610013+0000","last_clean":"2023-11-24T06:12:08.610013+0000","last_became_active":"2023-11-24T06:12:08.604484+0000","last_became_peered":"2023-11-24T06:12:08.604484+0000","last_unstale":"2023-11-24T06:12:08.610013+0000","last_undegraded":"2023-11-24T06:12:08.610013+0000","last_fullsized":"2023-11-24T06:12:08.610013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:38.195599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593572+0000","last_change":"2023-11-24T06:12:08.590609+0000","last_active":"2023-11-24T06:12:08.593572+0000","last_peered":"2023-11-24T06:12:08.593572+0000","last_clean":"2023-11-24T06:12:08.593572+0000","last_became_active":"2023-11-24T06:12:08.590342+0000","last_became_peered":"2023-11-24T06:12:08.590342+0000","last_unstale":"2023-11-24T06:12:08.593572+0000","last_undegraded":"2023-11-24T06:12:08.593572+0000","last_fullsized":"2023-11-24T06:12:08.593572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:33.825970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.613611+0000","last_change":"2023-11-24T06:12:08.609358+0000","last_active":"2023-11-24T06:12:08.613611+0000","last_peered":"2023-11-24T06:12:08.613611+0000","last_clean":"2023-11-24T06:12:08.613611+0000","last_became_active":"2023-11-24T06:12:08.609098+0000","last_became_peered":"2023-11-24T06:12:08.609098+0000","last_unstale":"2023-11-24T06:12:08.613611+0000","last_undegraded":"2023-11-24T06:12:08.613611+0000","last_fullsized":"2023-11-24T06:12:08.613611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:14.952893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.598820+0000","last_change":"2023-11-24T06:12:08.592525+0000","last_active":"2023-11-24T06:12:08.598820+0000","last_peered":"2023-11-24T06:12:08.598820+0000","last_clean":"2023-11-24T06:12:08.598820+0000","last_became_active":"2023-11-24T06:12:08.592205+0000","last_became_peered":"2023-11-24T06:12:08.592205+0000","last_unstale":"2023-11-24T06:12:08.598820+0000","last_undegraded":"2023-11-24T06:12:08.598820+0000","last_fullsized":"2023-11-24T06:12:08.598820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:00.716649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.602389+0000","last_change":"2023-11-24T06:12:08.597798+0000","last_active":"2023-11-24T06:12:08.602389+0000","last_peered":"2023-11-24T06:12:08.602389+0000","last_clean":"2023-11-24T06:12:08.602389+0000","last_became_active":"2023-11-24T06:12:08.597479+0000","last_became_peered":"2023-11-24T06:12:08.597479+0000","last_unstale":"2023-11-24T06:12:08.602389+0000","last_undegraded":"2023-11-24T06:12:08.602389+0000","last_fullsized":"2023-11-24T06:12:08.602389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:18.532081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.611753+0000","last_change":"2023-11-24T06:12:08.606137+0000","last_active":"2023-11-24T06:12:08.611753+0000","last_peered":"2023-11-24T06:12:08.611753+0000","last_clean":"2023-11-24T06:12:08.611753+0000","last_became_active":"2023-11-24T06:12:08.605815+0000","last_became_peered":"2023-11-24T06:12:08.605815+0000","last_unstale":"2023-11-24T06:12:08.611753+0000","last_undegraded":"2023-11-24T06:12:08.611753+0000","last_fullsized":"2023-11-24T06:12:08.611753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:56.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597364+0000","last_change":"2023-11-24T06:12:08.593815+0000","last_active":"2023-11-24T06:12:08.597364+0000","last_peered":"2023-11-24T06:12:08.597364+0000","last_clean":"2023-11-24T06:12:08.597364+0000","last_became_active":"2023-11-24T06:12:08.593522+0000","last_became_peered":"2023-11-24T06:12:08.593522+0000","last_unstale":"2023-11-24T06:12:08.597364+0000","last_undegraded":"2023-11-24T06:12:08.597364+0000","last_fullsized":"2023-11-24T06:12:08.597364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:35.391656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.595868+0000","last_change":"2023-11-24T06:12:08.589186+0000","last_active":"2023-11-24T06:12:08.595868+0000","last_peered":"2023-11-24T06:12:08.595868+0000","last_clean":"2023-11-24T06:12:08.595868+0000","last_became_active":"2023-11-24T06:12:08.588847+0000","last_became_peered":"2023-11-24T06:12:08.588847+0000","last_unstale":"2023-11-24T06:12:08.595868+0000","last_undegraded":"2023-11-24T06:12:08.595868+0000","last_fullsized":"2023-11-24T06:12:08.595868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:25.624664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599776+0000","last_change":"2023-11-24T06:12:08.594763+0000","last_active":"2023-11-24T06:12:08.599776+0000","last_peered":"2023-11-24T06:12:08.599776+0000","last_clean":"2023-11-24T06:12:08.599776+0000","last_became_active":"2023-11-24T06:12:08.594444+0000","last_became_peered":"2023-11-24T06:12:08.594444+0000","last_unstale":"2023-11-24T06:12:08.599776+0000","last_undegraded":"2023-11-24T06:12:08.599776+0000","last_fullsized":"2023-11-24T06:12:08.599776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:06.650930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610044+0000","last_change":"2023-11-24T06:12:08.605004+0000","last_active":"2023-11-24T06:12:08.610044+0000","last_peered":"2023-11-24T06:12:08.610044+0000","last_clean":"2023-11-24T06:12:08.610044+0000","last_became_active":"2023-11-24T06:12:08.604569+0000","last_became_peered":"2023-11-24T06:12:08.604569+0000","last_unstale":"2023-11-24T06:12:08.610044+0000","last_undegraded":"2023-11-24T06:12:08.610044+0000","last_fullsized":"2023-11-24T06:12:08.610044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:38.091512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.609327+0000","last_change":"2023-11-24T06:12:08.605922+0000","last_active":"2023-11-24T06:12:08.609327+0000","last_peered":"2023-11-24T06:12:08.609327+0000","last_clean":"2023-11-24T06:12:08.609327+0000","last_became_active":"2023-11-24T06:12:08.605683+0000","last_became_peered":"2023-11-24T06:12:08.605683+0000","last_unstale":"2023-11-24T06:12:08.609327+0000","last_undegraded":"2023-11-24T06:12:08.609327+0000","last_fullsized":"2023-11-24T06:12:08.609327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:11.145592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616530+0000","last_change":"2023-11-24T06:12:07.588966+0000","last_active":"2023-11-24T06:12:08.616530+0000","last_peered":"2023-11-24T06:12:08.616530+0000","last_clean":"2023-11-24T06:12:08.616530+0000","last_became_active":"2023-11-24T06:12:07.588693+0000","last_became_peered":"2023-11-24T06:12:07.588693+0000","last_unstale":"2023-11-24T06:12:08.616530+0000","last_undegraded":"2023-11-24T06:12:08.616530+0000","last_fullsized":"2023-11-24T06:12:08.616530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:44:44.699163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610185+0000","last_change":"2023-11-24T06:12:08.605831+0000","last_active":"2023-11-24T06:12:08.610185+0000","last_peered":"2023-11-24T06:12:08.610185+0000","last_clean":"2023-11-24T06:12:08.610185+0000","last_became_active":"2023-11-24T06:12:08.605392+0000","last_became_peered":"2023-11-24T06:12:08.605392+0000","last_unstale":"2023-11-24T06:12:08.610185+0000","last_undegraded":"2023-11-24T06:12:08.610185+0000","last_fullsized":"2023-11-24T06:12:08.610185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:54.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.609246+0000","last_change":"2023-11-24T06:12:07.584590+0000","last_active":"2023-11-24T06:12:08.609246+0000","last_peered":"2023-11-24T06:12:08.609246+0000","last_clean":"2023-11-24T06:12:08.609246+0000","last_became_active":"2023-11-24T06:12:07.584334+0000","last_became_peered":"2023-11-24T06:12:07.584334+0000","last_unstale":"2023-11-24T06:12:08.609246+0000","last_undegraded":"2023-11-24T06:12:08.609246+0000","last_fullsized":"2023-11-24T06:12:08.609246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:36.818741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.612267+0000","last_change":"2023-11-24T06:12:08.606797+0000","last_active":"2023-11-24T06:12:08.612267+0000","last_peered":"2023-11-24T06:12:08.612267+0000","last_clean":"2023-11-24T06:12:08.612267+0000","last_became_active":"2023-11-24T06:12:08.606472+0000","last_became_peered":"2023-11-24T06:12:08.606472+0000","last_unstale":"2023-11-24T06:12:08.612267+0000","last_undegraded":"2023-11-24T06:12:08.612267+0000","last_fullsized":"2023-11-24T06:12:08.612267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:07.836285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616646+0000","last_change":"2023-11-24T06:12:07.592666+0000","last_active":"2023-11-24T06:12:08.616646+0000","last_peered":"2023-11-24T06:12:08.616646+0000","last_clean":"2023-11-24T06:12:08.616646+0000","last_became_active":"2023-11-24T06:12:07.592449+0000","last_became_peered":"2023-11-24T06:12:07.592449+0000","last_unstale":"2023-11-24T06:12:08.616646+0000","last_undegraded":"2023-11-24T06:12:08.616646+0000","last_fullsized":"2023-11-24T06:12:08.616646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:16.278761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600307+0000","last_change":"2023-11-24T06:12:08.592130+0000","last_active":"2023-11-24T06:12:08.600307+0000","last_peered":"2023-11-24T06:12:08.600307+0000","last_clean":"2023-11-24T06:12:08.600307+0000","last_became_active":"2023-11-24T06:12:08.591718+0000","last_became_peered":"2023-11-24T06:12:08.591718+0000","last_unstale":"2023-11-24T06:12:08.600307+0000","last_undegraded":"2023-11-24T06:12:08.600307+0000","last_fullsized":"2023-11-24T06:12:08.600307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:12.107805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616718+0000","last_change":"2023-11-24T06:12:07.589767+0000","last_active":"2023-11-24T06:12:08.616718+0000","last_peered":"2023-11-24T06:12:08.616718+0000","last_clean":"2023-11-24T06:12:08.616718+0000","last_became_active":"2023-11-24T06:12:07.589469+0000","last_became_peered":"2023-11-24T06:12:07.589469+0000","last_unstale":"2023-11-24T06:12:08.616718+0000","last_undegraded":"2023-11-24T06:12:08.616718+0000","last_fullsized":"2023-11-24T06:12:08.616718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:49.365378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.595850+0000","last_change":"2023-11-24T06:12:08.589131+0000","last_active":"2023-11-24T06:12:08.595850+0000","last_peered":"2023-11-24T06:12:08.595850+0000","last_clean":"2023-11-24T06:12:08.595850+0000","last_became_active":"2023-11-24T06:12:08.588752+0000","last_became_peered":"2023-11-24T06:12:08.588752+0000","last_unstale":"2023-11-24T06:12:08.595850+0000","last_undegraded":"2023-11-24T06:12:08.595850+0000","last_fullsized":"2023-11-24T06:12:08.595850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:16.262263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.608392+0000","last_change":"2023-11-24T06:12:07.590119+0000","last_active":"2023-11-24T06:12:08.608392+0000","last_peered":"2023-11-24T06:12:08.608392+0000","last_clean":"2023-11-24T06:12:08.608392+0000","last_became_active":"2023-11-24T06:12:07.589831+0000","last_became_peered":"2023-11-24T06:12:07.589831+0000","last_unstale":"2023-11-24T06:12:08.608392+0000","last_undegraded":"2023-11-24T06:12:08.608392+0000","last_fullsized":"2023-11-24T06:12:08.608392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:22.171468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600187+0000","last_change":"2023-11-24T06:12:08.594612+0000","last_active":"2023-11-24T06:12:08.600187+0000","last_peered":"2023-11-24T06:12:08.600187+0000","last_clean":"2023-11-24T06:12:08.600187+0000","last_became_active":"2023-11-24T06:12:08.594269+0000","last_became_peered":"2023-11-24T06:12:08.594269+0000","last_unstale":"2023-11-24T06:12:08.600187+0000","last_undegraded":"2023-11-24T06:12:08.600187+0000","last_fullsized":"2023-11-24T06:12:08.600187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:29.499761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.607024+0000","last_change":"2023-11-24T06:12:07.584244+0000","last_active":"2023-11-24T06:12:08.607024+0000","last_peered":"2023-11-24T06:12:08.607024+0000","last_clean":"2023-11-24T06:12:08.607024+0000","last_became_active":"2023-11-24T06:12:07.583933+0000","last_became_peered":"2023-11-24T06:12:07.583933+0000","last_unstale":"2023-11-24T06:12:08.607024+0000","last_undegraded":"2023-11-24T06:12:08.607024+0000","last_fullsized":"2023-11-24T06:12:08.607024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:11.492197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.598798+0000","last_change":"2023-11-24T06:12:08.592385+0000","last_active":"2023-11-24T06:12:08.598798+0000","last_peered":"2023-11-24T06:12:08.598798+0000","last_clean":"2023-11-24T06:12:08.598798+0000","last_became_active":"2023-11-24T06:12:08.592066+0000","last_became_peered":"2023-11-24T06:12:08.592066+0000","last_unstale":"2023-11-24T06:12:08.598798+0000","last_undegraded":"2023-11-24T06:12:08.598798+0000","last_fullsized":"2023-11-24T06:12:08.598798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:17.283186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.584838+0000","last_change":"2023-11-24T06:12:07.586038+0000","last_active":"2023-11-24T06:12:08.584838+0000","last_peered":"2023-11-24T06:12:08.584838+0000","last_clean":"2023-11-24T06:12:08.584838+0000","last_became_active":"2023-11-24T06:12:07.585777+0000","last_became_peered":"2023-11-24T06:12:07.585777+0000","last_unstale":"2023-11-24T06:12:08.584838+0000","last_undegraded":"2023-11-24T06:12:08.584838+0000","last_fullsized":"2023-11-24T06:12:08.584838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:05.957392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.595450+0000","last_change":"2023-11-24T06:12:08.593209+0000","last_active":"2023-11-24T06:12:08.595450+0000","last_peered":"2023-11-24T06:12:08.595450+0000","last_clean":"2023-11-24T06:12:08.595450+0000","last_became_active":"2023-11-24T06:12:08.592628+0000","last_became_peered":"2023-11-24T06:12:08.592628+0000","last_unstale":"2023-11-24T06:12:08.595450+0000","last_undegraded":"2023-11-24T06:12:08.595450+0000","last_fullsized":"2023-11-24T06:12:08.595450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:39.740310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.584698+0000","last_change":"2023-11-24T06:12:07.584696+0000","last_active":"2023-11-24T06:12:08.584698+0000","last_peered":"2023-11-24T06:12:08.584698+0000","last_clean":"2023-11-24T06:12:08.584698+0000","last_became_active":"2023-11-24T06:12:07.584369+0000","last_became_peered":"2023-11-24T06:12:07.584369+0000","last_unstale":"2023-11-24T06:12:08.584698+0000","last_undegraded":"2023-11-24T06:12:08.584698+0000","last_fullsized":"2023-11-24T06:12:08.584698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:59:42.227709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600263+0000","last_change":"2023-11-24T06:12:08.595547+0000","last_active":"2023-11-24T06:12:08.600263+0000","last_peered":"2023-11-24T06:12:08.600263+0000","last_clean":"2023-11-24T06:12:08.600263+0000","last_became_active":"2023-11-24T06:12:08.595285+0000","last_became_peered":"2023-11-24T06:12:08.595285+0000","last_unstale":"2023-11-24T06:12:08.600263+0000","last_undegraded":"2023-11-24T06:12:08.600263+0000","last_fullsized":"2023-11-24T06:12:08.600263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:29.035100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597392+0000","last_change":"2023-11-24T06:12:07.589536+0000","last_active":"2023-11-24T06:12:08.597392+0000","last_peered":"2023-11-24T06:12:08.597392+0000","last_clean":"2023-11-24T06:12:08.597392+0000","last_became_active":"2023-11-24T06:12:07.589262+0000","last_became_peered":"2023-11-24T06:12:07.589262+0000","last_unstale":"2023-11-24T06:12:08.597392+0000","last_undegraded":"2023-11-24T06:12:08.597392+0000","last_fullsized":"2023-11-24T06:12:08.597392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:18.585330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600088+0000","last_change":"2023-11-24T06:12:08.595682+0000","last_active":"2023-11-24T06:12:08.600088+0000","last_peered":"2023-11-24T06:12:08.600088+0000","last_clean":"2023-11-24T06:12:08.600088+0000","last_became_active":"2023-11-24T06:12:08.595033+0000","last_became_peered":"2023-11-24T06:12:08.595033+0000","last_unstale":"2023-11-24T06:12:08.600088+0000","last_undegraded":"2023-11-24T06:12:08.600088+0000","last_fullsized":"2023-11-24T06:12:08.600088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:38.195599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.594348+0000","last_change":"2023-11-24T06:12:07.585545+0000","last_active":"2023-11-24T06:12:08.594348+0000","last_peered":"2023-11-24T06:12:08.594348+0000","last_clean":"2023-11-24T06:12:08.594348+0000","last_became_active":"2023-11-24T06:12:07.585140+0000","last_became_peered":"2023-11-24T06:12:07.585140+0000","last_unstale":"2023-11-24T06:12:08.594348+0000","last_undegraded":"2023-11-24T06:12:08.594348+0000","last_fullsized":"2023-11-24T06:12:08.594348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:22.171468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.615706+0000","last_change":"2023-11-24T06:12:08.607055+0000","last_active":"2023-11-24T06:12:08.615706+0000","last_peered":"2023-11-24T06:12:08.615706+0000","last_clean":"2023-11-24T06:12:08.615706+0000","last_became_active":"2023-11-24T06:12:08.606759+0000","last_became_peered":"2023-11-24T06:12:08.606759+0000","last_unstale":"2023-11-24T06:12:08.615706+0000","last_undegraded":"2023-11-24T06:12:08.615706+0000","last_fullsized":"2023-11-24T06:12:08.615706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:11.639234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.613868+0000","last_change":"2023-11-24T06:12:07.590857+0000","last_active":"2023-11-24T06:12:08.613868+0000","last_peered":"2023-11-24T06:12:08.613868+0000","last_clean":"2023-11-24T06:12:08.613868+0000","last_became_active":"2023-11-24T06:12:07.590530+0000","last_became_peered":"2023-11-24T06:12:07.590530+0000","last_unstale":"2023-11-24T06:12:08.613868+0000","last_undegraded":"2023-11-24T06:12:08.613868+0000","last_fullsized":"2023-11-24T06:12:08.613868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:54.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601887+0000","last_change":"2023-11-24T06:12:08.597714+0000","last_active":"2023-11-24T06:12:08.601887+0000","last_peered":"2023-11-24T06:12:08.601887+0000","last_clean":"2023-11-24T06:12:08.601887+0000","last_became_active":"2023-11-24T06:12:08.597152+0000","last_became_peered":"2023-11-24T06:12:08.597152+0000","last_unstale":"2023-11-24T06:12:08.601887+0000","last_undegraded":"2023-11-24T06:12:08.601887+0000","last_fullsized":"2023-11-24T06:12:08.601887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:55.350642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.613790+0000","last_change":"2023-11-24T06:12:07.583645+0000","last_active":"2023-11-24T06:12:08.613790+0000","last_peered":"2023-11-24T06:12:08.613790+0000","last_clean":"2023-11-24T06:12:08.613790+0000","last_became_active":"2023-11-24T06:12:07.583072+0000","last_became_peered":"2023-11-24T06:12:07.583072+0000","last_unstale":"2023-11-24T06:12:08.613790+0000","last_undegraded":"2023-11-24T06:12:08.613790+0000","last_fullsized":"2023-11-24T06:12:08.613790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:10.141167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.598457+0000","last_change":"2023-11-24T06:12:08.596484+0000","last_active":"2023-11-24T06:12:08.598457+0000","last_peered":"2023-11-24T06:12:08.598457+0000","last_clean":"2023-11-24T06:12:08.598457+0000","last_became_active":"2023-11-24T06:12:08.596195+0000","last_became_peered":"2023-11-24T06:12:08.596195+0000","last_unstale":"2023-11-24T06:12:08.598457+0000","last_undegraded":"2023-11-24T06:12:08.598457+0000","last_fullsized":"2023-11-24T06:12:08.598457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:14.486732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616200+0000","last_change":"2023-11-24T06:12:07.590421+0000","last_active":"2023-11-24T06:12:08.616200+0000","last_peered":"2023-11-24T06:12:08.616200+0000","last_clean":"2023-11-24T06:12:08.616200+0000","last_became_active":"2023-11-24T06:12:07.590141+0000","last_became_peered":"2023-11-24T06:12:07.590141+0000","last_unstale":"2023-11-24T06:12:08.616200+0000","last_undegraded":"2023-11-24T06:12:08.616200+0000","last_fullsized":"2023-11-24T06:12:08.616200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:29.089675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599596+0000","last_change":"2023-11-24T06:12:08.592977+0000","last_active":"2023-11-24T06:12:08.599596+0000","last_peered":"2023-11-24T06:12:08.599596+0000","last_clean":"2023-11-24T06:12:08.599596+0000","last_became_active":"2023-11-24T06:12:08.592671+0000","last_became_peered":"2023-11-24T06:12:08.592671+0000","last_unstale":"2023-11-24T06:12:08.599596+0000","last_undegraded":"2023-11-24T06:12:08.599596+0000","last_fullsized":"2023-11-24T06:12:08.599596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:16.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.582783+0000","last_change":"2023-11-24T06:12:07.590320+0000","last_active":"2023-11-24T06:12:08.582783+0000","last_peered":"2023-11-24T06:12:08.582783+0000","last_clean":"2023-11-24T06:12:08.582783+0000","last_became_active":"2023-11-24T06:12:07.590055+0000","last_became_peered":"2023-11-24T06:12:07.590055+0000","last_unstale":"2023-11-24T06:12:08.582783+0000","last_undegraded":"2023-11-24T06:12:08.582783+0000","last_fullsized":"2023-11-24T06:12:08.582783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:57.773893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597511+0000","last_change":"2023-11-24T06:12:08.592222+0000","last_active":"2023-11-24T06:12:08.597511+0000","last_peered":"2023-11-24T06:12:08.597511+0000","last_clean":"2023-11-24T06:12:08.597511+0000","last_became_active":"2023-11-24T06:12:08.591943+0000","last_became_peered":"2023-11-24T06:12:08.591943+0000","last_unstale":"2023-11-24T06:12:08.597511+0000","last_undegraded":"2023-11-24T06:12:08.597511+0000","last_fullsized":"2023-11-24T06:12:08.597511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:09.055567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.590802+0000","last_change":"2023-11-24T06:12:07.582347+0000","last_active":"2023-11-24T06:12:08.590802+0000","last_peered":"2023-11-24T06:12:08.590802+0000","last_clean":"2023-11-24T06:12:08.590802+0000","last_became_active":"2023-11-24T06:12:07.582014+0000","last_became_peered":"2023-11-24T06:12:07.582014+0000","last_unstale":"2023-11-24T06:12:08.590802+0000","last_undegraded":"2023-11-24T06:12:08.590802+0000","last_fullsized":"2023-11-24T06:12:08.590802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:54.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.608305+0000","last_change":"2023-11-24T06:12:08.605849+0000","last_active":"2023-11-24T06:12:08.608305+0000","last_peered":"2023-11-24T06:12:08.608305+0000","last_clean":"2023-11-24T06:12:08.608305+0000","last_became_active":"2023-11-24T06:12:08.604824+0000","last_became_peered":"2023-11-24T06:12:08.604824+0000","last_unstale":"2023-11-24T06:12:08.608305+0000","last_undegraded":"2023-11-24T06:12:08.608305+0000","last_fullsized":"2023-11-24T06:12:08.608305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:47.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.615942+0000","last_change":"2023-11-24T06:12:07.589310+0000","last_active":"2023-11-24T06:12:08.615942+0000","last_peered":"2023-11-24T06:12:08.615942+0000","last_clean":"2023-11-24T06:12:08.615942+0000","last_became_active":"2023-11-24T06:12:07.589018+0000","last_became_peered":"2023-11-24T06:12:07.589018+0000","last_unstale":"2023-11-24T06:12:08.615942+0000","last_undegraded":"2023-11-24T06:12:08.615942+0000","last_fullsized":"2023-11-24T06:12:08.615942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:23.200946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600328+0000","last_change":"2023-11-24T06:12:08.595654+0000","last_active":"2023-11-24T06:12:08.600328+0000","last_peered":"2023-11-24T06:12:08.600328+0000","last_clean":"2023-11-24T06:12:08.600328+0000","last_became_active":"2023-11-24T06:12:08.595394+0000","last_became_peered":"2023-11-24T06:12:08.595394+0000","last_unstale":"2023-11-24T06:12:08.600328+0000","last_undegraded":"2023-11-24T06:12:08.600328+0000","last_fullsized":"2023-11-24T06:12:08.600328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:50.369803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610120+0000","last_change":"2023-11-24T06:12:07.589046+0000","last_active":"2023-11-24T06:12:08.610120+0000","last_peered":"2023-11-24T06:12:08.610120+0000","last_clean":"2023-11-24T06:12:08.610120+0000","last_became_active":"2023-11-24T06:12:07.588761+0000","last_became_peered":"2023-11-24T06:12:07.588761+0000","last_unstale":"2023-11-24T06:12:08.610120+0000","last_undegraded":"2023-11-24T06:12:08.610120+0000","last_fullsized":"2023-11-24T06:12:08.610120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:12.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610165+0000","last_change":"2023-11-24T06:12:08.605654+0000","last_active":"2023-11-24T06:12:08.610165+0000","last_peered":"2023-11-24T06:12:08.610165+0000","last_clean":"2023-11-24T06:12:08.610165+0000","last_became_active":"2023-11-24T06:12:08.605035+0000","last_became_peered":"2023-11-24T06:12:08.605035+0000","last_unstale":"2023-11-24T06:12:08.610165+0000","last_undegraded":"2023-11-24T06:12:08.610165+0000","last_fullsized":"2023-11-24T06:12:08.610165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:36.334024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.615262+0000","last_change":"2023-11-24T06:12:07.590485+0000","last_active":"2023-11-24T06:12:08.615262+0000","last_peered":"2023-11-24T06:12:08.615262+0000","last_clean":"2023-11-24T06:12:08.615262+0000","last_became_active":"2023-11-24T06:12:07.590233+0000","last_became_peered":"2023-11-24T06:12:07.590233+0000","last_unstale":"2023-11-24T06:12:08.615262+0000","last_undegraded":"2023-11-24T06:12:08.615262+0000","last_fullsized":"2023-11-24T06:12:08.615262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:11.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601866+0000","last_change":"2023-11-24T06:12:08.597250+0000","last_active":"2023-11-24T06:12:08.601866+0000","last_peered":"2023-11-24T06:12:08.601866+0000","last_clean":"2023-11-24T06:12:08.601866+0000","last_became_active":"2023-11-24T06:12:08.596791+0000","last_became_peered":"2023-11-24T06:12:08.596791+0000","last_unstale":"2023-11-24T06:12:08.601866+0000","last_undegraded":"2023-11-24T06:12:08.601866+0000","last_fullsized":"2023-11-24T06:12:08.601866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:07.836285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.585361+0000","last_change":"2023-11-24T06:12:07.589623+0000","last_active":"2023-11-24T06:12:08.585361+0000","last_peered":"2023-11-24T06:12:08.585361+0000","last_clean":"2023-11-24T06:12:08.585361+0000","last_became_active":"2023-11-24T06:12:07.589182+0000","last_became_peered":"2023-11-24T06:12:07.589182+0000","last_unstale":"2023-11-24T06:12:08.585361+0000","last_undegraded":"2023-11-24T06:12:08.585361+0000","last_fullsized":"2023-11-24T06:12:08.585361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:15.571602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599054+0000","last_change":"2023-11-24T06:12:08.592347+0000","last_active":"2023-11-24T06:12:08.599054+0000","last_peered":"2023-11-24T06:12:08.599054+0000","last_clean":"2023-11-24T06:12:08.599054+0000","last_became_active":"2023-11-24T06:12:08.591558+0000","last_became_peered":"2023-11-24T06:12:08.591558+0000","last_unstale":"2023-11-24T06:12:08.599054+0000","last_undegraded":"2023-11-24T06:12:08.599054+0000","last_fullsized":"2023-11-24T06:12:08.599054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:16.262263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616256+0000","last_change":"2023-11-24T06:12:07.586220+0000","last_active":"2023-11-24T06:12:08.616256+0000","last_peered":"2023-11-24T06:12:08.616256+0000","last_clean":"2023-11-24T06:12:08.616256+0000","last_became_active":"2023-11-24T06:12:07.585925+0000","last_became_peered":"2023-11-24T06:12:07.585925+0000","last_unstale":"2023-11-24T06:12:08.616256+0000","last_undegraded":"2023-11-24T06:12:08.616256+0000","last_fullsized":"2023-11-24T06:12:08.616256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:15.548597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600045+0000","last_change":"2023-11-24T06:12:08.595032+0000","last_active":"2023-11-24T06:12:08.600045+0000","last_peered":"2023-11-24T06:12:08.600045+0000","last_clean":"2023-11-24T06:12:08.600045+0000","last_became_active":"2023-11-24T06:12:08.594772+0000","last_became_peered":"2023-11-24T06:12:08.594772+0000","last_unstale":"2023-11-24T06:12:08.600045+0000","last_undegraded":"2023-11-24T06:12:08.600045+0000","last_fullsized":"2023-11-24T06:12:08.600045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:15.556683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.580306+0000","last_change":"2023-11-24T06:12:07.591435+0000","last_active":"2023-11-24T06:12:08.580306+0000","last_peered":"2023-11-24T06:12:08.580306+0000","last_clean":"2023-11-24T06:12:08.580306+0000","last_became_active":"2023-11-24T06:12:07.591160+0000","last_became_peered":"2023-11-24T06:12:07.591160+0000","last_unstale":"2023-11-24T06:12:08.580306+0000","last_undegraded":"2023-11-24T06:12:08.580306+0000","last_fullsized":"2023-11-24T06:12:08.580306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:38.002561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.615337+0000","last_change":"2023-11-24T06:12:08.600097+0000","last_active":"2023-11-24T06:12:08.615337+0000","last_peered":"2023-11-24T06:12:08.615337+0000","last_clean":"2023-11-24T06:12:08.615337+0000","last_became_active":"2023-11-24T06:12:08.599512+0000","last_became_peered":"2023-11-24T06:12:08.599512+0000","last_unstale":"2023-11-24T06:12:08.615337+0000","last_undegraded":"2023-11-24T06:12:08.615337+0000","last_fullsized":"2023-11-24T06:12:08.615337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:21.299475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.579936+0000","last_change":"2023-11-24T06:12:07.585645+0000","last_active":"2023-11-24T06:12:08.579936+0000","last_peered":"2023-11-24T06:12:08.579936+0000","last_clean":"2023-11-24T06:12:08.579936+0000","last_became_active":"2023-11-24T06:12:07.585429+0000","last_became_peered":"2023-11-24T06:12:07.585429+0000","last_unstale":"2023-11-24T06:12:08.579936+0000","last_undegraded":"2023-11-24T06:12:08.579936+0000","last_fullsized":"2023-11-24T06:12:08.579936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:37.580115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601552+0000","last_change":"2023-11-24T06:12:08.596908+0000","last_active":"2023-11-24T06:12:08.601552+0000","last_peered":"2023-11-24T06:12:08.601552+0000","last_clean":"2023-11-24T06:12:08.601552+0000","last_became_active":"2023-11-24T06:12:08.596652+0000","last_became_peered":"2023-11-24T06:12:08.596652+0000","last_unstale":"2023-11-24T06:12:08.601552+0000","last_undegraded":"2023-11-24T06:12:08.601552+0000","last_fullsized":"2023-11-24T06:12:08.601552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:07.836285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.582474+0000","last_change":"2023-11-24T06:12:07.586276+0000","last_active":"2023-11-24T06:12:08.582474+0000","last_peered":"2023-11-24T06:12:08.582474+0000","last_clean":"2023-11-24T06:12:08.582474+0000","last_became_active":"2023-11-24T06:12:07.586012+0000","last_became_peered":"2023-11-24T06:12:07.586012+0000","last_unstale":"2023-11-24T06:12:08.582474+0000","last_undegraded":"2023-11-24T06:12:08.582474+0000","last_fullsized":"2023-11-24T06:12:08.582474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:35.156697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.606179+0000","last_change":"2023-11-24T06:12:08.597564+0000","last_active":"2023-11-24T06:12:08.606179+0000","last_peered":"2023-11-24T06:12:08.606179+0000","last_clean":"2023-11-24T06:12:08.606179+0000","last_became_active":"2023-11-24T06:12:08.597269+0000","last_became_peered":"2023-11-24T06:12:08.597269+0000","last_unstale":"2023-11-24T06:12:08.606179+0000","last_undegraded":"2023-11-24T06:12:08.606179+0000","last_fullsized":"2023-11-24T06:12:08.606179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:56.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610656+0000","last_change":"2023-11-24T06:12:07.589425+0000","last_active":"2023-11-24T06:12:08.610656+0000","last_peered":"2023-11-24T06:12:08.610656+0000","last_clean":"2023-11-24T06:12:08.610656+0000","last_became_active":"2023-11-24T06:12:07.589149+0000","last_became_peered":"2023-11-24T06:12:07.589149+0000","last_unstale":"2023-11-24T06:12:08.610656+0000","last_undegraded":"2023-11-24T06:12:08.610656+0000","last_fullsized":"2023-11-24T06:12:08.610656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:53:07.619654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.611946+0000","last_change":"2023-11-24T06:12:08.608600+0000","last_active":"2023-11-24T06:12:08.611946+0000","last_peered":"2023-11-24T06:12:08.611946+0000","last_clean":"2023-11-24T06:12:08.611946+0000","last_became_active":"2023-11-24T06:12:08.608347+0000","last_became_peered":"2023-11-24T06:12:08.608347+0000","last_unstale":"2023-11-24T06:12:08.611946+0000","last_undegraded":"2023-11-24T06:12:08.611946+0000","last_fullsized":"2023-11-24T06:12:08.611946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:06.961817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.583338+0000","last_change":"2023-11-24T06:12:07.591565+0000","last_active":"2023-11-24T06:12:08.583338+0000","last_peered":"2023-11-24T06:12:08.583338+0000","last_clean":"2023-11-24T06:12:08.583338+0000","last_became_active":"2023-11-24T06:12:07.591173+0000","last_became_peered":"2023-11-24T06:12:07.591173+0000","last_unstale":"2023-11-24T06:12:08.583338+0000","last_undegraded":"2023-11-24T06:12:08.583338+0000","last_fullsized":"2023-11-24T06:12:08.583338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:30.399381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610183+0000","last_change":"2023-11-24T06:12:08.605785+0000","last_active":"2023-11-24T06:12:08.610183+0000","last_peered":"2023-11-24T06:12:08.610183+0000","last_clean":"2023-11-24T06:12:08.610183+0000","last_became_active":"2023-11-24T06:12:08.605476+0000","last_became_peered":"2023-11-24T06:12:08.605476+0000","last_unstale":"2023-11-24T06:12:08.610183+0000","last_undegraded":"2023-11-24T06:12:08.610183+0000","last_fullsized":"2023-11-24T06:12:08.610183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:29.499761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593453+0000","last_change":"2023-11-24T06:12:07.587808+0000","last_active":"2023-11-24T06:12:08.593453+0000","last_peered":"2023-11-24T06:12:08.593453+0000","last_clean":"2023-11-24T06:12:08.593453+0000","last_became_active":"2023-11-24T06:12:07.587548+0000","last_became_peered":"2023-11-24T06:12:07.587548+0000","last_unstale":"2023-11-24T06:12:08.593453+0000","last_undegraded":"2023-11-24T06:12:08.593453+0000","last_fullsized":"2023-11-24T06:12:08.593453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:36.818741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600712+0000","last_change":"2023-11-24T06:12:08.595502+0000","last_active":"2023-11-24T06:12:08.600712+0000","last_peered":"2023-11-24T06:12:08.600712+0000","last_clean":"2023-11-24T06:12:08.600712+0000","last_became_active":"2023-11-24T06:12:08.595237+0000","last_became_peered":"2023-11-24T06:12:08.595237+0000","last_unstale":"2023-11-24T06:12:08.600712+0000","last_undegraded":"2023-11-24T06:12:08.600712+0000","last_fullsized":"2023-11-24T06:12:08.600712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:38.091512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593789+0000","last_change":"2023-11-24T06:12:07.587358+0000","last_active":"2023-11-24T06:12:08.593789+0000","last_peered":"2023-11-24T06:12:08.593789+0000","last_clean":"2023-11-24T06:12:08.593789+0000","last_became_active":"2023-11-24T06:12:07.587068+0000","last_became_peered":"2023-11-24T06:12:07.587068+0000","last_unstale":"2023-11-24T06:12:08.593789+0000","last_undegraded":"2023-11-24T06:12:08.593789+0000","last_fullsized":"2023-11-24T06:12:08.593789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:17.527656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600020+0000","last_change":"2023-11-24T06:12:08.595718+0000","last_active":"2023-11-24T06:12:08.600020+0000","last_peered":"2023-11-24T06:12:08.600020+0000","last_clean":"2023-11-24T06:12:08.600020+0000","last_became_active":"2023-11-24T06:12:08.594739+0000","last_became_peered":"2023-11-24T06:12:08.594739+0000","last_unstale":"2023-11-24T06:12:08.600020+0000","last_undegraded":"2023-11-24T06:12:08.600020+0000","last_fullsized":"2023-11-24T06:12:08.600020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.496622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.585408+0000","last_change":"2023-11-24T06:12:07.589985+0000","last_active":"2023-11-24T06:12:08.585408+0000","last_peered":"2023-11-24T06:12:08.585408+0000","last_clean":"2023-11-24T06:12:08.585408+0000","last_became_active":"2023-11-24T06:12:07.589718+0000","last_became_peered":"2023-11-24T06:12:07.589718+0000","last_unstale":"2023-11-24T06:12:08.585408+0000","last_undegraded":"2023-11-24T06:12:08.585408+0000","last_fullsized":"2023-11-24T06:12:08.585408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:51:56.017224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600625+0000","last_change":"2023-11-24T06:12:08.597537+0000","last_active":"2023-11-24T06:12:08.600625+0000","last_peered":"2023-11-24T06:12:08.600625+0000","last_clean":"2023-11-24T06:12:08.600625+0000","last_became_active":"2023-11-24T06:12:08.597230+0000","last_became_peered":"2023-11-24T06:12:08.597230+0000","last_unstale":"2023-11-24T06:12:08.600625+0000","last_undegraded":"2023-11-24T06:12:08.600625+0000","last_fullsized":"2023-11-24T06:12:08.600625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:48.821384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.587890+0000","last_change":"2023-11-24T06:12:07.584235+0000","last_active":"2023-11-24T06:12:08.587890+0000","last_peered":"2023-11-24T06:12:08.587890+0000","last_clean":"2023-11-24T06:12:08.587890+0000","last_became_active":"2023-11-24T06:12:07.583915+0000","last_became_peered":"2023-11-24T06:12:07.583915+0000","last_unstale":"2023-11-24T06:12:08.587890+0000","last_undegraded":"2023-11-24T06:12:08.587890+0000","last_fullsized":"2023-11-24T06:12:08.587890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:35.329599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.598778+0000","last_change":"2023-11-24T06:12:08.592740+0000","last_active":"2023-11-24T06:12:08.598778+0000","last_peered":"2023-11-24T06:12:08.598778+0000","last_clean":"2023-11-24T06:12:08.598778+0000","last_became_active":"2023-11-24T06:12:08.592327+0000","last_became_peered":"2023-11-24T06:12:08.592327+0000","last_unstale":"2023-11-24T06:12:08.598778+0000","last_undegraded":"2023-11-24T06:12:08.598778+0000","last_fullsized":"2023-11-24T06:12:08.598778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:48.821384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593106+0000","last_change":"2023-11-24T06:12:07.587426+0000","last_active":"2023-11-24T06:12:08.593106+0000","last_peered":"2023-11-24T06:12:08.593106+0000","last_clean":"2023-11-24T06:12:08.593106+0000","last_became_active":"2023-11-24T06:12:07.587151+0000","last_became_peered":"2023-11-24T06:12:07.587151+0000","last_unstale":"2023-11-24T06:12:08.593106+0000","last_undegraded":"2023-11-24T06:12:08.593106+0000","last_fullsized":"2023-11-24T06:12:08.593106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:10.141167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.609908+0000","last_change":"2023-11-24T06:12:08.602477+0000","last_active":"2023-11-24T06:12:08.609908+0000","last_peered":"2023-11-24T06:12:08.609908+0000","last_clean":"2023-11-24T06:12:08.609908+0000","last_became_active":"2023-11-24T06:12:08.601299+0000","last_became_peered":"2023-11-24T06:12:08.601299+0000","last_unstale":"2023-11-24T06:12:08.609908+0000","last_undegraded":"2023-11-24T06:12:08.609908+0000","last_fullsized":"2023-11-24T06:12:08.609908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:15.801019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.587573+0000","last_change":"2023-11-24T06:12:07.585625+0000","last_active":"2023-11-24T06:12:08.587573+0000","last_peered":"2023-11-24T06:12:08.587573+0000","last_clean":"2023-11-24T06:12:08.587573+0000","last_became_active":"2023-11-24T06:12:07.585257+0000","last_became_peered":"2023-11-24T06:12:07.585257+0000","last_unstale":"2023-11-24T06:12:08.587573+0000","last_undegraded":"2023-11-24T06:12:08.587573+0000","last_fullsized":"2023-11-24T06:12:08.587573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:37.191174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.606575+0000","last_change":"2023-11-24T06:12:08.597276+0000","last_active":"2023-11-24T06:12:08.606575+0000","last_peered":"2023-11-24T06:12:08.606575+0000","last_clean":"2023-11-24T06:12:08.606575+0000","last_became_active":"2023-11-24T06:12:08.596919+0000","last_became_peered":"2023-11-24T06:12:08.596919+0000","last_unstale":"2023-11-24T06:12:08.606575+0000","last_undegraded":"2023-11-24T06:12:08.606575+0000","last_fullsized":"2023-11-24T06:12:08.606575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:26.973356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610467+0000","last_change":"2023-11-24T06:12:07.590006+0000","last_active":"2023-11-24T06:12:08.610467+0000","last_peered":"2023-11-24T06:12:08.610467+0000","last_clean":"2023-11-24T06:12:08.610467+0000","last_became_active":"2023-11-24T06:12:07.589501+0000","last_became_peered":"2023-11-24T06:12:07.589501+0000","last_unstale":"2023-11-24T06:12:08.610467+0000","last_undegraded":"2023-11-24T06:12:08.610467+0000","last_fullsized":"2023-11-24T06:12:08.610467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:51:56.017224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":218,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-24T06:12:07.575317+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:12:07.575317+0000","last_peered":"2023-11-24T06:12:07.575317+0000","last_clean":"2023-11-24T06:12:07.575317+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:12:07.575317+0000","last_undegraded":"2023-11-24T06:12:07.575317+0000","last_fullsized":"2023-11-24T06:12:07.575317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28600,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343240,"statfs":{"total":96636764160,"available":96607477760,"internally_reserved":0,"allocated":1613824,"data_stored":1025496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.755,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.625},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":435128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.71299999999999997,"15min":0.79300000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.75,"15min":0.86199999999999999},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68999999999999995,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":1.371,"15min":1.371},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69099999999999995,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.76400000000000001,"15min":0.85299999999999998},"min":{"1min":0.38800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.629,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69899999999999995,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.499},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":1.407,"15min":1.407},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.439,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80600000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1044480,"data_stored":442125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33900000000000002,"15min":0.34499999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"min":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.379},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.624},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80600000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.79300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343224,"statfs":{"total":96636764160,"available":96607461376,"internally_reserved":0,"allocated":1630208,"data_stored":1032333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.624,"15min":0.75800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47899999999999998,"5min":12.378,"15min":12.378},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.45900000000000002,"15min":0.502},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.496,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.46700000000000003,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.48499999999999999,"15min":0.53000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.28999999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.46300000000000002,"15min":0.50600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52000000000000002,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56000000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.312}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.96099999999999997,"15min":1.125},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73299999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68300000000000005,"15min":0.72699999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82199999999999995,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69299999999999995,"15min":0.73299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.96799999999999997,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":15.356,"15min":15.356},"last":0.622}]},{"osd":6,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71499999999999997,"15min":0.76100000000000001},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":4.726,"15min":4.726},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.71999999999999997,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.997,"15min":1.165},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75,"5min":15.388,"15min":15.388},"last":0.65700000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":435128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32700000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.34399999999999997,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":1.5569999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.95399999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54200000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.173,"15min":1.173},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":4.5540000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.83599999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.92000000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.65900000000000003,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":1.417},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1044480,"data_stored":443543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.32700000000000001,"15min":0.33800000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.39200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.34999999999999998,"15min":0.36699999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.36399999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34499999999999997,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45700000000000002,"5min":1.032,"15min":1.032},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.252,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44400000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.435},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.48599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.622,"15min":0.65200000000000002},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64800000000000002,"5min":1.728,"15min":1.728},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.67100000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.626,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69199999999999995,"5min":1.304,"15min":1.304},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.044,"5min":1.222,"15min":1.222},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69199999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1032192,"data_stored":441931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.437,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.45500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.745,"15min":0.745},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":435128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.63200000000000001,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":3.335,"15min":3.335},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.622,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66900000000000004,"15min":0.72999999999999998},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71299999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.0419999999999998,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.76300000000000001},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":1.536,"15min":1.536},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81999999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93700000000000006,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90700000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.65500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:12.701 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.853+0000 7f6be68ce700 1 Processor -- start 2023-11-24T06:12:12.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.853+0000 7f6be68ce700 1 -- start start 2023-11-24T06:12:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6be68ce700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6be015e080 0x7f6be00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6be68ce700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6be68ce700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be00ae430 con 0x7f6be00af4d0 2023-11-24T06:12:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6be68ce700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be00af0b0 con 0x7f6be015e080 2023-11-24T06:12:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6be68ce700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00b05a0 con 0x7f6be006c710 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6bdffff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6be015e080 0x7f6be00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6be4e6b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6bdffff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6be015e080 0x7f6be00b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:45322/0 (socket says 172.21.15.97:45322) 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6be4e6b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:38202/0 (socket says 172.21.15.97:38202) 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.854+0000 7f6bdffff700 1 -- 172.21.15.97:0/3477627954 learned_addr learned my addr 172.21.15.97:0/3477627954 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6bdffff700 1 -- 172.21.15.97:0/3477627954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 msgr2=0x7f6be00afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6bdffff700 1 --2- 172.21.15.97:0/3477627954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be00afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6bdffff700 1 -- 172.21.15.97:0/3477627954 >> v1:172.21.15.183:6789/0 conn(0x7f6be00af4d0 legacy=0x7f6be006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6bdffff700 1 -- 172.21.15.97:0/3477627954 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be00ae710 con 0x7f6be015e080 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6be4e6b700 1 --2- 172.21.15.97:0/3477627954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be00afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:12.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6bdffff700 1 --2- 172.21.15.97:0/3477627954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be015e080 0x7f6be00b5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009920 tx=0x7f6bd4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c97a2d0cc261ca44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6bdeffd700 1 -- 172.21.15.97:0/3477627954 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4011830 con 0x7f6be015e080 2023-11-24T06:12:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6bdeffd700 1 -- 172.21.15.97:0/3477627954 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4011990 con 0x7f6be015e080 2023-11-24T06:12:12.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.855+0000 7f6bdeffd700 1 -- 172.21.15.97:0/3477627954 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4011d20 con 0x7f6be015e080 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.856+0000 7f6be68ce700 1 -- 172.21.15.97:0/3477627954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be015e080 msgr2=0x7f6be00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.856+0000 7f6be68ce700 1 --2- 172.21.15.97:0/3477627954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be015e080 0x7f6be00b5050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009920 tx=0x7f6bd4009c30 comp rx=0 tx=0).stop 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.856+0000 7f6be68ce700 1 -- 172.21.15.97:0/3477627954 shutdown_connections 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.856+0000 7f6be68ce700 1 --2- 172.21.15.97:0/3477627954 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be00afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.856+0000 7f6be68ce700 1 --2- 172.21.15.97:0/3477627954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be015e080 0x7f6be00b5050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.856+0000 7f6be68ce700 1 -- 172.21.15.97:0/3477627954 >> 172.21.15.97:0/3477627954 conn(0x7f6be0071b90 msgr2=0x7f6be0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.856+0000 7f6be68ce700 1 -- 172.21.15.97:0/3477627954 shutdown_connections 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.856+0000 7f6be68ce700 1 -- 172.21.15.97:0/3477627954 wait complete. 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.857+0000 7f6be68ce700 1 Processor -- start 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.857+0000 7f6be68ce700 1 -- start start 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be68ce700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be0149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be68ce700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be00af4d0 0x7f6be0154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be68ce700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6be015e080 0x7f6be0157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be68ce700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0160310 con 0x7f6be015e080 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be68ce700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be015f720 con 0x7f6be00af4d0 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be68ce700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be015f9e0 con 0x7f6be006c710 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6be015e080 0x7f6be0157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6be015e080 0x7f6be0157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51172/0 (socket says 172.21.15.97:51172) 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 -- 172.21.15.97:0/2411607524 learned_addr learned my addr 172.21.15.97:0/2411607524 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6bdffff700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be0149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6bdf7fe700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be00af4d0 0x7f6be0154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.864 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 -- 172.21.15.97:0/2411607524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 msgr2=0x7f6be0149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be0149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 -- 172.21.15.97:0/2411607524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be00af4d0 msgr2=0x7f6be0154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be00af4d0 0x7f6be0154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 -- 172.21.15.97:0/2411607524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd8009a70 con 0x7f6be015e080 2023-11-24T06:12:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be4e6b700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6be015e080 0x7f6be0157c80 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800ef90 tx=0x7f6bd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6bdcff9700 1 -- 172.21.15.97:0/2411607524 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800cc90 con 0x7f6be015e080 2023-11-24T06:12:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6bdcff9700 1 -- 172.21.15.97:0/2411607524 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd800f040 con 0x7f6be015e080 2023-11-24T06:12:12.865 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.858+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd4009550 con 0x7f6be015e080 2023-11-24T06:12:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.859+0000 7f6bdcff9700 1 -- 172.21.15.97:0/2411607524 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80146c0 con 0x7f6be015e080 2023-11-24T06:12:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.859+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be015ffa0 con 0x7f6be015e080 2023-11-24T06:12:12.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.859+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be00753f0 con 0x7f6be015e080 2023-11-24T06:12:12.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.860+0000 7f6bdcff9700 1 -- 172.21.15.97:0/2411607524 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6bd8014820 con 0x7f6be015e080 2023-11-24T06:12:12.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.860+0000 7f6bdcff9700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6bcc06b470 0x7f6bcc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:12.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.860+0000 7f6bdcff9700 1 -- 172.21.15.97:0/2411607524 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6bd808ae80 con 0x7f6be015e080 2023-11-24T06:12:12.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.860+0000 7f6bdffff700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6bcc06b470 0x7f6bcc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:12.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.861+0000 7f6bdffff700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6bcc06b470 0x7f6bcc06d930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4000c00 tx=0x7f6bd4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:12.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:12.865+0000 7f6bdcff9700 1 -- 172.21.15.97:0/2411607524 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bd808f070 con 0x7f6be015e080 2023-11-24T06:12:13.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.052+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6be00aecd0 con 0x7f6bcc06b470 2023-11-24T06:12:13.063 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.060+0000 7f6bdcff9700 1 -- 172.21.15.97:0/2411607524 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289346 (secure 0 0 0) 0x7f6bd4011b80 con 0x7f6bcc06b470 2023-11-24T06:12:13.067 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6bcc06b470 msgr2=0x7f6bcc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6bcc06b470 0x7f6bcc06d930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4000c00 tx=0x7f6bd4008030 comp rx=0 tx=0).stop 2023-11-24T06:12:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6be015e080 msgr2=0x7f6be0157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6be015e080 0x7f6be0157c80 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800ef90 tx=0x7f6bd800c3a0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 shutdown_connections 2023-11-24T06:12:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6be015e080 0x7f6be0157c80 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6bcc06b470 0x7f6bcc06d930 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6be006c710 0x7f6be0149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 --2- 172.21.15.97:0/2411607524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6be00af4d0 0x7f6be0154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.063+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 >> 172.21.15.97:0/2411607524 conn(0x7f6be0071b90 msgr2=0x7f6be006ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.064+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 shutdown_connections 2023-11-24T06:12:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.064+0000 7f6be68ce700 1 -- 172.21.15.97:0/2411607524 wait complete. 2023-11-24T06:12:13.070 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:12:13.079 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2023-11-24T06:12:12.362696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":103,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225392,"kb_used_data":9204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749328,"statfs":{"total":773094113280,"available":772863311872,"internally_reserved":0,"allocated":9424896,"data_stored":4690812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.795742"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.611798+0000","last_change":"2023-11-24T06:12:08.606482+0000","last_active":"2023-11-24T06:12:08.611798+0000","last_peered":"2023-11-24T06:12:08.611798+0000","last_clean":"2023-11-24T06:12:08.611798+0000","last_became_active":"2023-11-24T06:12:08.604689+0000","last_became_peered":"2023-11-24T06:12:08.604689+0000","last_unstale":"2023-11-24T06:12:08.611798+0000","last_undegraded":"2023-11-24T06:12:08.611798+0000","last_fullsized":"2023-11-24T06:12:08.611798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.496622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.608282+0000","last_change":"2023-11-24T06:12:08.605860+0000","last_active":"2023-11-24T06:12:08.608282+0000","last_peered":"2023-11-24T06:12:08.608282+0000","last_clean":"2023-11-24T06:12:08.608282+0000","last_became_active":"2023-11-24T06:12:08.605561+0000","last_became_peered":"2023-11-24T06:12:08.605561+0000","last_unstale":"2023-11-24T06:12:08.608282+0000","last_undegraded":"2023-11-24T06:12:08.608282+0000","last_fullsized":"2023-11-24T06:12:08.608282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:07:12.226762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610533+0000","last_change":"2023-11-24T06:12:08.606534+0000","last_active":"2023-11-24T06:12:08.610533+0000","last_peered":"2023-11-24T06:12:08.610533+0000","last_clean":"2023-11-24T06:12:08.610533+0000","last_became_active":"2023-11-24T06:12:08.606262+0000","last_became_peered":"2023-11-24T06:12:08.606262+0000","last_unstale":"2023-11-24T06:12:08.610533+0000","last_undegraded":"2023-11-24T06:12:08.610533+0000","last_fullsized":"2023-11-24T06:12:08.610533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:11.639234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.591333+0000","last_change":"2023-11-24T06:12:08.590098+0000","last_active":"2023-11-24T06:12:08.591333+0000","last_peered":"2023-11-24T06:12:08.591333+0000","last_clean":"2023-11-24T06:12:08.591333+0000","last_became_active":"2023-11-24T06:12:08.589831+0000","last_became_peered":"2023-11-24T06:12:08.589831+0000","last_unstale":"2023-11-24T06:12:08.591333+0000","last_undegraded":"2023-11-24T06:12:08.591333+0000","last_fullsized":"2023-11-24T06:12:08.591333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:16.508157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601151+0000","last_change":"2023-11-24T06:12:08.596231+0000","last_active":"2023-11-24T06:12:08.601151+0000","last_peered":"2023-11-24T06:12:08.601151+0000","last_clean":"2023-11-24T06:12:08.601151+0000","last_became_active":"2023-11-24T06:12:08.595963+0000","last_became_peered":"2023-11-24T06:12:08.595963+0000","last_unstale":"2023-11-24T06:12:08.601151+0000","last_undegraded":"2023-11-24T06:12:08.601151+0000","last_fullsized":"2023-11-24T06:12:08.601151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:15.556683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600001+0000","last_change":"2023-11-24T06:12:08.595454+0000","last_active":"2023-11-24T06:12:08.600001+0000","last_peered":"2023-11-24T06:12:08.600001+0000","last_clean":"2023-11-24T06:12:08.600001+0000","last_became_active":"2023-11-24T06:12:08.595255+0000","last_became_peered":"2023-11-24T06:12:08.595255+0000","last_unstale":"2023-11-24T06:12:08.600001+0000","last_undegraded":"2023-11-24T06:12:08.600001+0000","last_fullsized":"2023-11-24T06:12:08.600001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:54.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.589967+0000","last_change":"2023-11-24T06:12:08.587062+0000","last_active":"2023-11-24T06:12:08.589967+0000","last_peered":"2023-11-24T06:12:08.589967+0000","last_clean":"2023-11-24T06:12:08.589967+0000","last_became_active":"2023-11-24T06:12:08.586573+0000","last_became_peered":"2023-11-24T06:12:08.586573+0000","last_unstale":"2023-11-24T06:12:08.589967+0000","last_undegraded":"2023-11-24T06:12:08.589967+0000","last_fullsized":"2023-11-24T06:12:08.589967+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:54:25.412669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601104+0000","last_change":"2023-11-24T06:12:08.596549+0000","last_active":"2023-11-24T06:12:08.601104+0000","last_peered":"2023-11-24T06:12:08.601104+0000","last_clean":"2023-11-24T06:12:08.601104+0000","last_became_active":"2023-11-24T06:12:08.596263+0000","last_became_peered":"2023-11-24T06:12:08.596263+0000","last_unstale":"2023-11-24T06:12:08.601104+0000","last_undegraded":"2023-11-24T06:12:08.601104+0000","last_fullsized":"2023-11-24T06:12:08.601104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:36.334024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600107+0000","last_change":"2023-11-24T06:12:08.595859+0000","last_active":"2023-11-24T06:12:08.600107+0000","last_peered":"2023-11-24T06:12:08.600107+0000","last_clean":"2023-11-24T06:12:08.600107+0000","last_became_active":"2023-11-24T06:12:08.595403+0000","last_became_peered":"2023-11-24T06:12:08.595403+0000","last_unstale":"2023-11-24T06:12:08.600107+0000","last_undegraded":"2023-11-24T06:12:08.600107+0000","last_fullsized":"2023-11-24T06:12:08.600107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:56.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610067+0000","last_change":"2023-11-24T06:12:08.605059+0000","last_active":"2023-11-24T06:12:08.610067+0000","last_peered":"2023-11-24T06:12:08.610067+0000","last_clean":"2023-11-24T06:12:08.610067+0000","last_became_active":"2023-11-24T06:12:08.604801+0000","last_became_peered":"2023-11-24T06:12:08.604801+0000","last_unstale":"2023-11-24T06:12:08.610067+0000","last_undegraded":"2023-11-24T06:12:08.610067+0000","last_fullsized":"2023-11-24T06:12:08.610067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:15.556683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.595187+0000","last_change":"2023-11-24T06:12:08.590038+0000","last_active":"2023-11-24T06:12:08.595187+0000","last_peered":"2023-11-24T06:12:08.595187+0000","last_clean":"2023-11-24T06:12:08.595187+0000","last_became_active":"2023-11-24T06:12:08.589682+0000","last_became_peered":"2023-11-24T06:12:08.589682+0000","last_unstale":"2023-11-24T06:12:08.595187+0000","last_undegraded":"2023-11-24T06:12:08.595187+0000","last_fullsized":"2023-11-24T06:12:08.595187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:34:24.568650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.605898+0000","last_change":"2023-11-24T06:12:08.598041+0000","last_active":"2023-11-24T06:12:08.605898+0000","last_peered":"2023-11-24T06:12:08.605898+0000","last_clean":"2023-11-24T06:12:08.605898+0000","last_became_active":"2023-11-24T06:12:08.597746+0000","last_became_peered":"2023-11-24T06:12:08.597746+0000","last_unstale":"2023-11-24T06:12:08.605898+0000","last_undegraded":"2023-11-24T06:12:08.605898+0000","last_fullsized":"2023-11-24T06:12:08.605898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:06.650930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.608735+0000","last_change":"2023-11-24T06:12:08.601118+0000","last_active":"2023-11-24T06:12:08.608735+0000","last_peered":"2023-11-24T06:12:08.608735+0000","last_clean":"2023-11-24T06:12:08.608735+0000","last_became_active":"2023-11-24T06:12:08.600819+0000","last_became_peered":"2023-11-24T06:12:08.600819+0000","last_unstale":"2023-11-24T06:12:08.608735+0000","last_undegraded":"2023-11-24T06:12:08.608735+0000","last_fullsized":"2023-11-24T06:12:08.608735+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:24.205371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599075+0000","last_change":"2023-11-24T06:12:08.592525+0000","last_active":"2023-11-24T06:12:08.599075+0000","last_peered":"2023-11-24T06:12:08.599075+0000","last_clean":"2023-11-24T06:12:08.599075+0000","last_became_active":"2023-11-24T06:12:08.591906+0000","last_became_peered":"2023-11-24T06:12:08.591906+0000","last_unstale":"2023-11-24T06:12:08.599075+0000","last_undegraded":"2023-11-24T06:12:08.599075+0000","last_fullsized":"2023-11-24T06:12:08.599075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:57.925990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597439+0000","last_change":"2023-11-24T06:12:08.593370+0000","last_active":"2023-11-24T06:12:08.597439+0000","last_peered":"2023-11-24T06:12:08.597439+0000","last_clean":"2023-11-24T06:12:08.597439+0000","last_became_active":"2023-11-24T06:12:08.592975+0000","last_became_peered":"2023-11-24T06:12:08.592975+0000","last_unstale":"2023-11-24T06:12:08.597439+0000","last_undegraded":"2023-11-24T06:12:08.597439+0000","last_fullsized":"2023-11-24T06:12:08.597439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:22:04.337279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.609293+0000","last_change":"2023-11-24T06:12:08.605870+0000","last_active":"2023-11-24T06:12:08.609293+0000","last_peered":"2023-11-24T06:12:08.609293+0000","last_clean":"2023-11-24T06:12:08.609293+0000","last_became_active":"2023-11-24T06:12:08.605594+0000","last_became_peered":"2023-11-24T06:12:08.605594+0000","last_unstale":"2023-11-24T06:12:08.609293+0000","last_undegraded":"2023-11-24T06:12:08.609293+0000","last_fullsized":"2023-11-24T06:12:08.609293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:15.801019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601990+0000","last_change":"2023-11-24T06:12:08.594594+0000","last_active":"2023-11-24T06:12:08.601990+0000","last_peered":"2023-11-24T06:12:08.601990+0000","last_clean":"2023-11-24T06:12:08.601990+0000","last_became_active":"2023-11-24T06:12:08.594289+0000","last_became_peered":"2023-11-24T06:12:08.594289+0000","last_unstale":"2023-11-24T06:12:08.601990+0000","last_undegraded":"2023-11-24T06:12:08.601990+0000","last_fullsized":"2023-11-24T06:12:08.601990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:54.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616292+0000","last_change":"2023-11-24T06:12:08.605733+0000","last_active":"2023-11-24T06:12:08.616292+0000","last_peered":"2023-11-24T06:12:08.616292+0000","last_clean":"2023-11-24T06:12:08.616292+0000","last_became_active":"2023-11-24T06:12:08.605448+0000","last_became_peered":"2023-11-24T06:12:08.605448+0000","last_unstale":"2023-11-24T06:12:08.616292+0000","last_undegraded":"2023-11-24T06:12:08.616292+0000","last_fullsized":"2023-11-24T06:12:08.616292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:19:56.537897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597888+0000","last_change":"2023-11-24T06:12:08.593304+0000","last_active":"2023-11-24T06:12:08.597888+0000","last_peered":"2023-11-24T06:12:08.597888+0000","last_clean":"2023-11-24T06:12:08.597888+0000","last_became_active":"2023-11-24T06:12:08.593037+0000","last_became_peered":"2023-11-24T06:12:08.593037+0000","last_unstale":"2023-11-24T06:12:08.597888+0000","last_undegraded":"2023-11-24T06:12:08.597888+0000","last_fullsized":"2023-11-24T06:12:08.597888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:00.716649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593637+0000","last_change":"2023-11-24T06:12:08.590634+0000","last_active":"2023-11-24T06:12:08.593637+0000","last_peered":"2023-11-24T06:12:08.593637+0000","last_clean":"2023-11-24T06:12:08.593637+0000","last_became_active":"2023-11-24T06:12:08.590369+0000","last_became_peered":"2023-11-24T06:12:08.590369+0000","last_unstale":"2023-11-24T06:12:08.593637+0000","last_undegraded":"2023-11-24T06:12:08.593637+0000","last_fullsized":"2023-11-24T06:12:08.593637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:52:59.601691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599113+0000","last_change":"2023-11-24T06:12:08.593687+0000","last_active":"2023-11-24T06:12:08.599113+0000","last_peered":"2023-11-24T06:12:08.599113+0000","last_clean":"2023-11-24T06:12:08.599113+0000","last_became_active":"2023-11-24T06:12:08.593415+0000","last_became_peered":"2023-11-24T06:12:08.593415+0000","last_unstale":"2023-11-24T06:12:08.599113+0000","last_undegraded":"2023-11-24T06:12:08.599113+0000","last_fullsized":"2023-11-24T06:12:08.599113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:28.388932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597625+0000","last_change":"2023-11-24T06:12:08.592618+0000","last_active":"2023-11-24T06:12:08.597625+0000","last_peered":"2023-11-24T06:12:08.597625+0000","last_clean":"2023-11-24T06:12:08.597625+0000","last_became_active":"2023-11-24T06:12:08.592361+0000","last_became_peered":"2023-11-24T06:12:08.592361+0000","last_unstale":"2023-11-24T06:12:08.597625+0000","last_undegraded":"2023-11-24T06:12:08.597625+0000","last_fullsized":"2023-11-24T06:12:08.597625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:54:23.514172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610013+0000","last_change":"2023-11-24T06:12:08.604842+0000","last_active":"2023-11-24T06:12:08.610013+0000","last_peered":"2023-11-24T06:12:08.610013+0000","last_clean":"2023-11-24T06:12:08.610013+0000","last_became_active":"2023-11-24T06:12:08.604484+0000","last_became_peered":"2023-11-24T06:12:08.604484+0000","last_unstale":"2023-11-24T06:12:08.610013+0000","last_undegraded":"2023-11-24T06:12:08.610013+0000","last_fullsized":"2023-11-24T06:12:08.610013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:38.195599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593572+0000","last_change":"2023-11-24T06:12:08.590609+0000","last_active":"2023-11-24T06:12:08.593572+0000","last_peered":"2023-11-24T06:12:08.593572+0000","last_clean":"2023-11-24T06:12:08.593572+0000","last_became_active":"2023-11-24T06:12:08.590342+0000","last_became_peered":"2023-11-24T06:12:08.590342+0000","last_unstale":"2023-11-24T06:12:08.593572+0000","last_undegraded":"2023-11-24T06:12:08.593572+0000","last_fullsized":"2023-11-24T06:12:08.593572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:33.825970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.613611+0000","last_change":"2023-11-24T06:12:08.609358+0000","last_active":"2023-11-24T06:12:08.613611+0000","last_peered":"2023-11-24T06:12:08.613611+0000","last_clean":"2023-11-24T06:12:08.613611+0000","last_became_active":"2023-11-24T06:12:08.609098+0000","last_became_peered":"2023-11-24T06:12:08.609098+0000","last_unstale":"2023-11-24T06:12:08.613611+0000","last_undegraded":"2023-11-24T06:12:08.613611+0000","last_fullsized":"2023-11-24T06:12:08.613611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:30:14.952893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.598820+0000","last_change":"2023-11-24T06:12:08.592525+0000","last_active":"2023-11-24T06:12:08.598820+0000","last_peered":"2023-11-24T06:12:08.598820+0000","last_clean":"2023-11-24T06:12:08.598820+0000","last_became_active":"2023-11-24T06:12:08.592205+0000","last_became_peered":"2023-11-24T06:12:08.592205+0000","last_unstale":"2023-11-24T06:12:08.598820+0000","last_undegraded":"2023-11-24T06:12:08.598820+0000","last_fullsized":"2023-11-24T06:12:08.598820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:00.716649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.602389+0000","last_change":"2023-11-24T06:12:08.597798+0000","last_active":"2023-11-24T06:12:08.602389+0000","last_peered":"2023-11-24T06:12:08.602389+0000","last_clean":"2023-11-24T06:12:08.602389+0000","last_became_active":"2023-11-24T06:12:08.597479+0000","last_became_peered":"2023-11-24T06:12:08.597479+0000","last_unstale":"2023-11-24T06:12:08.602389+0000","last_undegraded":"2023-11-24T06:12:08.602389+0000","last_fullsized":"2023-11-24T06:12:08.602389+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:18.532081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.611753+0000","last_change":"2023-11-24T06:12:08.606137+0000","last_active":"2023-11-24T06:12:08.611753+0000","last_peered":"2023-11-24T06:12:08.611753+0000","last_clean":"2023-11-24T06:12:08.611753+0000","last_became_active":"2023-11-24T06:12:08.605815+0000","last_became_peered":"2023-11-24T06:12:08.605815+0000","last_unstale":"2023-11-24T06:12:08.611753+0000","last_undegraded":"2023-11-24T06:12:08.611753+0000","last_fullsized":"2023-11-24T06:12:08.611753+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:56.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597364+0000","last_change":"2023-11-24T06:12:08.593815+0000","last_active":"2023-11-24T06:12:08.597364+0000","last_peered":"2023-11-24T06:12:08.597364+0000","last_clean":"2023-11-24T06:12:08.597364+0000","last_became_active":"2023-11-24T06:12:08.593522+0000","last_became_peered":"2023-11-24T06:12:08.593522+0000","last_unstale":"2023-11-24T06:12:08.597364+0000","last_undegraded":"2023-11-24T06:12:08.597364+0000","last_fullsized":"2023-11-24T06:12:08.597364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:35.391656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.595868+0000","last_change":"2023-11-24T06:12:08.589186+0000","last_active":"2023-11-24T06:12:08.595868+0000","last_peered":"2023-11-24T06:12:08.595868+0000","last_clean":"2023-11-24T06:12:08.595868+0000","last_became_active":"2023-11-24T06:12:08.588847+0000","last_became_peered":"2023-11-24T06:12:08.588847+0000","last_unstale":"2023-11-24T06:12:08.595868+0000","last_undegraded":"2023-11-24T06:12:08.595868+0000","last_fullsized":"2023-11-24T06:12:08.595868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:25.624664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599776+0000","last_change":"2023-11-24T06:12:08.594763+0000","last_active":"2023-11-24T06:12:08.599776+0000","last_peered":"2023-11-24T06:12:08.599776+0000","last_clean":"2023-11-24T06:12:08.599776+0000","last_became_active":"2023-11-24T06:12:08.594444+0000","last_became_peered":"2023-11-24T06:12:08.594444+0000","last_unstale":"2023-11-24T06:12:08.599776+0000","last_undegraded":"2023-11-24T06:12:08.599776+0000","last_fullsized":"2023-11-24T06:12:08.599776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:43:06.650930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610044+0000","last_change":"2023-11-24T06:12:08.605004+0000","last_active":"2023-11-24T06:12:08.610044+0000","last_peered":"2023-11-24T06:12:08.610044+0000","last_clean":"2023-11-24T06:12:08.610044+0000","last_became_active":"2023-11-24T06:12:08.604569+0000","last_became_peered":"2023-11-24T06:12:08.604569+0000","last_unstale":"2023-11-24T06:12:08.610044+0000","last_undegraded":"2023-11-24T06:12:08.610044+0000","last_fullsized":"2023-11-24T06:12:08.610044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:38.091512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.609327+0000","last_change":"2023-11-24T06:12:08.605922+0000","last_active":"2023-11-24T06:12:08.609327+0000","last_peered":"2023-11-24T06:12:08.609327+0000","last_clean":"2023-11-24T06:12:08.609327+0000","last_became_active":"2023-11-24T06:12:08.605683+0000","last_became_peered":"2023-11-24T06:12:08.605683+0000","last_unstale":"2023-11-24T06:12:08.609327+0000","last_undegraded":"2023-11-24T06:12:08.609327+0000","last_fullsized":"2023-11-24T06:12:08.609327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:11.145592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616530+0000","last_change":"2023-11-24T06:12:07.588966+0000","last_active":"2023-11-24T06:12:08.616530+0000","last_peered":"2023-11-24T06:12:08.616530+0000","last_clean":"2023-11-24T06:12:08.616530+0000","last_became_active":"2023-11-24T06:12:07.588693+0000","last_became_peered":"2023-11-24T06:12:07.588693+0000","last_unstale":"2023-11-24T06:12:08.616530+0000","last_undegraded":"2023-11-24T06:12:08.616530+0000","last_fullsized":"2023-11-24T06:12:08.616530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:44:44.699163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610185+0000","last_change":"2023-11-24T06:12:08.605831+0000","last_active":"2023-11-24T06:12:08.610185+0000","last_peered":"2023-11-24T06:12:08.610185+0000","last_clean":"2023-11-24T06:12:08.610185+0000","last_became_active":"2023-11-24T06:12:08.605392+0000","last_became_peered":"2023-11-24T06:12:08.605392+0000","last_unstale":"2023-11-24T06:12:08.610185+0000","last_undegraded":"2023-11-24T06:12:08.610185+0000","last_fullsized":"2023-11-24T06:12:08.610185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:54.143347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.609246+0000","last_change":"2023-11-24T06:12:07.584590+0000","last_active":"2023-11-24T06:12:08.609246+0000","last_peered":"2023-11-24T06:12:08.609246+0000","last_clean":"2023-11-24T06:12:08.609246+0000","last_became_active":"2023-11-24T06:12:07.584334+0000","last_became_peered":"2023-11-24T06:12:07.584334+0000","last_unstale":"2023-11-24T06:12:08.609246+0000","last_undegraded":"2023-11-24T06:12:08.609246+0000","last_fullsized":"2023-11-24T06:12:08.609246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:36.818741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.612267+0000","last_change":"2023-11-24T06:12:08.606797+0000","last_active":"2023-11-24T06:12:08.612267+0000","last_peered":"2023-11-24T06:12:08.612267+0000","last_clean":"2023-11-24T06:12:08.612267+0000","last_became_active":"2023-11-24T06:12:08.606472+0000","last_became_peered":"2023-11-24T06:12:08.606472+0000","last_unstale":"2023-11-24T06:12:08.612267+0000","last_undegraded":"2023-11-24T06:12:08.612267+0000","last_fullsized":"2023-11-24T06:12:08.612267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:07.836285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616646+0000","last_change":"2023-11-24T06:12:07.592666+0000","last_active":"2023-11-24T06:12:08.616646+0000","last_peered":"2023-11-24T06:12:08.616646+0000","last_clean":"2023-11-24T06:12:08.616646+0000","last_became_active":"2023-11-24T06:12:07.592449+0000","last_became_peered":"2023-11-24T06:12:07.592449+0000","last_unstale":"2023-11-24T06:12:08.616646+0000","last_undegraded":"2023-11-24T06:12:08.616646+0000","last_fullsized":"2023-11-24T06:12:08.616646+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:16.278761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600307+0000","last_change":"2023-11-24T06:12:08.592130+0000","last_active":"2023-11-24T06:12:08.600307+0000","last_peered":"2023-11-24T06:12:08.600307+0000","last_clean":"2023-11-24T06:12:08.600307+0000","last_became_active":"2023-11-24T06:12:08.591718+0000","last_became_peered":"2023-11-24T06:12:08.591718+0000","last_unstale":"2023-11-24T06:12:08.600307+0000","last_undegraded":"2023-11-24T06:12:08.600307+0000","last_fullsized":"2023-11-24T06:12:08.600307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:12.107805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616718+0000","last_change":"2023-11-24T06:12:07.589767+0000","last_active":"2023-11-24T06:12:08.616718+0000","last_peered":"2023-11-24T06:12:08.616718+0000","last_clean":"2023-11-24T06:12:08.616718+0000","last_became_active":"2023-11-24T06:12:07.589469+0000","last_became_peered":"2023-11-24T06:12:07.589469+0000","last_unstale":"2023-11-24T06:12:08.616718+0000","last_undegraded":"2023-11-24T06:12:08.616718+0000","last_fullsized":"2023-11-24T06:12:08.616718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:49.365378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.595850+0000","last_change":"2023-11-24T06:12:08.589131+0000","last_active":"2023-11-24T06:12:08.595850+0000","last_peered":"2023-11-24T06:12:08.595850+0000","last_clean":"2023-11-24T06:12:08.595850+0000","last_became_active":"2023-11-24T06:12:08.588752+0000","last_became_peered":"2023-11-24T06:12:08.588752+0000","last_unstale":"2023-11-24T06:12:08.595850+0000","last_undegraded":"2023-11-24T06:12:08.595850+0000","last_fullsized":"2023-11-24T06:12:08.595850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:16.262263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.608392+0000","last_change":"2023-11-24T06:12:07.590119+0000","last_active":"2023-11-24T06:12:08.608392+0000","last_peered":"2023-11-24T06:12:08.608392+0000","last_clean":"2023-11-24T06:12:08.608392+0000","last_became_active":"2023-11-24T06:12:07.589831+0000","last_became_peered":"2023-11-24T06:12:07.589831+0000","last_unstale":"2023-11-24T06:12:08.608392+0000","last_undegraded":"2023-11-24T06:12:08.608392+0000","last_fullsized":"2023-11-24T06:12:08.608392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:22.171468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600187+0000","last_change":"2023-11-24T06:12:08.594612+0000","last_active":"2023-11-24T06:12:08.600187+0000","last_peered":"2023-11-24T06:12:08.600187+0000","last_clean":"2023-11-24T06:12:08.600187+0000","last_became_active":"2023-11-24T06:12:08.594269+0000","last_became_peered":"2023-11-24T06:12:08.594269+0000","last_unstale":"2023-11-24T06:12:08.600187+0000","last_undegraded":"2023-11-24T06:12:08.600187+0000","last_fullsized":"2023-11-24T06:12:08.600187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:29.499761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.607024+0000","last_change":"2023-11-24T06:12:07.584244+0000","last_active":"2023-11-24T06:12:08.607024+0000","last_peered":"2023-11-24T06:12:08.607024+0000","last_clean":"2023-11-24T06:12:08.607024+0000","last_became_active":"2023-11-24T06:12:07.583933+0000","last_became_peered":"2023-11-24T06:12:07.583933+0000","last_unstale":"2023-11-24T06:12:08.607024+0000","last_undegraded":"2023-11-24T06:12:08.607024+0000","last_fullsized":"2023-11-24T06:12:08.607024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:11.492197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.598798+0000","last_change":"2023-11-24T06:12:08.592385+0000","last_active":"2023-11-24T06:12:08.598798+0000","last_peered":"2023-11-24T06:12:08.598798+0000","last_clean":"2023-11-24T06:12:08.598798+0000","last_became_active":"2023-11-24T06:12:08.592066+0000","last_became_peered":"2023-11-24T06:12:08.592066+0000","last_unstale":"2023-11-24T06:12:08.598798+0000","last_undegraded":"2023-11-24T06:12:08.598798+0000","last_fullsized":"2023-11-24T06:12:08.598798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:17.283186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.584838+0000","last_change":"2023-11-24T06:12:07.586038+0000","last_active":"2023-11-24T06:12:08.584838+0000","last_peered":"2023-11-24T06:12:08.584838+0000","last_clean":"2023-11-24T06:12:08.584838+0000","last_became_active":"2023-11-24T06:12:07.585777+0000","last_became_peered":"2023-11-24T06:12:07.585777+0000","last_unstale":"2023-11-24T06:12:08.584838+0000","last_undegraded":"2023-11-24T06:12:08.584838+0000","last_fullsized":"2023-11-24T06:12:08.584838+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:05.957392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.595450+0000","last_change":"2023-11-24T06:12:08.593209+0000","last_active":"2023-11-24T06:12:08.595450+0000","last_peered":"2023-11-24T06:12:08.595450+0000","last_clean":"2023-11-24T06:12:08.595450+0000","last_became_active":"2023-11-24T06:12:08.592628+0000","last_became_peered":"2023-11-24T06:12:08.592628+0000","last_unstale":"2023-11-24T06:12:08.595450+0000","last_undegraded":"2023-11-24T06:12:08.595450+0000","last_fullsized":"2023-11-24T06:12:08.595450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:39.740310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.584698+0000","last_change":"2023-11-24T06:12:07.584696+0000","last_active":"2023-11-24T06:12:08.584698+0000","last_peered":"2023-11-24T06:12:08.584698+0000","last_clean":"2023-11-24T06:12:08.584698+0000","last_became_active":"2023-11-24T06:12:07.584369+0000","last_became_peered":"2023-11-24T06:12:07.584369+0000","last_unstale":"2023-11-24T06:12:08.584698+0000","last_undegraded":"2023-11-24T06:12:08.584698+0000","last_fullsized":"2023-11-24T06:12:08.584698+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:59:42.227709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600263+0000","last_change":"2023-11-24T06:12:08.595547+0000","last_active":"2023-11-24T06:12:08.600263+0000","last_peered":"2023-11-24T06:12:08.600263+0000","last_clean":"2023-11-24T06:12:08.600263+0000","last_became_active":"2023-11-24T06:12:08.595285+0000","last_became_peered":"2023-11-24T06:12:08.595285+0000","last_unstale":"2023-11-24T06:12:08.600263+0000","last_undegraded":"2023-11-24T06:12:08.600263+0000","last_fullsized":"2023-11-24T06:12:08.600263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:29.035100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597392+0000","last_change":"2023-11-24T06:12:07.589536+0000","last_active":"2023-11-24T06:12:08.597392+0000","last_peered":"2023-11-24T06:12:08.597392+0000","last_clean":"2023-11-24T06:12:08.597392+0000","last_became_active":"2023-11-24T06:12:07.589262+0000","last_became_peered":"2023-11-24T06:12:07.589262+0000","last_unstale":"2023-11-24T06:12:08.597392+0000","last_undegraded":"2023-11-24T06:12:08.597392+0000","last_fullsized":"2023-11-24T06:12:08.597392+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:18.585330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600088+0000","last_change":"2023-11-24T06:12:08.595682+0000","last_active":"2023-11-24T06:12:08.600088+0000","last_peered":"2023-11-24T06:12:08.600088+0000","last_clean":"2023-11-24T06:12:08.600088+0000","last_became_active":"2023-11-24T06:12:08.595033+0000","last_became_peered":"2023-11-24T06:12:08.595033+0000","last_unstale":"2023-11-24T06:12:08.600088+0000","last_undegraded":"2023-11-24T06:12:08.600088+0000","last_fullsized":"2023-11-24T06:12:08.600088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:38.195599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.594348+0000","last_change":"2023-11-24T06:12:07.585545+0000","last_active":"2023-11-24T06:12:08.594348+0000","last_peered":"2023-11-24T06:12:08.594348+0000","last_clean":"2023-11-24T06:12:08.594348+0000","last_became_active":"2023-11-24T06:12:07.585140+0000","last_became_peered":"2023-11-24T06:12:07.585140+0000","last_unstale":"2023-11-24T06:12:08.594348+0000","last_undegraded":"2023-11-24T06:12:08.594348+0000","last_fullsized":"2023-11-24T06:12:08.594348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:16:22.171468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.615706+0000","last_change":"2023-11-24T06:12:08.607055+0000","last_active":"2023-11-24T06:12:08.615706+0000","last_peered":"2023-11-24T06:12:08.615706+0000","last_clean":"2023-11-24T06:12:08.615706+0000","last_became_active":"2023-11-24T06:12:08.606759+0000","last_became_peered":"2023-11-24T06:12:08.606759+0000","last_unstale":"2023-11-24T06:12:08.615706+0000","last_undegraded":"2023-11-24T06:12:08.615706+0000","last_fullsized":"2023-11-24T06:12:08.615706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:11.639234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.613868+0000","last_change":"2023-11-24T06:12:07.590857+0000","last_active":"2023-11-24T06:12:08.613868+0000","last_peered":"2023-11-24T06:12:08.613868+0000","last_clean":"2023-11-24T06:12:08.613868+0000","last_became_active":"2023-11-24T06:12:07.590530+0000","last_became_peered":"2023-11-24T06:12:07.590530+0000","last_unstale":"2023-11-24T06:12:08.613868+0000","last_undegraded":"2023-11-24T06:12:08.613868+0000","last_fullsized":"2023-11-24T06:12:08.613868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:54.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601887+0000","last_change":"2023-11-24T06:12:08.597714+0000","last_active":"2023-11-24T06:12:08.601887+0000","last_peered":"2023-11-24T06:12:08.601887+0000","last_clean":"2023-11-24T06:12:08.601887+0000","last_became_active":"2023-11-24T06:12:08.597152+0000","last_became_peered":"2023-11-24T06:12:08.597152+0000","last_unstale":"2023-11-24T06:12:08.601887+0000","last_undegraded":"2023-11-24T06:12:08.601887+0000","last_fullsized":"2023-11-24T06:12:08.601887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:55.350642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.613790+0000","last_change":"2023-11-24T06:12:07.583645+0000","last_active":"2023-11-24T06:12:08.613790+0000","last_peered":"2023-11-24T06:12:08.613790+0000","last_clean":"2023-11-24T06:12:08.613790+0000","last_became_active":"2023-11-24T06:12:07.583072+0000","last_became_peered":"2023-11-24T06:12:07.583072+0000","last_unstale":"2023-11-24T06:12:08.613790+0000","last_undegraded":"2023-11-24T06:12:08.613790+0000","last_fullsized":"2023-11-24T06:12:08.613790+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:10.141167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.598457+0000","last_change":"2023-11-24T06:12:08.596484+0000","last_active":"2023-11-24T06:12:08.598457+0000","last_peered":"2023-11-24T06:12:08.598457+0000","last_clean":"2023-11-24T06:12:08.598457+0000","last_became_active":"2023-11-24T06:12:08.596195+0000","last_became_peered":"2023-11-24T06:12:08.596195+0000","last_unstale":"2023-11-24T06:12:08.598457+0000","last_undegraded":"2023-11-24T06:12:08.598457+0000","last_fullsized":"2023-11-24T06:12:08.598457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:14.486732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616200+0000","last_change":"2023-11-24T06:12:07.590421+0000","last_active":"2023-11-24T06:12:08.616200+0000","last_peered":"2023-11-24T06:12:08.616200+0000","last_clean":"2023-11-24T06:12:08.616200+0000","last_became_active":"2023-11-24T06:12:07.590141+0000","last_became_peered":"2023-11-24T06:12:07.590141+0000","last_unstale":"2023-11-24T06:12:08.616200+0000","last_undegraded":"2023-11-24T06:12:08.616200+0000","last_fullsized":"2023-11-24T06:12:08.616200+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:29.089675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599596+0000","last_change":"2023-11-24T06:12:08.592977+0000","last_active":"2023-11-24T06:12:08.599596+0000","last_peered":"2023-11-24T06:12:08.599596+0000","last_clean":"2023-11-24T06:12:08.599596+0000","last_became_active":"2023-11-24T06:12:08.592671+0000","last_became_peered":"2023-11-24T06:12:08.592671+0000","last_unstale":"2023-11-24T06:12:08.599596+0000","last_undegraded":"2023-11-24T06:12:08.599596+0000","last_fullsized":"2023-11-24T06:12:08.599596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:16.576027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.582783+0000","last_change":"2023-11-24T06:12:07.590320+0000","last_active":"2023-11-24T06:12:08.582783+0000","last_peered":"2023-11-24T06:12:08.582783+0000","last_clean":"2023-11-24T06:12:08.582783+0000","last_became_active":"2023-11-24T06:12:07.590055+0000","last_became_peered":"2023-11-24T06:12:07.590055+0000","last_unstale":"2023-11-24T06:12:08.582783+0000","last_undegraded":"2023-11-24T06:12:08.582783+0000","last_fullsized":"2023-11-24T06:12:08.582783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:57.773893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.597511+0000","last_change":"2023-11-24T06:12:08.592222+0000","last_active":"2023-11-24T06:12:08.597511+0000","last_peered":"2023-11-24T06:12:08.597511+0000","last_clean":"2023-11-24T06:12:08.597511+0000","last_became_active":"2023-11-24T06:12:08.591943+0000","last_became_peered":"2023-11-24T06:12:08.591943+0000","last_unstale":"2023-11-24T06:12:08.597511+0000","last_undegraded":"2023-11-24T06:12:08.597511+0000","last_fullsized":"2023-11-24T06:12:08.597511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:09.055567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.590802+0000","last_change":"2023-11-24T06:12:07.582347+0000","last_active":"2023-11-24T06:12:08.590802+0000","last_peered":"2023-11-24T06:12:08.590802+0000","last_clean":"2023-11-24T06:12:08.590802+0000","last_became_active":"2023-11-24T06:12:07.582014+0000","last_became_peered":"2023-11-24T06:12:07.582014+0000","last_unstale":"2023-11-24T06:12:08.590802+0000","last_undegraded":"2023-11-24T06:12:08.590802+0000","last_fullsized":"2023-11-24T06:12:08.590802+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:54.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.608305+0000","last_change":"2023-11-24T06:12:08.605849+0000","last_active":"2023-11-24T06:12:08.608305+0000","last_peered":"2023-11-24T06:12:08.608305+0000","last_clean":"2023-11-24T06:12:08.608305+0000","last_became_active":"2023-11-24T06:12:08.604824+0000","last_became_peered":"2023-11-24T06:12:08.604824+0000","last_unstale":"2023-11-24T06:12:08.608305+0000","last_undegraded":"2023-11-24T06:12:08.608305+0000","last_fullsized":"2023-11-24T06:12:08.608305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:13:47.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.615942+0000","last_change":"2023-11-24T06:12:07.589310+0000","last_active":"2023-11-24T06:12:08.615942+0000","last_peered":"2023-11-24T06:12:08.615942+0000","last_clean":"2023-11-24T06:12:08.615942+0000","last_became_active":"2023-11-24T06:12:07.589018+0000","last_became_peered":"2023-11-24T06:12:07.589018+0000","last_unstale":"2023-11-24T06:12:08.615942+0000","last_undegraded":"2023-11-24T06:12:08.615942+0000","last_fullsized":"2023-11-24T06:12:08.615942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:21:23.200946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600328+0000","last_change":"2023-11-24T06:12:08.595654+0000","last_active":"2023-11-24T06:12:08.600328+0000","last_peered":"2023-11-24T06:12:08.600328+0000","last_clean":"2023-11-24T06:12:08.600328+0000","last_became_active":"2023-11-24T06:12:08.595394+0000","last_became_peered":"2023-11-24T06:12:08.595394+0000","last_unstale":"2023-11-24T06:12:08.600328+0000","last_undegraded":"2023-11-24T06:12:08.600328+0000","last_fullsized":"2023-11-24T06:12:08.600328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:50.369803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610120+0000","last_change":"2023-11-24T06:12:07.589046+0000","last_active":"2023-11-24T06:12:08.610120+0000","last_peered":"2023-11-24T06:12:08.610120+0000","last_clean":"2023-11-24T06:12:08.610120+0000","last_became_active":"2023-11-24T06:12:07.588761+0000","last_became_peered":"2023-11-24T06:12:07.588761+0000","last_unstale":"2023-11-24T06:12:08.610120+0000","last_undegraded":"2023-11-24T06:12:08.610120+0000","last_fullsized":"2023-11-24T06:12:08.610120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:12.484471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610165+0000","last_change":"2023-11-24T06:12:08.605654+0000","last_active":"2023-11-24T06:12:08.610165+0000","last_peered":"2023-11-24T06:12:08.610165+0000","last_clean":"2023-11-24T06:12:08.610165+0000","last_became_active":"2023-11-24T06:12:08.605035+0000","last_became_peered":"2023-11-24T06:12:08.605035+0000","last_unstale":"2023-11-24T06:12:08.610165+0000","last_undegraded":"2023-11-24T06:12:08.610165+0000","last_fullsized":"2023-11-24T06:12:08.610165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:36.334024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.615262+0000","last_change":"2023-11-24T06:12:07.590485+0000","last_active":"2023-11-24T06:12:08.615262+0000","last_peered":"2023-11-24T06:12:08.615262+0000","last_clean":"2023-11-24T06:12:08.615262+0000","last_became_active":"2023-11-24T06:12:07.590233+0000","last_became_peered":"2023-11-24T06:12:07.590233+0000","last_unstale":"2023-11-24T06:12:08.615262+0000","last_undegraded":"2023-11-24T06:12:08.615262+0000","last_fullsized":"2023-11-24T06:12:08.615262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:11.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601866+0000","last_change":"2023-11-24T06:12:08.597250+0000","last_active":"2023-11-24T06:12:08.601866+0000","last_peered":"2023-11-24T06:12:08.601866+0000","last_clean":"2023-11-24T06:12:08.601866+0000","last_became_active":"2023-11-24T06:12:08.596791+0000","last_became_peered":"2023-11-24T06:12:08.596791+0000","last_unstale":"2023-11-24T06:12:08.601866+0000","last_undegraded":"2023-11-24T06:12:08.601866+0000","last_fullsized":"2023-11-24T06:12:08.601866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:07.836285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.585361+0000","last_change":"2023-11-24T06:12:07.589623+0000","last_active":"2023-11-24T06:12:08.585361+0000","last_peered":"2023-11-24T06:12:08.585361+0000","last_clean":"2023-11-24T06:12:08.585361+0000","last_became_active":"2023-11-24T06:12:07.589182+0000","last_became_peered":"2023-11-24T06:12:07.589182+0000","last_unstale":"2023-11-24T06:12:08.585361+0000","last_undegraded":"2023-11-24T06:12:08.585361+0000","last_fullsized":"2023-11-24T06:12:08.585361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:15.571602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.599054+0000","last_change":"2023-11-24T06:12:08.592347+0000","last_active":"2023-11-24T06:12:08.599054+0000","last_peered":"2023-11-24T06:12:08.599054+0000","last_clean":"2023-11-24T06:12:08.599054+0000","last_became_active":"2023-11-24T06:12:08.591558+0000","last_became_peered":"2023-11-24T06:12:08.591558+0000","last_unstale":"2023-11-24T06:12:08.599054+0000","last_undegraded":"2023-11-24T06:12:08.599054+0000","last_fullsized":"2023-11-24T06:12:08.599054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:07:16.262263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.616256+0000","last_change":"2023-11-24T06:12:07.586220+0000","last_active":"2023-11-24T06:12:08.616256+0000","last_peered":"2023-11-24T06:12:08.616256+0000","last_clean":"2023-11-24T06:12:08.616256+0000","last_became_active":"2023-11-24T06:12:07.585925+0000","last_became_peered":"2023-11-24T06:12:07.585925+0000","last_unstale":"2023-11-24T06:12:08.616256+0000","last_undegraded":"2023-11-24T06:12:08.616256+0000","last_fullsized":"2023-11-24T06:12:08.616256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:15.548597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600045+0000","last_change":"2023-11-24T06:12:08.595032+0000","last_active":"2023-11-24T06:12:08.600045+0000","last_peered":"2023-11-24T06:12:08.600045+0000","last_clean":"2023-11-24T06:12:08.600045+0000","last_became_active":"2023-11-24T06:12:08.594772+0000","last_became_peered":"2023-11-24T06:12:08.594772+0000","last_unstale":"2023-11-24T06:12:08.600045+0000","last_undegraded":"2023-11-24T06:12:08.600045+0000","last_fullsized":"2023-11-24T06:12:08.600045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:22:15.556683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.580306+0000","last_change":"2023-11-24T06:12:07.591435+0000","last_active":"2023-11-24T06:12:08.580306+0000","last_peered":"2023-11-24T06:12:08.580306+0000","last_clean":"2023-11-24T06:12:08.580306+0000","last_became_active":"2023-11-24T06:12:07.591160+0000","last_became_peered":"2023-11-24T06:12:07.591160+0000","last_unstale":"2023-11-24T06:12:08.580306+0000","last_undegraded":"2023-11-24T06:12:08.580306+0000","last_fullsized":"2023-11-24T06:12:08.580306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:38.002561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.615337+0000","last_change":"2023-11-24T06:12:08.600097+0000","last_active":"2023-11-24T06:12:08.615337+0000","last_peered":"2023-11-24T06:12:08.615337+0000","last_clean":"2023-11-24T06:12:08.615337+0000","last_became_active":"2023-11-24T06:12:08.599512+0000","last_became_peered":"2023-11-24T06:12:08.599512+0000","last_unstale":"2023-11-24T06:12:08.615337+0000","last_undegraded":"2023-11-24T06:12:08.615337+0000","last_fullsized":"2023-11-24T06:12:08.615337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:21.299475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.579936+0000","last_change":"2023-11-24T06:12:07.585645+0000","last_active":"2023-11-24T06:12:08.579936+0000","last_peered":"2023-11-24T06:12:08.579936+0000","last_clean":"2023-11-24T06:12:08.579936+0000","last_became_active":"2023-11-24T06:12:07.585429+0000","last_became_peered":"2023-11-24T06:12:07.585429+0000","last_unstale":"2023-11-24T06:12:08.579936+0000","last_undegraded":"2023-11-24T06:12:08.579936+0000","last_fullsized":"2023-11-24T06:12:08.579936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:00:37.580115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.601552+0000","last_change":"2023-11-24T06:12:08.596908+0000","last_active":"2023-11-24T06:12:08.601552+0000","last_peered":"2023-11-24T06:12:08.601552+0000","last_clean":"2023-11-24T06:12:08.601552+0000","last_became_active":"2023-11-24T06:12:08.596652+0000","last_became_peered":"2023-11-24T06:12:08.596652+0000","last_unstale":"2023-11-24T06:12:08.601552+0000","last_undegraded":"2023-11-24T06:12:08.601552+0000","last_fullsized":"2023-11-24T06:12:08.601552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:31:07.836285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.582474+0000","last_change":"2023-11-24T06:12:07.586276+0000","last_active":"2023-11-24T06:12:08.582474+0000","last_peered":"2023-11-24T06:12:08.582474+0000","last_clean":"2023-11-24T06:12:08.582474+0000","last_became_active":"2023-11-24T06:12:07.586012+0000","last_became_peered":"2023-11-24T06:12:07.586012+0000","last_unstale":"2023-11-24T06:12:08.582474+0000","last_undegraded":"2023-11-24T06:12:08.582474+0000","last_fullsized":"2023-11-24T06:12:08.582474+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:35.156697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.606179+0000","last_change":"2023-11-24T06:12:08.597564+0000","last_active":"2023-11-24T06:12:08.606179+0000","last_peered":"2023-11-24T06:12:08.606179+0000","last_clean":"2023-11-24T06:12:08.606179+0000","last_became_active":"2023-11-24T06:12:08.597269+0000","last_became_peered":"2023-11-24T06:12:08.597269+0000","last_unstale":"2023-11-24T06:12:08.606179+0000","last_undegraded":"2023-11-24T06:12:08.606179+0000","last_fullsized":"2023-11-24T06:12:08.606179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:56.149751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610656+0000","last_change":"2023-11-24T06:12:07.589425+0000","last_active":"2023-11-24T06:12:08.610656+0000","last_peered":"2023-11-24T06:12:08.610656+0000","last_clean":"2023-11-24T06:12:08.610656+0000","last_became_active":"2023-11-24T06:12:07.589149+0000","last_became_peered":"2023-11-24T06:12:07.589149+0000","last_unstale":"2023-11-24T06:12:08.610656+0000","last_undegraded":"2023-11-24T06:12:08.610656+0000","last_fullsized":"2023-11-24T06:12:08.610656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:53:07.619654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.611946+0000","last_change":"2023-11-24T06:12:08.608600+0000","last_active":"2023-11-24T06:12:08.611946+0000","last_peered":"2023-11-24T06:12:08.611946+0000","last_clean":"2023-11-24T06:12:08.611946+0000","last_became_active":"2023-11-24T06:12:08.608347+0000","last_became_peered":"2023-11-24T06:12:08.608347+0000","last_unstale":"2023-11-24T06:12:08.611946+0000","last_undegraded":"2023-11-24T06:12:08.611946+0000","last_fullsized":"2023-11-24T06:12:08.611946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:27:06.961817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.583338+0000","last_change":"2023-11-24T06:12:07.591565+0000","last_active":"2023-11-24T06:12:08.583338+0000","last_peered":"2023-11-24T06:12:08.583338+0000","last_clean":"2023-11-24T06:12:08.583338+0000","last_became_active":"2023-11-24T06:12:07.591173+0000","last_became_peered":"2023-11-24T06:12:07.591173+0000","last_unstale":"2023-11-24T06:12:08.583338+0000","last_undegraded":"2023-11-24T06:12:08.583338+0000","last_fullsized":"2023-11-24T06:12:08.583338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:30.399381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610183+0000","last_change":"2023-11-24T06:12:08.605785+0000","last_active":"2023-11-24T06:12:08.610183+0000","last_peered":"2023-11-24T06:12:08.610183+0000","last_clean":"2023-11-24T06:12:08.610183+0000","last_became_active":"2023-11-24T06:12:08.605476+0000","last_became_peered":"2023-11-24T06:12:08.605476+0000","last_unstale":"2023-11-24T06:12:08.610183+0000","last_undegraded":"2023-11-24T06:12:08.610183+0000","last_fullsized":"2023-11-24T06:12:08.610183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:29.499761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593453+0000","last_change":"2023-11-24T06:12:07.587808+0000","last_active":"2023-11-24T06:12:08.593453+0000","last_peered":"2023-11-24T06:12:08.593453+0000","last_clean":"2023-11-24T06:12:08.593453+0000","last_became_active":"2023-11-24T06:12:07.587548+0000","last_became_peered":"2023-11-24T06:12:07.587548+0000","last_unstale":"2023-11-24T06:12:08.593453+0000","last_undegraded":"2023-11-24T06:12:08.593453+0000","last_fullsized":"2023-11-24T06:12:08.593453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:01:36.818741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600712+0000","last_change":"2023-11-24T06:12:08.595502+0000","last_active":"2023-11-24T06:12:08.600712+0000","last_peered":"2023-11-24T06:12:08.600712+0000","last_clean":"2023-11-24T06:12:08.600712+0000","last_became_active":"2023-11-24T06:12:08.595237+0000","last_became_peered":"2023-11-24T06:12:08.595237+0000","last_unstale":"2023-11-24T06:12:08.600712+0000","last_undegraded":"2023-11-24T06:12:08.600712+0000","last_fullsized":"2023-11-24T06:12:08.600712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:38.091512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593789+0000","last_change":"2023-11-24T06:12:07.587358+0000","last_active":"2023-11-24T06:12:08.593789+0000","last_peered":"2023-11-24T06:12:08.593789+0000","last_clean":"2023-11-24T06:12:08.593789+0000","last_became_active":"2023-11-24T06:12:07.587068+0000","last_became_peered":"2023-11-24T06:12:07.587068+0000","last_unstale":"2023-11-24T06:12:08.593789+0000","last_undegraded":"2023-11-24T06:12:08.593789+0000","last_fullsized":"2023-11-24T06:12:08.593789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:17.527656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600020+0000","last_change":"2023-11-24T06:12:08.595718+0000","last_active":"2023-11-24T06:12:08.600020+0000","last_peered":"2023-11-24T06:12:08.600020+0000","last_clean":"2023-11-24T06:12:08.600020+0000","last_became_active":"2023-11-24T06:12:08.594739+0000","last_became_peered":"2023-11-24T06:12:08.594739+0000","last_unstale":"2023-11-24T06:12:08.600020+0000","last_undegraded":"2023-11-24T06:12:08.600020+0000","last_fullsized":"2023-11-24T06:12:08.600020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:12.496622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.585408+0000","last_change":"2023-11-24T06:12:07.589985+0000","last_active":"2023-11-24T06:12:08.585408+0000","last_peered":"2023-11-24T06:12:08.585408+0000","last_clean":"2023-11-24T06:12:08.585408+0000","last_became_active":"2023-11-24T06:12:07.589718+0000","last_became_peered":"2023-11-24T06:12:07.589718+0000","last_unstale":"2023-11-24T06:12:08.585408+0000","last_undegraded":"2023-11-24T06:12:08.585408+0000","last_fullsized":"2023-11-24T06:12:08.585408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:51:56.017224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.600625+0000","last_change":"2023-11-24T06:12:08.597537+0000","last_active":"2023-11-24T06:12:08.600625+0000","last_peered":"2023-11-24T06:12:08.600625+0000","last_clean":"2023-11-24T06:12:08.600625+0000","last_became_active":"2023-11-24T06:12:08.597230+0000","last_became_peered":"2023-11-24T06:12:08.597230+0000","last_unstale":"2023-11-24T06:12:08.600625+0000","last_undegraded":"2023-11-24T06:12:08.600625+0000","last_fullsized":"2023-11-24T06:12:08.600625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:48.821384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.587890+0000","last_change":"2023-11-24T06:12:07.584235+0000","last_active":"2023-11-24T06:12:08.587890+0000","last_peered":"2023-11-24T06:12:08.587890+0000","last_clean":"2023-11-24T06:12:08.587890+0000","last_became_active":"2023-11-24T06:12:07.583915+0000","last_became_peered":"2023-11-24T06:12:07.583915+0000","last_unstale":"2023-11-24T06:12:08.587890+0000","last_undegraded":"2023-11-24T06:12:08.587890+0000","last_fullsized":"2023-11-24T06:12:08.587890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:35.329599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.598778+0000","last_change":"2023-11-24T06:12:08.592740+0000","last_active":"2023-11-24T06:12:08.598778+0000","last_peered":"2023-11-24T06:12:08.598778+0000","last_clean":"2023-11-24T06:12:08.598778+0000","last_became_active":"2023-11-24T06:12:08.592327+0000","last_became_peered":"2023-11-24T06:12:08.592327+0000","last_unstale":"2023-11-24T06:12:08.598778+0000","last_undegraded":"2023-11-24T06:12:08.598778+0000","last_fullsized":"2023-11-24T06:12:08.598778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:48.821384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.593106+0000","last_change":"2023-11-24T06:12:07.587426+0000","last_active":"2023-11-24T06:12:08.593106+0000","last_peered":"2023-11-24T06:12:08.593106+0000","last_clean":"2023-11-24T06:12:08.593106+0000","last_became_active":"2023-11-24T06:12:07.587151+0000","last_became_peered":"2023-11-24T06:12:07.587151+0000","last_unstale":"2023-11-24T06:12:08.593106+0000","last_undegraded":"2023-11-24T06:12:08.593106+0000","last_fullsized":"2023-11-24T06:12:08.593106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:06:10.141167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.609908+0000","last_change":"2023-11-24T06:12:08.602477+0000","last_active":"2023-11-24T06:12:08.609908+0000","last_peered":"2023-11-24T06:12:08.609908+0000","last_clean":"2023-11-24T06:12:08.609908+0000","last_became_active":"2023-11-24T06:12:08.601299+0000","last_became_peered":"2023-11-24T06:12:08.601299+0000","last_unstale":"2023-11-24T06:12:08.609908+0000","last_undegraded":"2023-11-24T06:12:08.609908+0000","last_fullsized":"2023-11-24T06:12:08.609908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:50:15.801019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.587573+0000","last_change":"2023-11-24T06:12:07.585625+0000","last_active":"2023-11-24T06:12:08.587573+0000","last_peered":"2023-11-24T06:12:08.587573+0000","last_clean":"2023-11-24T06:12:08.587573+0000","last_became_active":"2023-11-24T06:12:07.585257+0000","last_became_peered":"2023-11-24T06:12:07.585257+0000","last_unstale":"2023-11-24T06:12:08.587573+0000","last_undegraded":"2023-11-24T06:12:08.587573+0000","last_fullsized":"2023-11-24T06:12:08.587573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:37.191174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.606575+0000","last_change":"2023-11-24T06:12:08.597276+0000","last_active":"2023-11-24T06:12:08.606575+0000","last_peered":"2023-11-24T06:12:08.606575+0000","last_clean":"2023-11-24T06:12:08.606575+0000","last_became_active":"2023-11-24T06:12:08.596919+0000","last_became_peered":"2023-11-24T06:12:08.596919+0000","last_unstale":"2023-11-24T06:12:08.606575+0000","last_undegraded":"2023-11-24T06:12:08.606575+0000","last_fullsized":"2023-11-24T06:12:08.606575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:07.566405+0000","last_clean_scrub_stamp":"2023-11-24T06:12:07.566405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:26.973356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-11-24T06:12:08.610467+0000","last_change":"2023-11-24T06:12:07.590006+0000","last_active":"2023-11-24T06:12:08.610467+0000","last_peered":"2023-11-24T06:12:08.610467+0000","last_clean":"2023-11-24T06:12:08.610467+0000","last_became_active":"2023-11-24T06:12:07.589501+0000","last_became_peered":"2023-11-24T06:12:07.589501+0000","last_unstale":"2023-11-24T06:12:08.610467+0000","last_undegraded":"2023-11-24T06:12:08.610467+0000","last_fullsized":"2023-11-24T06:12:08.610467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:12:06.561979+0000","last_clean_scrub_stamp":"2023-11-24T06:12:06.561979+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:51:56.017224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":218,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-24T06:12:07.575317+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:12:07.575317+0000","last_peered":"2023-11-24T06:12:07.575317+0000","last_clean":"2023-11-24T06:12:07.575317+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:12:07.575317+0000","last_undegraded":"2023-11-24T06:12:07.575317+0000","last_fullsized":"2023-11-24T06:12:07.575317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28600,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343240,"statfs":{"total":96636764160,"available":96607477760,"internally_reserved":0,"allocated":1613824,"data_stored":1025496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.755,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.625},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.73599999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":435128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.71299999999999997,"15min":0.79300000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.75,"15min":0.86199999999999999},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68999999999999995,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.54200000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":1.371,"15min":1.371},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69099999999999995,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.76400000000000001,"15min":0.85299999999999998},"min":{"1min":0.38800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.629,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69899999999999995,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.499},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":1.407,"15min":1.407},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.439,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80600000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1044480,"data_stored":442125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33900000000000002,"15min":0.34499999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.38900000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"min":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.379},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.374,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.43099999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.624},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80600000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.79300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343224,"statfs":{"total":96636764160,"available":96607461376,"internally_reserved":0,"allocated":1630208,"data_stored":1032333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.624,"15min":0.75800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47899999999999998,"5min":12.378,"15min":12.378},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.45900000000000002,"15min":0.502},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.496,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.46700000000000003,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.48499999999999999,"15min":0.53000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.28999999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.46300000000000002,"15min":0.50600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52000000000000002,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56000000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.312}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.96099999999999997,"15min":1.125},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73299999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68300000000000005,"15min":0.72699999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82199999999999995,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69299999999999995,"15min":0.73299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.96799999999999997,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":15.356,"15min":15.356},"last":0.622}]},{"osd":6,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71499999999999997,"15min":0.76100000000000001},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":4.726,"15min":4.726},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.71999999999999997,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.997,"15min":1.165},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75,"5min":15.388,"15min":15.388},"last":0.65700000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":435128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32700000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.34399999999999997,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":1.5569999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.95399999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54200000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.173,"15min":1.173},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":4.5540000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.83599999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.92000000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.65900000000000003,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.63100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":1.417},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1044480,"data_stored":443543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.32700000000000001,"15min":0.33800000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.39200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.34999999999999998,"15min":0.36699999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.36399999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34499999999999997,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45700000000000002,"5min":1.032,"15min":1.032},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.252,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44400000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.435},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.48599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.622,"15min":0.65200000000000002},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64800000000000002,"5min":1.728,"15min":1.728},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.67100000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.626,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69199999999999995,"5min":1.304,"15min":1.304},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.044,"5min":1.222,"15min":1.222},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69199999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1032192,"data_stored":441931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.437,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.45500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.745,"15min":0.745},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.72299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":435128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.63200000000000001,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":3.335,"15min":3.335},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.622,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66900000000000004,"15min":0.72999999999999998},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71299999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.0419999999999998,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.76300000000000001},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.55500000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":1.536,"15min":1.536},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81999999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93700000000000006,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.61399999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90700000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.65500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:13.084 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:12:13.084 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:12:13.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.244+0000 7f1db7bdd700 1 Processor -- start 2023-11-24T06:12:13.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.244+0000 7f1db7bdd700 1 -- start start 2023-11-24T06:12:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db7bdd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1db015e160 0x7f1db00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db7bdd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1db00afc90 0x7f1db0073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db7bdd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db00744d0 0x7f1db006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db7bdd700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db00b4500 con 0x7f1db00afc90 2023-11-24T06:12:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db7bdd700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db0074960 con 0x7f1db015e160 2023-11-24T06:12:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db7bdd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b3c60 con 0x7f1db00744d0 2023-11-24T06:12:13.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1db00afc90 0x7f1db0073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1db00afc90 0x7f1db0073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51176/0 (socket says 172.21.15.97:51176) 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 -- 172.21.15.97:0/4204813743 learned_addr learned my addr 172.21.15.97:0/4204813743 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5979700 1 --2- 172.21.15.97:0/4204813743 >> v2:172.21.15.183:3300/0 conn(0x7f1db015e160 0x7f1db00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db617a700 1 --2- 172.21.15.97:0/4204813743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db00744d0 0x7f1db006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 -- 172.21.15.97:0/4204813743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db00744d0 msgr2=0x7f1db006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 --2- 172.21.15.97:0/4204813743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db00744d0 0x7f1db006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 -- 172.21.15.97:0/4204813743 >> v2:172.21.15.183:3300/0 conn(0x7f1db015e160 msgr2=0x7f1db00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 --2- 172.21.15.97:0/4204813743 >> v2:172.21.15.183:3300/0 conn(0x7f1db015e160 0x7f1db00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 -- 172.21.15.97:0/4204813743 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00b58c0 con 0x7f1db00afc90 2023-11-24T06:12:13.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5979700 1 --2- 172.21.15.97:0/4204813743 >> v2:172.21.15.183:3300/0 conn(0x7f1db015e160 0x7f1db00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db617a700 1 --2- 172.21.15.97:0/4204813743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db00744d0 0x7f1db006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db5178700 1 --2- 172.21.15.97:0/4204813743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00afc90 0x7f1db0073f90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1db009efa0 tx=0x7f1dac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b11571172cc575 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db4977700 1 -- 172.21.15.97:0/4204813743 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00b870 con 0x7f1db00afc90 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db4977700 1 -- 172.21.15.97:0/4204813743 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac00b9d0 con 0x7f1db00afc90 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.245+0000 7f1db4977700 1 -- 172.21.15.97:0/4204813743 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00bd60 con 0x7f1db00afc90 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 -- 172.21.15.97:0/4204813743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00afc90 msgr2=0x7f1db0073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/4204813743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00afc90 0x7f1db0073f90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1db009efa0 tx=0x7f1dac009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 -- 172.21.15.97:0/4204813743 shutdown_connections 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/4204813743 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db00744d0 0x7f1db006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/4204813743 >> v2:172.21.15.183:3300/0 conn(0x7f1db015e160 0x7f1db00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/4204813743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00afc90 0x7f1db0073f90 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 -- 172.21.15.97:0/4204813743 >> 172.21.15.97:0/4204813743 conn(0x7f1db00d0bf0 msgr2=0x7f1db00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 -- 172.21.15.97:0/4204813743 shutdown_connections 2023-11-24T06:12:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.246+0000 7f1db7bdd700 1 -- 172.21.15.97:0/4204813743 wait complete. 2023-11-24T06:12:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.247+0000 7f1db7bdd700 1 Processor -- start 2023-11-24T06:12:13.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.247+0000 7f1db7bdd700 1 -- start start 2023-11-24T06:12:13.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db7bdd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00744d0 0x7f1db013ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db7bdd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1db00afc90 0x7f1db00b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db7bdd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db015e160 0x7f1db01516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db7bdd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db01611d0 con 0x7f1db00744d0 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db7bdd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db013f4b0 con 0x7f1db00afc90 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db7bdd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b07c0 con 0x7f1db015e160 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5979700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00744d0 0x7f1db013ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5979700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00744d0 0x7f1db013ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51182/0 (socket says 172.21.15.97:51182) 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5979700 1 -- 172.21.15.97:0/2109590823 learned_addr learned my addr 172.21.15.97:0/2109590823 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5178700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1db00afc90 0x7f1db00b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db617a700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db015e160 0x7f1db01516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5979700 1 -- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db015e160 msgr2=0x7f1db01516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5979700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db015e160 0x7f1db01516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5979700 1 -- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1db00afc90 msgr2=0x7f1db00b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5979700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1db00afc90 0x7f1db00b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5979700 1 -- 172.21.15.97:0/2109590823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac009550 con 0x7f1db00744d0 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db5178700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1db00afc90 0x7f1db00b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.248+0000 7f1db617a700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db015e160 0x7f1db01516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.249+0000 7f1db5979700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00744d0 0x7f1db013ef70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c820 tx=0x7f1da000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.249+0000 7f1da67fc700 1 -- 172.21.15.97:0/2109590823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0011800 con 0x7f1db00744d0 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.249+0000 7f1da67fc700 1 -- 172.21.15.97:0/2109590823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da0011960 con 0x7f1db00744d0 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.249+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db00b0aa0 con 0x7f1db00744d0 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.249+0000 7f1da67fc700 1 -- 172.21.15.97:0/2109590823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0011d10 con 0x7f1db00744d0 2023-11-24T06:12:13.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.249+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db00b1060 con 0x7f1db00744d0 2023-11-24T06:12:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.250+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db0068fb0 con 0x7f1db00744d0 2023-11-24T06:12:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.250+0000 7f1da67fc700 1 -- 172.21.15.97:0/2109590823 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1da0018360 con 0x7f1db00744d0 2023-11-24T06:12:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.251+0000 7f1da67fc700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d9406b470 0x7f1d9406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.251+0000 7f1da67fc700 1 -- 172.21.15.97:0/2109590823 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1da008a110 con 0x7f1db00744d0 2023-11-24T06:12:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.251+0000 7f1db5178700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d9406b470 0x7f1d9406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.251+0000 7f1db5178700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d9406b470 0x7f1d9406d930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1db01525a0 tx=0x7f1dac008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.255+0000 7f1da67fc700 1 -- 172.21.15.97:0/2109590823 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1da00bce90 con 0x7f1db00744d0 2023-11-24T06:12:13.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.522+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1db015e540 con 0x7f1db00744d0 2023-11-24T06:12:13.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.524+0000 7f1da67fc700 1 -- 172.21.15.97:0/2109590823 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1da00582c0 con 0x7f1db00744d0 2023-11-24T06:12:13.528 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-11-24T06:12:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQC8PWBlLaT3OhAA+EuTqgrqxGPG0TX5DRLHaA== 2023-11-24T06:12:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: caps mds = "allow" 2023-11-24T06:12:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: caps mon = "allow r" 2023-11-24T06:12:13.528 INFO:teuthology.orchestra.run.smithi097.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d9406b470 msgr2=0x7f1d9406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d9406b470 0x7f1d9406d930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1db01525a0 tx=0x7f1dac008030 comp rx=0 tx=0).stop 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00744d0 msgr2=0x7f1db013ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00744d0 0x7f1db013ef70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c820 tx=0x7f1da000cbe0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 shutdown_connections 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1d9406b470 0x7f1d9406d930 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1db015e160 0x7f1db01516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1db00744d0 0x7f1db013ef70 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 --2- 172.21.15.97:0/2109590823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1db00afc90 0x7f1db00b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.531 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.526+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 >> 172.21.15.97:0/2109590823 conn(0x7f1db00d0bf0 msgr2=0x7f1db015d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.527+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 shutdown_connections 2023-11-24T06:12:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.527+0000 7f1db7bdd700 1 -- 172.21.15.97:0/2109590823 wait complete. 2023-11-24T06:12:13.532 INFO:teuthology.orchestra.run.smithi097.stderr:updated caps for client.0 2023-11-24T06:12:13.542 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:13.569 INFO:tasks.ceph.mds.d.smithi183.stderr:2023-11-24T06:12:13.566+0000 7fd9c58ac700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:12:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.659+0000 7faecb3d7700 1 Processor -- start 2023-11-24T06:12:13.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.659+0000 7faecb3d7700 1 -- start start 2023-11-24T06:12:13.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faecb3d7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faec4152cc0 0x7faec406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faecb3d7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7faec40b49a0 0x7faec406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faecb3d7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 0x7faec40d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faecb3d7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec415c4b0 con 0x7faec4152cc0 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faecb3d7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec415c790 con 0x7faec40b49a0 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faecb3d7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec415ca70 con 0x7faec40cff20 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faec4152cc0 0x7faec406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faec4152cc0 0x7faec406c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51190/0 (socket says 172.21.15.97:51190) 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 -- 172.21.15.97:0/2325804266 learned_addr learned my addr 172.21.15.97:0/2325804266 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec8972700 1 --2- 172.21.15.97:0/2325804266 >> v2:172.21.15.183:3300/0 conn(0x7faec40b49a0 0x7faec406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9974700 1 --2- 172.21.15.97:0/2325804266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 0x7faec40d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 -- 172.21.15.97:0/2325804266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 msgr2=0x7faec40d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:13.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 --2- 172.21.15.97:0/2325804266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 0x7faec40d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 -- 172.21.15.97:0/2325804266 >> v2:172.21.15.183:3300/0 conn(0x7faec40b49a0 msgr2=0x7faec406c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 --2- 172.21.15.97:0/2325804266 >> v2:172.21.15.183:3300/0 conn(0x7faec40b49a0 0x7faec406c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 -- 172.21.15.97:0/2325804266 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec415cd50 con 0x7faec4152cc0 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.660+0000 7faec9173700 1 --2- 172.21.15.97:0/2325804266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec4152cc0 0x7faec406c000 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7faec40b4e50 tx=0x7faec0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41a00cf9df1d488 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faebbfff700 1 -- 172.21.15.97:0/2325804266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec000b870 con 0x7faec4152cc0 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faebbfff700 1 -- 172.21.15.97:0/2325804266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec000b9d0 con 0x7faec4152cc0 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faebbfff700 1 -- 172.21.15.97:0/2325804266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec000bd60 con 0x7faec4152cc0 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 -- 172.21.15.97:0/2325804266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec4152cc0 msgr2=0x7faec406c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 --2- 172.21.15.97:0/2325804266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec4152cc0 0x7faec406c000 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7faec40b4e50 tx=0x7faec0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 -- 172.21.15.97:0/2325804266 shutdown_connections 2023-11-24T06:12:13.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 --2- 172.21.15.97:0/2325804266 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 0x7faec40d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 --2- 172.21.15.97:0/2325804266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec4152cc0 0x7faec406c000 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 --2- 172.21.15.97:0/2325804266 >> v2:172.21.15.183:3300/0 conn(0x7faec40b49a0 0x7faec406c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 -- 172.21.15.97:0/2325804266 >> 172.21.15.97:0/2325804266 conn(0x7faec4071b90 msgr2=0x7faec4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 -- 172.21.15.97:0/2325804266 shutdown_connections 2023-11-24T06:12:13.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.661+0000 7faecb3d7700 1 -- 172.21.15.97:0/2325804266 wait complete. 2023-11-24T06:12:13.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.662+0000 7faecb3d7700 1 Processor -- start 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.662+0000 7faecb3d7700 1 -- start start 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faecb3d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec40b49a0 0x7faec41f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faecb3d7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 0x7faec4154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faecb3d7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec4152cc0 0x7faec40b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faecb3d7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec4153c30 con 0x7faec40b49a0 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faecb3d7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec41532b0 con 0x7faec4152cc0 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faecb3d7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec41f2e50 con 0x7faec40cff20 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec9173700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec40b49a0 0x7faec41f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec9173700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec40b49a0 0x7faec41f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51196/0 (socket says 172.21.15.97:51196) 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec9173700 1 -- 172.21.15.97:0/3235606736 learned_addr learned my addr 172.21.15.97:0/3235606736 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec8972700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 0x7faec4154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec9974700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec4152cc0 0x7faec40b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec9173700 1 -- 172.21.15.97:0/3235606736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 msgr2=0x7faec4154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec9173700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 0x7faec4154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec9173700 1 -- 172.21.15.97:0/3235606736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec4152cc0 msgr2=0x7faec40b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.663+0000 7faec9173700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec4152cc0 0x7faec40b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.664+0000 7faec9173700 1 -- 172.21.15.97:0/3235606736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec0009550 con 0x7faec40b49a0 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.664+0000 7faec9173700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec40b49a0 0x7faec41f5d70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7faec0000c00 tx=0x7faec0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.664+0000 7faeb9ffb700 1 -- 172.21.15.97:0/3235606736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec0004520 con 0x7faec40b49a0 2023-11-24T06:12:13.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.664+0000 7faeb9ffb700 1 -- 172.21.15.97:0/3235606736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec001e070 con 0x7faec40b49a0 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.664+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec4153570 con 0x7faec40b49a0 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.664+0000 7faeb9ffb700 1 -- 172.21.15.97:0/3235606736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00038c0 con 0x7faec40b49a0 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.664+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec41f62b0 con 0x7faec40b49a0 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.665+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec40753f0 con 0x7faec40b49a0 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.665+0000 7faeb9ffb700 1 -- 172.21.15.97:0/3235606736 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7faec0003a20 con 0x7faec40b49a0 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.666+0000 7faeb9ffb700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faea806b470 0x7faea806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.666+0000 7faec8972700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faea806b470 0x7faea806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.666+0000 7faeb9ffb700 1 -- 172.21.15.97:0/3235606736 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7faec0020070 con 0x7faec40b49a0 2023-11-24T06:12:13.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.666+0000 7faec8972700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faea806b470 0x7faea806d930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faec415aa60 tx=0x7faeb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:13.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.670+0000 7faeb9ffb700 1 -- 172.21.15.97:0/3235606736 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faec0055e60 con 0x7faec40b49a0 2023-11-24T06:12:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.901+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faec4153d70 con 0x7faec40b49a0 2023-11-24T06:12:13.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.902+0000 7faeb9ffb700 1 -- 172.21.15.97:0/3235606736 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4921 (secure 0 0 0) 0x7faec4153d70 con 0x7faec40b49a0 2023-11-24T06:12:13.906 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:13.906 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6903,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3848239046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3848239046},{"type":"v1","addr":"172.21.15.97:6836","nonce":3848239046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3181686967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3181686967},{"type":"v1","addr":"172.21.15.97:6837","nonce":3181686967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/4023096288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4023096288},{"type":"v1","addr":"172.21.15.183:6834","nonce":4023096288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:07.978316+0000","modified":"2023-11-24T06:12:11.003557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6921},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3868240464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3868240464},{"type":"v1","addr":"172.21.15.183:6835","nonce":3868240464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:12:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faea806b470 msgr2=0x7faea806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faea806b470 0x7faea806d930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7faec415aa60 tx=0x7faeb4008040 comp rx=0 tx=0).stop 2023-11-24T06:12:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec40b49a0 msgr2=0x7faec41f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec40b49a0 0x7faec41f5d70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7faec0000c00 tx=0x7faec0004750 comp rx=0 tx=0).stop 2023-11-24T06:12:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 shutdown_connections 2023-11-24T06:12:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faec4152cc0 0x7faec40b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec40b49a0 0x7faec41f5d70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7faea806b470 0x7faea806d930 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 --2- 172.21.15.97:0/3235606736 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7faec40cff20 0x7faec4154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 >> 172.21.15.97:0/3235606736 conn(0x7faec4071b90 msgr2=0x7faec40b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 shutdown_connections 2023-11-24T06:12:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:13.904+0000 7faecb3d7700 1 -- 172.21.15.97:0/3235606736 wait complete. 2023-11-24T06:12:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 55 2023-11-24T06:12:14.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:12:07.978316+0000', 'modified': '2023-11-24T06:12:11.003557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6921}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6921': {'gid': 6921, 'name': 'd', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6835/3868240464', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6833', 'nonce': 3868240464}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 3868240464}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:12:14.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:12:14.323 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-24T06:12:14.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.442+0000 7f4ddd610700 1 Processor -- start 2023-11-24T06:12:14.446 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.442+0000 7f4ddd610700 1 -- start start 2023-11-24T06:12:14.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4ddd610700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4dd80b2b60 0x7f4dd80ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4ddd610700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd80cf0d0 0x7f4dd80d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4ddd610700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd806bc00 con 0x7f4dd815e0f0 2023-11-24T06:12:14.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4ddd610700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd80cf560 con 0x7f4dd80b2b60 2023-11-24T06:12:14.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4ddd610700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd815e4d0 con 0x7f4dd80cf0d0 2023-11-24T06:12:14.449 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd6ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4dd815e0f0 0x7f4dd80b9700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43656/0 (socket says 172.21.15.97:43656) 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd67fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4dd80b2b60 0x7f4dd80ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd6ffd700 1 -- 172.21.15.97:0/1924032719 learned_addr learned my addr 172.21.15.97:0/1924032719 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/1924032719 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd80cf0d0 0x7f4dd80d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd5ffb700 1 -- 172.21.15.97:0/1924032719 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122169750 0 0) 0x7f4dd806bc00 con 0x7f4dd815e0f0 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd5ffb700 1 -- 172.21.15.97:0/1924032719 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dc00035b0 con 0x7f4dd815e0f0 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd67fc700 1 -- 172.21.15.97:0/1924032719 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd80cf0d0 msgr2=0x7f4dd80d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/1924032719 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd80cf0d0 0x7f4dd80d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd67fc700 1 -- 172.21.15.97:0/1924032719 >> v1:172.21.15.97:6789/0 conn(0x7f4dd815e0f0 legacy=0x7f4dd80b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd67fc700 1 -- 172.21.15.97:0/1924032719 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd806aff0 con 0x7f4dd80b2b60 2023-11-24T06:12:14.450 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/1924032719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd80ceb90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009840 tx=0x7f4dc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8dbbd24615124dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.443+0000 7f4dd5ffb700 1 -- 172.21.15.97:0/1924032719 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8011830 con 0x7f4dd80b2b60 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4dd5ffb700 1 -- 172.21.15.97:0/1924032719 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc8011990 con 0x7f4dd80b2b60 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4dd5ffb700 1 -- 172.21.15.97:0/1924032719 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8011d20 con 0x7f4dd80b2b60 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4ddd610700 1 -- 172.21.15.97:0/1924032719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 msgr2=0x7f4dd80ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4ddd610700 1 --2- 172.21.15.97:0/1924032719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd80ceb90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009840 tx=0x7f4dc8009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4ddd610700 1 -- 172.21.15.97:0/1924032719 shutdown_connections 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4ddd610700 1 --2- 172.21.15.97:0/1924032719 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd80cf0d0 0x7f4dd80d1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4ddd610700 1 --2- 172.21.15.97:0/1924032719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd80ceb90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4ddd610700 1 -- 172.21.15.97:0/1924032719 >> 172.21.15.97:0/1924032719 conn(0x7f4dd80b5ce0 msgr2=0x7f4dd80b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4ddd610700 1 -- 172.21.15.97:0/1924032719 shutdown_connections 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.444+0000 7f4ddd610700 1 -- 172.21.15.97:0/1924032719 wait complete. 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.445+0000 7f4ddd610700 1 Processor -- start 2023-11-24T06:12:14.451 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.445+0000 7f4ddd610700 1 -- start start 2023-11-24T06:12:14.454 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4ddd610700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd8070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4ddd610700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4dd80cf0d0 0x7f4dd80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4ddd610700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd815e0f0 0x7f4dd815b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4ddd610700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd81f88b0 con 0x7f4dd80b2b60 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4ddd610700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80cbae0 con 0x7f4dd80cf0d0 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4ddd610700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80cc2d0 con 0x7f4dd815e0f0 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4dd6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd8070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4dd6ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd8070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51210/0 (socket says 172.21.15.97:51210) 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4dd6ffd700 1 -- 172.21.15.97:0/3289547360 learned_addr learned my addr 172.21.15.97:0/3289547360 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4dd80cf0d0 0x7f4dd80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.446+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd815e0f0 0x7f4dd815b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dd6ffd700 1 -- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd815e0f0 msgr2=0x7f4dd815b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.455 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd815e0f0 0x7f4dd815b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dd6ffd700 1 -- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4dd80cf0d0 msgr2=0x7f4dd80cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4dd80cf0d0 0x7f4dd80cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dd6ffd700 1 -- 172.21.15.97:0/3289547360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc80094a0 con 0x7f4dd80b2b60 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd815e0f0 0x7f4dd815b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd8070f80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc001eb0 tx=0x7f4dcc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dbf7fe700 1 -- 172.21.15.97:0/3289547360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc005d70 con 0x7f4dd80b2b60 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dbf7fe700 1 -- 172.21.15.97:0/3289547360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc0070d0 con 0x7f4dd80b2b60 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4ddd610700 1 -- 172.21.15.97:0/3289547360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd815b8a0 con 0x7f4dd80b2b60 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4dbf7fe700 1 -- 172.21.15.97:0/3289547360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc007460 con 0x7f4dd80b2b60 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.447+0000 7f4ddd610700 1 -- 172.21.15.97:0/3289547360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd80cbd10 con 0x7f4dd80b2b60 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.448+0000 7f4dbf7fe700 1 -- 172.21.15.97:0/3289547360 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4dcc0075c0 con 0x7f4dd80b2b60 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.449+0000 7f4dbf7fe700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4dc406b470 0x7f4dc406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.449+0000 7f4dbf7fe700 1 -- 172.21.15.97:0/3289547360 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4dcc089490 con 0x7f4dd80b2b60 2023-11-24T06:12:14.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.449+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4dc406b470 0x7f4dc406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.449+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4dc406b470 0x7f4dc406d930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8000c00 tx=0x7f4dc8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.451+0000 7f4de9809b80 1 Processor -- start 2023-11-24T06:12:14.457 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.451+0000 7f4de9809b80 1 -- start start 2023-11-24T06:12:14.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cd1080 0x55b763d17840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d17d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cce490 0x55b763cfae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4de9809b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763c44770 con 0x55b763ce6ed0 2023-11-24T06:12:14.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4de9809b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763c2b750 con 0x55b763cce490 2023-11-24T06:12:14.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4de9809b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b763c29070 con 0x55b763cd1080 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d17d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d17d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51220/0 (socket says 172.21.15.97:51220) 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 -- 172.21.15.97:0/4220240179 learned_addr learned my addr 172.21.15.97:0/4220240179 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 -- 172.21.15.97:0/4220240179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cd1080 msgr2=0x55b763d17840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/4220240179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cd1080 0x55b763d17840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 -- 172.21.15.97:0/4220240179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cce490 msgr2=0x55b763cfae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/4220240179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cce490 0x55b763cfae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 -- 172.21.15.97:0/4220240179 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b763aa15e0 con 0x55b763ce6ed0 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/4220240179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d17d80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4dd808c570 tx=0x7f4dc8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df414995718a3b35 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dbcff9700 1 -- 172.21.15.97:0/4220240179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc802c070 con 0x55b763ce6ed0 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dbcff9700 1 -- 172.21.15.97:0/4220240179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc80036a0 con 0x55b763ce6ed0 2023-11-24T06:12:14.464 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.452+0000 7f4dbcff9700 1 -- 172.21.15.97:0/4220240179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8020070 con 0x55b763ce6ed0 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 -- 172.21.15.97:0/4220240179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 msgr2=0x55b763d17d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4220240179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d17d80 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4dd808c570 tx=0x7f4dc8010000 comp rx=0 tx=0).stop 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 -- 172.21.15.97:0/4220240179 shutdown_connections 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4220240179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cce490 0x55b763cfae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4220240179 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cd1080 0x55b763d17840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4220240179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d17d80 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 -- 172.21.15.97:0/4220240179 >> 172.21.15.97:0/4220240179 conn(0x55b763bff310 msgr2=0x55b763d15e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 -- 172.21.15.97:0/4220240179 shutdown_connections 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.453+0000 7f4de9809b80 1 -- 172.21.15.97:0/4220240179 wait complete. 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 1 -- start start 2023-11-24T06:12:14.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 1 Processor -- start 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763cfe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 0x55b763cc5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763cc5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763c24960 con 0x55b763ce6ed0 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763bde510 con 0x55b763cd1080 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b763bde1b0 con 0x55b763cce490 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.454+0000 7f4de9809b80 10 client.0 _ll_register_callbacks cb 0x55b763b09a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763cc5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763cc5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51228/0 (socket says 172.21.15.97:51228) 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 -- 172.21.15.97:0/4137994264 learned_addr learned my addr 172.21.15.97:0/4137994264 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 0x55b763cc5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763cfe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 -- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 msgr2=0x55b763cfe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763cfe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 -- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 msgr2=0x55b763cc5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 0x55b763cc5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 -- 172.21.15.97:0/4137994264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd806aff0 con 0x55b763ce6ed0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763cc5870 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0007990 tx=0x7f4dd0007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4914a0dc2e486527 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4db5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4137994264 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00147e0 con 0x55b763ce6ed0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4137994264 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4dd0014940 con 0x55b763ce6ed0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4de9809b80 1 -- 172.21.15.97:0/4137994264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b763cc3a40 con 0x55b763ce6ed0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4137994264 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd0009a70 con 0x55b763ce6ed0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4de9809b80 10 client.7053 fetch_fsmap learned FSMap version 55 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4de9809b80 1 -- 172.21.15.97:0/4137994264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55b763a4dd90 con 0x55b763ce6ed0 2023-11-24T06:12:14.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.455+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4137994264 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b763cc3a40 con 0x55b763ce6ed0 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4137994264 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4dd0014b50 con 0x55b763ce6ed0 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4de9809b80 10 client.7053 fetch_fsmap finished waiting for FSMap version 55 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4de9809b80 10 client.7053 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4de9809b80 20 client.7053 populate_metadata read hostname 'smithi097' 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] conn(0x55b763cfb210 0x55b763d0f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4de9809b80 4 client.7053 mds_command: new command op to 6921 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4de9809b80 1 -- 172.21.15.97:0/4137994264 --> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b763c9f840 con 0x55b763cfb210 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] conn(0x55b763cfb210 0x55b763d0f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4db5ffb700 10 client.7053 ms_handle_connect on v2:172.21.15.183:6833/3868240464 2023-11-24T06:12:14.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.456+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] conn(0x55b763cfb210 0x55b763d0f1a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1abb1a4178af86e server_cookie=2a90c6a46381905 in_seq=0 out_seq=0 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.457+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4137994264 <== mds.0 v2:172.21.15.183:6833/3868240464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b763c9f840 con 0x55b763cfb210 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.457+0000 7f4db5ffb700 10 client.7053 handle_command_reply: tid=0 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.457+0000 7f4de9809b80 1 client.7053 shutdown 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.457+0000 7f4de9809b80 1 -- 172.21.15.97:0/4137994264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 msgr2=0x55b763cc5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.457+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763cc5870 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0007990 tx=0x7f4dd0007ca0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.457+0000 7f4de9809b80 1 -- 172.21.15.97:0/4137994264 shutdown_connections 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.457+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] conn(0x55b763cfb210 0x55b763d0f1a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.458+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763cc5870 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.458+0000 7f4db5ffb700 0 client.7053 ms_handle_reset on v2:172.21.15.183:6833/3868240464 2023-11-24T06:12:14.469 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.458+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763cfe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.458+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4137994264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 0x55b763cc5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.458+0000 7f4de9809b80 1 -- 172.21.15.97:0/4137994264 >> 172.21.15.97:0/4137994264 conn(0x55b763bff310 msgr2=0x55b763d19c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.458+0000 7f4de9809b80 1 -- 172.21.15.97:0/4137994264 shutdown_connections 2023-11-24T06:12:14.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.458+0000 7f4de9809b80 1 -- 172.21.15.97:0/4137994264 wait complete. 2023-11-24T06:12:14.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.458+0000 7f4de9809b80 20 client.7053 trim_cache size 0 max 16384 2023-11-24T06:12:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.476+0000 7f4de9809b80 1 Processor -- start 2023-11-24T06:12:14.479 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.476+0000 7f4de9809b80 1 -- start start 2023-11-24T06:12:14.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763d0b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763cd1080 0x55b763cc4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763ce6ed0 0x55b763cc51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4de9809b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763aa38d0 con 0x55b763cd1080 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4de9809b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763d184d0 con 0x55b763ce6ed0 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4de9809b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b763cc39f0 con 0x55b763cce490 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763cd1080 0x55b763cc4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763cd1080 0x55b763cc4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51232/0 (socket says 172.21.15.97:51232) 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 -- 172.21.15.97:0/422076191 learned_addr learned my addr 172.21.15.97:0/422076191 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763d0b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763ce6ed0 0x55b763cc51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 -- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 msgr2=0x55b763d0b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.490 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763d0b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 -- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763ce6ed0 msgr2=0x55b763cc51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763ce6ed0 0x55b763cc51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 -- 172.21.15.97:0/422076191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b763a36c10 con 0x55b763cd1080 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763ce6ed0 0x55b763cc51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763cd1080 0x55b763cc4cb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8011c90 tx=0x7f4dc8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e926c0ce3c0b12 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dbd7fa700 1 -- 172.21.15.97:0/422076191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc802c040 con 0x55b763cd1080 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dbd7fa700 1 -- 172.21.15.97:0/422076191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc8003e00 con 0x55b763cd1080 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.477+0000 7f4dbd7fa700 1 -- 172.21.15.97:0/422076191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8031670 con 0x55b763cd1080 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 -- 172.21.15.97:0/422076191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763cd1080 msgr2=0x55b763cc4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763cd1080 0x55b763cc4cb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8011c90 tx=0x7f4dc8020000 comp rx=0 tx=0).stop 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 -- 172.21.15.97:0/422076191 shutdown_connections 2023-11-24T06:12:14.491 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763ce6ed0 0x55b763cc51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763d0b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 --2- 172.21.15.97:0/422076191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763cd1080 0x55b763cc4cb0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 -- 172.21.15.97:0/422076191 >> 172.21.15.97:0/422076191 conn(0x55b763bff310 msgr2=0x55b763ccb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 -- 172.21.15.97:0/422076191 shutdown_connections 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 -- 172.21.15.97:0/422076191 wait complete. 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 -- start start 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.478+0000 7f4de9809b80 1 Processor -- start 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763d7e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 0x55b763d0b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4de9809b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d0bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4de9809b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763d29a50 con 0x55b763ce6ed0 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4de9809b80 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b763d7f130 con 0x55b763cd1080 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4de9809b80 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b763d19c80 con 0x55b763cce490 2023-11-24T06:12:14.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4de9809b80 10 client.0 _ll_register_callbacks cb 0x55b763b09a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4dd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d0bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4dd77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d0bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51242/0 (socket says 172.21.15.97:51242) 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4dd77fe700 1 -- 172.21.15.97:0/4660329 learned_addr learned my addr 172.21.15.97:0/4660329 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763d7e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.479+0000 7f4dd67fc700 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 0x55b763d0b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4dd77fe700 1 -- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 msgr2=0x55b763d7e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763d7e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4dd77fe700 1 -- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 msgr2=0x55b763d0b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 0x55b763d0b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4dd77fe700 1 -- 172.21.15.97:0/4660329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b763aa15e0 con 0x55b763ce6ed0 2023-11-24T06:12:14.493 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4dd77fe700 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d0bf30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000f2c0 tx=0x7f4dd0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f268153956c69 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4db5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4660329 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd00149f0 con 0x55b763ce6ed0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4660329 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4dd0014b50 con 0x55b763ce6ed0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 1 -- 172.21.15.97:0/4660329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dc80036a0 con 0x55b763ce6ed0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4660329 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dd0016420 con 0x55b763ce6ed0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 10 client.7059 fetch_fsmap learned FSMap version 55 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 1 -- 172.21.15.97:0/4660329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x55b763a74200 con 0x55b763ce6ed0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4660329 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc80036a0 con 0x55b763ce6ed0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4660329 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 55) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4dd0016580 con 0x55b763ce6ed0 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 10 client.7059 fetch_fsmap finished waiting for FSMap version 55 2023-11-24T06:12:14.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 10 client.7059 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 20 client.7059 populate_metadata read hostname 'smithi097' 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] conn(0x55b763cfb210 0x55b763d14440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 4 client.7059 mds_command: new command op to 6921 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.480+0000 7f4de9809b80 1 -- 172.21.15.97:0/4660329 --> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b763beaf80 con 0x55b763cfb210 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] conn(0x55b763cfb210 0x55b763d14440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4db5ffb700 10 client.7059 ms_handle_connect on v2:172.21.15.183:6833/3868240464 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4dd6ffd700 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] conn(0x55b763cfb210 0x55b763d14440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c2127ce7900472 server_cookie=67c72eda6c8f0d5b in_seq=0 out_seq=0 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4db5ffb700 1 -- 172.21.15.97:0/4660329 <== mds.0 v2:172.21.15.183:6833/3868240464 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b763beaf80 con 0x55b763cfb210 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4db5ffb700 10 client.7059 handle_command_reply: tid=0 2023-11-24T06:12:14.495 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 client.7059 shutdown 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 -- 172.21.15.97:0/4660329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 msgr2=0x55b763d0bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d0bf30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4dd000f2c0 tx=0x7f4dd0004ad0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 -- 172.21.15.97:0/4660329 shutdown_connections 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:6833/3868240464,v1:172.21.15.183:6835/3868240464] conn(0x55b763cfb210 0x55b763d14440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b763ce6ed0 0x55b763d0bf30 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4db5ffb700 0 client.7059 ms_handle_reset on v2:172.21.15.183:6833/3868240464 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x55b763cce490 0x55b763d7e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 --2- 172.21.15.97:0/4660329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b763cd1080 0x55b763d0b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 -- 172.21.15.97:0/4660329 >> 172.21.15.97:0/4660329 conn(0x55b763bff310 msgr2=0x55b763ccbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 -- 172.21.15.97:0/4660329 shutdown_connections 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.481+0000 7f4de9809b80 1 -- 172.21.15.97:0/4660329 wait complete. 2023-11-24T06:12:14.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4de9809b80 20 client.7059 trim_cache size 0 max 16384 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9", 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 6921, 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 5.8921989459999997, 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 55, 2023-11-24T06:12:14.497 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 85, 2023-11-24T06:12:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 85, 2023-11-24T06:12:14.498 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 11.830328227000001 2023-11-24T06:12:14.498 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 -- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4dc406b470 msgr2=0x7f4dc406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4dc406b470 0x7f4dc406d930 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8000c00 tx=0x7f4dc8008030 comp rx=0 tx=0).stop 2023-11-24T06:12:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 -- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 msgr2=0x7f4dd8070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd8070f80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc001eb0 tx=0x7f4dcc00c7b0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 -- 172.21.15.97:0/3289547360 shutdown_connections 2023-11-24T06:12:14.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4dc406b470 0x7f4dc406d930 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4dd815e0f0 0x7f4dd815b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dd80b2b60 0x7f4dd8070f80 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 --2- 172.21.15.97:0/3289547360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4dd80cf0d0 0x7f4dd80cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 -- 172.21.15.97:0/3289547360 >> 172.21.15.97:0/3289547360 conn(0x7f4dd80b5ce0 msgr2=0x7f4dd80b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 -- 172.21.15.97:0/3289547360 shutdown_connections 2023-11-24T06:12:14.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.482+0000 7f4ddd610700 1 -- 172.21.15.97:0/3289547360 wait complete. 2023-11-24T06:12:14.500 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:12:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.617+0000 7f79f8ba5700 1 Processor -- start 2023-11-24T06:12:14.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.617+0000 7f79f8ba5700 1 -- start start 2023-11-24T06:12:14.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f8ba5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f79f40b3370 0x7f79f41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f8ba5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79f40d1c70 0x7f79f40d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.622 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f8ba5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f40b22d0 con 0x7f79f415bdd0 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f8ba5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40b2f50 con 0x7f79f40d1c70 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f8ba5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40d2640 con 0x7f79f40b3370 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f2d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79f40d1c70 0x7f79f40d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f259c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f79f415bdd0 0x7f79f40b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43698/0 (socket says 172.21.15.97:43698) 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f259c700 1 -- 172.21.15.97:0/3663828773 learned_addr learned my addr 172.21.15.97:0/3663828773 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f1d9b700 1 --2- 172.21.15.97:0/3663828773 >> v2:172.21.15.183:3300/0 conn(0x7f79f40b3370 0x7f79f41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f2d9d700 1 -- 172.21.15.97:0/3663828773 >> v2:172.21.15.183:3300/0 conn(0x7f79f40b3370 msgr2=0x7f79f41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f2d9d700 1 --2- 172.21.15.97:0/3663828773 >> v2:172.21.15.183:3300/0 conn(0x7f79f40b3370 0x7f79f41604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.623 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f2d9d700 1 -- 172.21.15.97:0/3663828773 >> v1:172.21.15.97:6789/0 conn(0x7f79f415bdd0 legacy=0x7f79f40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f159a700 1 -- 172.21.15.97:0/3663828773 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249905926 0 0) 0x7f79f40b22d0 con 0x7f79f415bdd0 2023-11-24T06:12:14.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.618+0000 7f79f2d9d700 1 -- 172.21.15.97:0/3663828773 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40b25b0 con 0x7f79f40d1c70 2023-11-24T06:12:14.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.619+0000 7f79f2d9d700 1 --2- 172.21.15.97:0/3663828773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f40d2100 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f79f415f910 tx=0x7f79ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2aa5412ce7babc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.619+0000 7f79f159a700 1 -- 172.21.15.97:0/3663828773 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00e070 con 0x7f79f40d1c70 2023-11-24T06:12:14.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.619+0000 7f79f159a700 1 -- 172.21.15.97:0/3663828773 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec00bbd0 con 0x7f79f40d1c70 2023-11-24T06:12:14.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.619+0000 7f79f159a700 1 -- 172.21.15.97:0/3663828773 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec012430 con 0x7f79f40d1c70 2023-11-24T06:12:14.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.619+0000 7f79f8ba5700 1 -- 172.21.15.97:0/3663828773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 msgr2=0x7f79f40d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.619+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/3663828773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f40d2100 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f79f415f910 tx=0x7f79ec00b330 comp rx=0 tx=0).stop 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.620+0000 7f79f8ba5700 1 -- 172.21.15.97:0/3663828773 shutdown_connections 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.620+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/3663828773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f40d2100 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.620+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/3663828773 >> v2:172.21.15.183:3300/0 conn(0x7f79f40b3370 0x7f79f41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.620+0000 7f79f8ba5700 1 -- 172.21.15.97:0/3663828773 >> 172.21.15.97:0/3663828773 conn(0x7f79f40731a0 msgr2=0x7f79f409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.620+0000 7f79f8ba5700 1 -- 172.21.15.97:0/3663828773 shutdown_connections 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.620+0000 7f79f8ba5700 1 -- 172.21.15.97:0/3663828773 wait complete. 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.621+0000 7f79f8ba5700 1 Processor -- start 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.621+0000 7f79f8ba5700 1 -- start start 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f8ba5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79f40b3370 0x7f79f40cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f8ba5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f4152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f8ba5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79f415bdd0 0x7f79f415d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f8ba5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f415d480 con 0x7f79f40d1c70 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f8ba5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40cbcb0 con 0x7f79f415bdd0 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f8ba5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f415c800 con 0x7f79f40b3370 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f4152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f4152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51252/0 (socket says 172.21.15.97:51252) 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 -- 172.21.15.97:0/956179301 learned_addr learned my addr 172.21.15.97:0/956179301 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f259c700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79f40b3370 0x7f79f40cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 -- 172.21.15.97:0/956179301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79f40b3370 msgr2=0x7f79f40cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79f40b3370 0x7f79f40cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 -- 172.21.15.97:0/956179301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79f415bdd0 msgr2=0x7f79f415d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79f415bdd0 0x7f79f415d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 -- 172.21.15.97:0/956179301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec009a70 con 0x7f79f40d1c70 2023-11-24T06:12:14.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f259c700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79f40b3370 0x7f79f40cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:14.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.622+0000 7f79f1d9b700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f4152640 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f79e400e810 tx=0x7f79e400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.623+0000 7f79e2ffd700 1 -- 172.21.15.97:0/956179301 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4013610 con 0x7f79f40d1c70 2023-11-24T06:12:14.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.623+0000 7f79e2ffd700 1 -- 172.21.15.97:0/956179301 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4013770 con 0x7f79f40d1c70 2023-11-24T06:12:14.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.623+0000 7f79e2ffd700 1 -- 172.21.15.97:0/956179301 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4013b20 con 0x7f79f40d1c70 2023-11-24T06:12:14.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.623+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f415cae0 con 0x7f79f40d1c70 2023-11-24T06:12:14.629 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.623+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f415d0a0 con 0x7f79f40d1c70 2023-11-24T06:12:14.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.624+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f415d5c0 con 0x7f79f40d1c70 2023-11-24T06:12:14.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.624+0000 7f79e2ffd700 1 -- 172.21.15.97:0/956179301 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f79e4013c80 con 0x7f79f40d1c70 2023-11-24T06:12:14.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.624+0000 7f79e2ffd700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79d406b4a0 0x7f79d406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:14.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.625+0000 7f79f259c700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79d406b4a0 0x7f79d406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:14.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.625+0000 7f79e2ffd700 1 -- 172.21.15.97:0/956179301 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f79e4089dc0 con 0x7f79f40d1c70 2023-11-24T06:12:14.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.625+0000 7f79f259c700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79d406b4a0 0x7f79d406d960 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f79e8000f60 tx=0x7f79e8007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:14.634 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.630+0000 7f79e2ffd700 1 -- 172.21.15.97:0/956179301 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79f415d5c0 con 0x7f79f40d1c70 2023-11-24T06:12:14.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.863+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f79f415d5c0 con 0x7f79f40d1c70 2023-11-24T06:12:14.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.864+0000 7f79e2ffd700 1 -- 172.21.15.97:0/956179301 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f79f415d5c0 con 0x7f79f40d1c70 2023-11-24T06:12:14.867 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:12:14.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.865+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79d406b4a0 msgr2=0x7f79d406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.865+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79d406b4a0 0x7f79d406d960 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f79e8000f60 tx=0x7f79e8007d30 comp rx=0 tx=0).stop 2023-11-24T06:12:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 msgr2=0x7f79f4152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f4152640 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f79e400e810 tx=0x7f79e400ebd0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 shutdown_connections 2023-11-24T06:12:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79f415bdd0 0x7f79f415d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f79d406b4a0 0x7f79d406d960 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f79f40b3370 0x7f79f40cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 --2- 172.21.15.97:0/956179301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40d1c70 0x7f79f4152640 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 >> 172.21.15.97:0/956179301 conn(0x7f79f40731a0 msgr2=0x7f79f406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 shutdown_connections 2023-11-24T06:12:14.871 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:14.866+0000 7f79f8ba5700 1 -- 172.21.15.97:0/956179301 wait complete. 2023-11-24T06:12:14.881 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:12:14.881 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:12:14.881 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-24T06:12:14.882 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:12:14.882 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:14.882 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:12:14.882 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:12:14.882 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:12:14.882 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:12:14.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:14.883 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-24T06:12:14.899 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:12:14.900 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:12:14.900 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:12:14.900 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:14.900 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-24T06:12:14.900 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:14.900 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:14.900 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:14.901 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:14.901 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:14.901 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:14.901 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:14.901 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:12:14.901 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:14.901 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:12:14.902 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42002sec preferred_lft 42002sec 2023-11-24T06:12:14.902 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c9d4:801c:12b3:b866/64 scope link noprefixroute 2023-11-24T06:12:14.902 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:14.902 INFO:teuthology.orchestra.run.smithi097.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:12:14.902 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:14.902 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:12:14.903 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:14.903 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::7c25:b1ff:fe16:36f1/64 scope link 2023-11-24T06:12:14.903 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:14.903 INFO:teuthology.orchestra.run.smithi097.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:12:14.903 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether a2:28:0d:2d:82:2b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:12:14.903 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::a028:dff:fe2d:822b/64 scope link 2023-11-24T06:12:14.903 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:14.904 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:12:14.904 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 1a:75:0d:c0:4a:f7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:12:14.904 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::1875:dff:fec0:4af7/64 scope link 2023-11-24T06:12:14.904 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:14.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:14.904 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:12:14.965 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-24T06:12:14.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:14.967 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-24T06:12:15.026 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-24T06:12:15.026 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:12:15.026 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:12:15.026 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:12:15.026 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:12:15.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:15.026 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-24T06:12:15.027 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-24T06:12:15.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:12:15.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:12:15.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:12:15.027 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-24T06:12:15.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:15.131 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:12:15.147 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:12:15.147 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:12:15.147 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:12:15.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:15.147 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-24T06:12:15.214 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:12:15.214 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:15.271 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:12:15.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:12:15.271 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:15.328 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T06:12:15.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:15.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:12:15.414 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:12:15.437 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:12:15.437 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:12:15.437 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:12:15.437 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:12:15.437 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:12:15.437 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:12:15.437 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:12:15.437 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:12:15.513 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:12:15.514 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:12:15.515 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:12:15.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:15.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:15.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:15.542 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:12:15.632 INFO:teuthology.orchestra.run.smithi097.stdout:client6307 2023-11-24T06:12:15.632 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:12:15.632 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:15.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:12:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.806+0000 7f158e2c9700 1 Processor -- start 2023-11-24T06:12:15.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.806+0000 7f158e2c9700 1 -- start start 2023-11-24T06:12:15.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.807+0000 7f158e2c9700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15880adf00 0x7f158806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.807+0000 7f158e2c9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880af950 con 0x7f15881446a0 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.807+0000 7f158e2c9700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880afc30 con 0x7f158806c540 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.807+0000 7f158e2c9700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f158806c9f0 con 0x7f15880adf00 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.807+0000 7f15877fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15880adf00 0x7f158806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.807+0000 7f1587fff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f15881446a0 0x7f15880b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43716/0 (socket says 172.21.15.97:43716) 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f15877fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15880adf00 0x7f158806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51264/0 (socket says 172.21.15.97:51264) 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f1587fff700 1 -- 172.21.15.97:0/2930137361 learned_addr learned my addr 172.21.15.97:0/2930137361 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f15877fe700 1 -- 172.21.15.97:0/2930137361 >> v1:172.21.15.183:6789/0 conn(0x7f158806c540 legacy=0x7f15880b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f15877fe700 1 -- 172.21.15.97:0/2930137361 >> v1:172.21.15.97:6789/0 conn(0x7f15881446a0 legacy=0x7f15880b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f1586ffd700 1 -- 172.21.15.97:0/2930137361 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510501137 0 0) 0x7f15880af950 con 0x7f15881446a0 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f15877fe700 1 -- 172.21.15.97:0/2930137361 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880aff10 con 0x7f15880adf00 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f15877fe700 1 --2- 172.21.15.97:0/2930137361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158806c000 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1578009840 tx=0x7f1578009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fdf55b85f71622 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f1586ffd700 1 -- 172.21.15.97:0/2930137361 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578011830 con 0x7f15880adf00 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f1586ffd700 1 -- 172.21.15.97:0/2930137361 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1578011990 con 0x7f15880adf00 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.808+0000 7f1586ffd700 1 -- 172.21.15.97:0/2930137361 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578011d20 con 0x7f15880adf00 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.809+0000 7f158e2c9700 1 -- 172.21.15.97:0/2930137361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 msgr2=0x7f158806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.809+0000 7f158e2c9700 1 --2- 172.21.15.97:0/2930137361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158806c000 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f1578009840 tx=0x7f1578009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.809+0000 7f158e2c9700 1 -- 172.21.15.97:0/2930137361 shutdown_connections 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.809+0000 7f158e2c9700 1 --2- 172.21.15.97:0/2930137361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158806c000 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.809+0000 7f158e2c9700 1 -- 172.21.15.97:0/2930137361 >> 172.21.15.97:0/2930137361 conn(0x7f1588071b90 msgr2=0x7f1588071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.809+0000 7f158e2c9700 1 -- 172.21.15.97:0/2930137361 shutdown_connections 2023-11-24T06:12:15.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.809+0000 7f158e2c9700 1 -- 172.21.15.97:0/2930137361 wait complete. 2023-11-24T06:12:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.810+0000 7f158e2c9700 1 Processor -- start 2023-11-24T06:12:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.810+0000 7f158e2c9700 1 -- start start 2023-11-24T06:12:15.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f158e2c9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f158806c540 0x7f15881f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f158e2c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f158e2c9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15881446a0 0x7f158815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f158e2c9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881f15c0 con 0x7f15880adf00 2023-11-24T06:12:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f158e2c9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158815be40 con 0x7f158806c540 2023-11-24T06:12:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f158e2c9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158815f1e0 con 0x7f15881446a0 2023-11-24T06:12:15.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158815b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51272/0 (socket says 172.21.15.97:51272) 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 -- 172.21.15.97:0/1003504994 learned_addr learned my addr 172.21.15.97:0/1003504994 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f158c866700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15881446a0 0x7f158815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f1587fff700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f158806c540 0x7f15881f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 -- 172.21.15.97:0/1003504994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15881446a0 msgr2=0x7f158815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15881446a0 0x7f158815eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 -- 172.21.15.97:0/1003504994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f158806c540 msgr2=0x7f15881f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f158806c540 0x7f15881f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:15.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 -- 172.21.15.97:0/1003504994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780094a0 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.811+0000 7f15877fe700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158815b900 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1578013000 tx=0x7f1578004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.812+0000 7f1584ff9700 1 -- 172.21.15.97:0/1003504994 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157801a7e0 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.812+0000 7f1584ff9700 1 -- 172.21.15.97:0/1003504994 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157801a940 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.812+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15881f5b30 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.812+0000 7f1584ff9700 1 -- 172.21.15.97:0/1003504994 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157801acd0 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.812+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15881f5570 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.813+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15880753f0 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.813+0000 7f1584ff9700 1 -- 172.21.15.97:0/1003504994 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1578022070 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.813+0000 7f1584ff9700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f157406b470 0x7f157406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.813+0000 7f1584ff9700 1 -- 172.21.15.97:0/1003504994 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f157808b110 con 0x7f15880adf00 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.813+0000 7f1587fff700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f157406b470 0x7f157406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:15.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.814+0000 7f1587fff700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f157406b470 0x7f157406d930 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f157c007c40 tx=0x7f157c007b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:15.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:15.818+0000 7f1584ff9700 1 -- 172.21.15.97:0/1003504994 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15780bde90 con 0x7f15880adf00 2023-11-24T06:12:16.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.052+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1588067f60 con 0x7f15880adf00 2023-11-24T06:12:16.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.053+0000 7f1584ff9700 1 -- 172.21.15.97:0/1003504994 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7f15780559c0 con 0x7f15880adf00 2023-11-24T06:12:16.056 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f157406b470 msgr2=0x7f157406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f157406b470 0x7f157406d930 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f157c007c40 tx=0x7f157c007b90 comp rx=0 tx=0).stop 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 msgr2=0x7f158815b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158815b900 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1578013000 tx=0x7f1578004750 comp rx=0 tx=0).stop 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 shutdown_connections 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f157406b470 0x7f157406d930 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15881446a0 0x7f158815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f158806c540 0x7f15881f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 --2- 172.21.15.97:0/1003504994 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15880adf00 0x7f158815b900 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:16.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 >> 172.21.15.97:0/1003504994 conn(0x7f1588071b90 msgr2=0x7f1588071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 shutdown_connections 2023-11-24T06:12:16.060 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:16.055+0000 7f158e2c9700 1 -- 172.21.15.97:0/1003504994 wait complete. 2023-11-24T06:12:16.070 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T06:12:16.070 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:12:16.071 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi183.front.sepia.ceph.com 2023-11-24T06:12:16.071 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:12:16.071 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:16.071 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T06:12:16.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T06:12:16.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:12:16.071 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:12:16.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.071 DEBUG:teuthology.orchestra.run.smithi183:> ip addr 2023-11-24T06:12:16.092 INFO:teuthology.orchestra.run.smithi183.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:12:16.092 INFO:teuthology.orchestra.run.smithi183.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 ::1/128 scope host 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:aa brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:d9:38:ab brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:84 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:16.093 INFO:teuthology.orchestra.run.smithi183.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 0c:c4:7a:88:80:85 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: inet 172.21.15.183/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft 41993sec preferred_lft 41993sec 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::5382:2bb9:1085:5950/64 scope link noprefixroute 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::b8cd:8aff:fe67:b24d/64 scope link 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:16.094 INFO:teuthology.orchestra.run.smithi183.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:12:16.095 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether fe:4f:a7:e6:1c:7f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:12:16.095 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::fc4f:a7ff:fee6:1c7f/64 scope link 2023-11-24T06:12:16.095 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:16.095 INFO:teuthology.orchestra.run.smithi183.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:12:16.095 INFO:teuthology.orchestra.run.smithi183.stdout: link/ether 42:34:29:35:b2:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:12:16.095 INFO:teuthology.orchestra.run.smithi183.stdout: inet6 fe80::4034:29ff:fe35:b2a5/64 scope link 2023-11-24T06:12:16.095 INFO:teuthology.orchestra.run.smithi183.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:12:16.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.095 DEBUG:teuthology.orchestra.run.smithi183:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:12:16.163 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-11-24T06:12:16.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.164 DEBUG:teuthology.orchestra.run.smithi183:> route 2023-11-24T06:12:16.225 INFO:teuthology.orchestra.run.smithi183.stdout:Kernel IP routing table 2023-11-24T06:12:16.226 INFO:teuthology.orchestra.run.smithi183.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:12:16.226 INFO:teuthology.orchestra.run.smithi183.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T06:12:16.226 INFO:teuthology.orchestra.run.smithi183.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T06:12:16.226 INFO:teuthology.orchestra.run.smithi183.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:12:16.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.226 DEBUG:teuthology.orchestra.run.smithi183:> (cd / && exec bash -c ' 2023-11-24T06:12:16.226 DEBUG:teuthology.orchestra.run.smithi183:> set -e 2023-11-24T06:12:16.227 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T06:12:16.227 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T06:12:16.227 DEBUG:teuthology.orchestra.run.smithi183:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T06:12:16.227 DEBUG:teuthology.orchestra.run.smithi183:> ') 2023-11-24T06:12:16.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.336 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:12:16.353 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:12:16.353 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:12:16.353 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:12:16.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.353 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link set brx.0 up 2023-11-24T06:12:16.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:12:16.420 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:16.476 INFO:teuthology.orchestra.run.smithi183.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:12:16.476 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:12:16.476 DEBUG:teuthology.orchestra.run.smithi183:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:16.531 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T06:12:16.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.532 DEBUG:teuthology.orchestra.run.smithi183:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:12:16.618 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:12:16.618 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:12:16.647 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:12:16.647 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:12:16.647 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:12:16.647 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:12:16.647 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:12:16.647 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:12:16.647 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:12:16.647 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:12:16.724 INFO:teuthology.orchestra.run.smithi183.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T06:12:16.724 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T06:12:16.724 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.183:3301" 2023-11-24T06:12:16.725 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with new device syntax: 1@d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9.cephfs=/ 2023-11-24T06:12:16.725 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.183:3301" will pass to kernel 2023-11-24T06:12:16.725 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T06:12:16.725 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T06:12:16.725 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T06:12:16.725 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: resolved to: "172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790" 2023-11-24T06:12:16.725 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.97:6789,172.21.15.183:6789,172.21.15.183:6790:/ 2023-11-24T06:12:16.725 INFO:teuthology.orchestra.run.smithi183.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9" will pass to kernel 2023-11-24T06:12:16.726 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T06:12:16.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.727 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:16.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:16.753 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T06:12:16.826 INFO:teuthology.orchestra.run.smithi183.stdout:client7080 2023-11-24T06:12:16.826 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:12:16.826 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:16.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:12:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.012+0000 7f085dd6c700 1 Processor -- start 2023-11-24T06:12:17.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.012+0000 7f085dd6c700 1 -- start start 2023-11-24T06:12:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f085dd6c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f08580ad8f0 0x7f0858165390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f085dd6c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08580aeff0 con 0x7f085815a510 2023-11-24T06:12:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f085dd6c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08580adcd0 con 0x7f08580ce5a0 2023-11-24T06:12:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f085dd6c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08580aebd0 con 0x7f08580ad8f0 2023-11-24T06:12:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f08577fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f085815a510 0x7f08580cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43730/0 (socket says 172.21.15.97:43730) 2023-11-24T06:12:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f08577fe700 1 -- 172.21.15.97:0/2962201634 learned_addr learned my addr 172.21.15.97:0/2962201634 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f0857fff700 1 --2- 172.21.15.97:0/2962201634 >> v2:172.21.15.183:3300/0 conn(0x7f08580ad8f0 0x7f0858165390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529129734 0 0) 0x7f0840001110 con 0x7f085815a510 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.013+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08440035b0 con 0x7f085815a510 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2277942054 0 0) 0x7f08440035b0 con 0x7f085815a510 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 >> v2:172.21.15.183:3300/0 conn(0x7f08580ad8f0 msgr2=0x7f0858165390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 --2- 172.21.15.97:0/2962201634 >> v2:172.21.15.183:3300/0 conn(0x7f08580ad8f0 0x7f0858165390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 >> v1:172.21.15.183:6789/0 conn(0x7f08580ce5a0 legacy=0x7f08580ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580adf50 con 0x7f085815a510 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f0840001800 con 0x7f085815a510 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 >> v1:172.21.15.97:6789/0 conn(0x7f085815a510 legacy=0x7f08580cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0844003a00 0x7f0844003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844004850 0x7f0844004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0844005210 0x7f084400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580aebd0 con 0x7f0844005210 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084400c0b0 con 0x7f0844003a00 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084400d380 con 0x7f0844004850 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08577fe700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0844003a00 0x7f0844003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0857fff700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0844005210 0x7f084400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0856ffd700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844004850 0x7f0844004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0857fff700 1 -- 172.21.15.97:0/2962201634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844004850 msgr2=0x7f0844004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0857fff700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844004850 0x7f0844004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0857fff700 1 -- 172.21.15.97:0/2962201634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0844003a00 msgr2=0x7f0844003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:17.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0857fff700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0844003a00 0x7f0844003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0857fff700 1 -- 172.21.15.97:0/2962201634 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08440035b0 con 0x7f0844005210 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0857fff700 1 -- 172.21.15.97:0/2962201634 reap_dead start 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f0857fff700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0844005210 0x7f084400ba10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0850009fc0 tx=0x7f0844005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1324811d384971 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850003a20 con 0x7f0844005210 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085000f040 con 0x7f0844005210 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.014+0000 7f08567fc700 1 -- 172.21.15.97:0/2962201634 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850003de0 con 0x7f0844005210 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.015+0000 7f085dd6c700 1 -- 172.21.15.97:0/2962201634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0844005210 msgr2=0x7f084400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.015+0000 7f085dd6c700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0844005210 0x7f084400ba10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0850009fc0 tx=0x7f0844005880 comp rx=0 tx=0).stop 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.015+0000 7f085dd6c700 1 -- 172.21.15.97:0/2962201634 shutdown_connections 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.015+0000 7f085dd6c700 1 --2- 172.21.15.97:0/2962201634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0844005210 0x7f084400ba10 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.015+0000 7f085dd6c700 1 -- 172.21.15.97:0/2962201634 >> 172.21.15.97:0/2962201634 conn(0x7f0858072710 msgr2=0x7f0858072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.015+0000 7f085dd6c700 1 -- 172.21.15.97:0/2962201634 shutdown_connections 2023-11-24T06:12:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.015+0000 7f085dd6c700 1 -- 172.21.15.97:0/2962201634 wait complete. 2023-11-24T06:12:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.016+0000 7f085dd6c700 1 Processor -- start 2023-11-24T06:12:17.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.016+0000 7f085dd6c700 1 -- start start 2023-11-24T06:12:17.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f085dd6c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844005210 0x7f08580996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f085dd6c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08580b5aa0 0x7f08580b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f085dd6c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08580b4560 0x7f08581e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f085dd6c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580b5610 con 0x7f08580b4560 2023-11-24T06:12:17.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f085dd6c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580b4a00 con 0x7f08580b5aa0 2023-11-24T06:12:17.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f085dd6c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08580b4ce0 con 0x7f0844005210 2023-11-24T06:12:17.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f0857fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08580b4560 0x7f08581e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f08577fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844005210 0x7f08580996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f0857fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08580b4560 0x7f08581e2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51294/0 (socket says 172.21.15.97:51294) 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f08577fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844005210 0x7f08580996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:38326/0 (socket says 172.21.15.97:38326) 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f0857fff700 1 -- 172.21.15.97:0/1093435691 learned_addr learned my addr 172.21.15.97:0/1093435691 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f0856ffd700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08580b5aa0 0x7f08580b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f0857fff700 1 -- 172.21.15.97:0/1093435691 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844005210 msgr2=0x7f08580996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f0857fff700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844005210 0x7f08580996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f0857fff700 1 -- 172.21.15.97:0/1093435691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08580b5aa0 msgr2=0x7f08580b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.017+0000 7f0857fff700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08580b5aa0 0x7f08580b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.018+0000 7f0857fff700 1 -- 172.21.15.97:0/1093435691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580b4fc0 con 0x7f08580b4560 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.018+0000 7f0857fff700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08580b4560 0x7f08581e2e90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f08580af920 tx=0x7f0850009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.018+0000 7f084ffff700 1 -- 172.21.15.97:0/1093435691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850003e10 con 0x7f08580b4560 2023-11-24T06:12:17.027 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.018+0000 7f084ffff700 1 -- 172.21.15.97:0/1093435691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0850004230 con 0x7f08580b4560 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.018+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08580b52a0 con 0x7f08580b4560 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.018+0000 7f084ffff700 1 -- 172.21.15.97:0/1093435691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08500045e0 con 0x7f08580b4560 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.018+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0858151320 con 0x7f08580b4560 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.019+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0858151540 con 0x7f08580b4560 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.019+0000 7f084ffff700 1 -- 172.21.15.97:0/1093435691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0850004740 con 0x7f08580b4560 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.020+0000 7f084ffff700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f083806b470 0x7f083806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.020+0000 7f084ffff700 1 -- 172.21.15.97:0/1093435691 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f085008b460 con 0x7f08580b4560 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.020+0000 7f08577fe700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f083806b470 0x7f083806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.020+0000 7f08577fe700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f083806b470 0x7f083806d930 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0840001050 tx=0x7f0840000f80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.024+0000 7f084ffff700 1 -- 172.21.15.97:0/1093435691 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0858151540 con 0x7f08580b4560 2023-11-24T06:12:17.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.257+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0858151540 con 0x7f08580b4560 2023-11-24T06:12:17.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.652+0000 7f084ffff700 1 -- 172.21.15.97:0/1093435691 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 136+0+0 (secure 0 0 0) 0x7f0858151540 con 0x7f08580b4560 2023-11-24T06:12:17.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f083806b470 msgr2=0x7f083806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f083806b470 0x7f083806d930 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0840001050 tx=0x7f0840000f80 comp rx=0 tx=0).stop 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08580b4560 msgr2=0x7f08581e2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08580b4560 0x7f08581e2e90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f08580af920 tx=0x7f0850009750 comp rx=0 tx=0).stop 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 shutdown_connections 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08580b4560 0x7f08581e2e90 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f083806b470 0x7f083806d930 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0844005210 0x7f08580996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.654+0000 7f085dd6c700 1 --2- 172.21.15.97:0/1093435691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08580b5aa0 0x7f08580b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.655+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 >> 172.21.15.97:0/1093435691 conn(0x7f0858072710 msgr2=0x7f08580ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.655+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 shutdown_connections 2023-11-24T06:12:17.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.655+0000 7f085dd6c700 1 -- 172.21.15.97:0/1093435691 wait complete. 2023-11-24T06:12:17.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:12:17.793 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc151b5e700 1 Processor -- start 2023-11-24T06:12:17.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc151b5e700 1 -- start start 2023-11-24T06:12:17.795 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc151b5e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc14c13e000 0x7fc14c15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc151b5e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 0x7fc14c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc151b5e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc14c0cfd40 0x7fc14c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc151b5e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14c15f430 con 0x7fc14c13e000 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc151b5e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc14c0b4290 con 0x7fc14c0cfd40 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc151b5e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c13e570 con 0x7fc14c15efb0 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc14c13e000 0x7fc14c15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc14c13e000 0x7fc14c15ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51298/0 (socket says 172.21.15.97:51298) 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 -- 172.21.15.97:0/308503559 learned_addr learned my addr 172.21.15.97:0/308503559 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 -- 172.21.15.97:0/308503559 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 msgr2=0x7fc14c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 --2- 172.21.15.97:0/308503559 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 0x7fc14c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.796 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 -- 172.21.15.97:0/308503559 >> v2:172.21.15.183:3300/0 conn(0x7fc14c0cfd40 msgr2=0x7fc14c15b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 --2- 172.21.15.97:0/308503559 >> v2:172.21.15.183:3300/0 conn(0x7fc14c0cfd40 0x7fc14c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 -- 172.21.15.97:0/308503559 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c0d05d0 con 0x7fc14c13e000 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14b7fe700 1 --2- 172.21.15.97:0/308503559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c13e000 0x7fc14c15ea70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0b3870 tx=0x7fc13c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95595b18e57484f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14a7fc700 1 -- 172.21.15.97:0/308503559 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00b870 con 0x7fc14c13e000 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14a7fc700 1 -- 172.21.15.97:0/308503559 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c00b9d0 con 0x7fc14c13e000 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.791+0000 7fc14a7fc700 1 -- 172.21.15.97:0/308503559 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00bd60 con 0x7fc14c13e000 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 -- 172.21.15.97:0/308503559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c13e000 msgr2=0x7fc14c15ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 --2- 172.21.15.97:0/308503559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c13e000 0x7fc14c15ea70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc14c0b3870 tx=0x7fc13c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 -- 172.21.15.97:0/308503559 shutdown_connections 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 --2- 172.21.15.97:0/308503559 >> v2:172.21.15.183:3300/0 conn(0x7fc14c0cfd40 0x7fc14c15b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 --2- 172.21.15.97:0/308503559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c13e000 0x7fc14c15ea70 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 --2- 172.21.15.97:0/308503559 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 0x7fc14c0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 -- 172.21.15.97:0/308503559 >> 172.21.15.97:0/308503559 conn(0x7fc14c072990 msgr2=0x7fc14c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:17.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 -- 172.21.15.97:0/308503559 shutdown_connections 2023-11-24T06:12:17.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.792+0000 7fc151b5e700 1 -- 172.21.15.97:0/308503559 wait complete. 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.793+0000 7fc151b5e700 1 Processor -- start 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.793+0000 7fc151b5e700 1 -- start start 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc151b5e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c0cfd40 0x7fc14c13d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc151b5e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc14c13e000 0x7fc14c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc151b5e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 0x7fc14c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc151b5e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c1f3d90 con 0x7fc14c0cfd40 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc151b5e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c06e6e0 con 0x7fc14c13e000 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc151b5e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14c06e9c0 con 0x7fc14c15efb0 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c0cfd40 0x7fc14c13d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c0cfd40 0x7fc14c13d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51304/0 (socket says 172.21.15.97:51304) 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14b7fe700 1 -- 172.21.15.97:0/3832938128 learned_addr learned my addr 172.21.15.97:0/3832938128 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14bfff700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 0x7fc14c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14affd700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc14c13e000 0x7fc14c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14b7fe700 1 -- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 msgr2=0x7fc14c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14b7fe700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 0x7fc14c06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14b7fe700 1 -- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc14c13e000 msgr2=0x7fc14c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.794+0000 7fc14b7fe700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc14c13e000 0x7fc14c06db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.795+0000 7fc14b7fe700 1 -- 172.21.15.97:0/3832938128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c009550 con 0x7fc14c0cfd40 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.795+0000 7fc14bfff700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 0x7fc14c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.795+0000 7fc14b7fe700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c0cfd40 0x7fc14c13d420 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc13c014040 tx=0x7fc13c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.802 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.795+0000 7fc133fff700 1 -- 172.21.15.97:0/3832938128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c004520 con 0x7fc14c0cfd40 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.795+0000 7fc133fff700 1 -- 172.21.15.97:0/3832938128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c01e070 con 0x7fc14c0cfd40 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.795+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14c06eca0 con 0x7fc14c0cfd40 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.795+0000 7fc133fff700 1 -- 172.21.15.97:0/3832938128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0038c0 con 0x7fc14c0cfd40 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.795+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14c157b80 con 0x7fc14c0cfd40 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.796+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14c06eeb0 con 0x7fc14c0cfd40 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.796+0000 7fc133fff700 1 -- 172.21.15.97:0/3832938128 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fc13c003a20 con 0x7fc14c0cfd40 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.797+0000 7fc133fff700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc13806b470 0x7fc13806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.797+0000 7fc133fff700 1 -- 172.21.15.97:0/3832938128 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc13c020070 con 0x7fc14c0cfd40 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.797+0000 7fc14affd700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc13806b470 0x7fc13806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:17.803 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.797+0000 7fc14affd700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc13806b470 0x7fc13806d930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc140006fd0 tx=0x7fc140008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:17.805 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:17.802+0000 7fc133fff700 1 -- 172.21.15.97:0/3832938128 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc13c055f90 con 0x7fc14c0cfd40 2023-11-24T06:12:18.022 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.018+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc14c06c4c0 con 0x7fc13806b470 2023-11-24T06:12:18.023 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.019+0000 7fc133fff700 1 -- 172.21.15.97:0/3832938128 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc14c06c4c0 con 0x7fc13806b470 2023-11-24T06:12:18.023 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2023-11-24T06:12:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: { 2023-11-24T06:12:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephfs" 2023-11-24T06:12:18.023 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-11-24T06:12:18.023 INFO:teuthology.orchestra.run.smithi097.stdout:] 2023-11-24T06:12:18.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc13806b470 msgr2=0x7fc13806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc13806b470 0x7fc13806d930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc140006fd0 tx=0x7fc140008040 comp rx=0 tx=0).stop 2023-11-24T06:12:18.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c0cfd40 msgr2=0x7fc14c13d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c0cfd40 0x7fc14c13d420 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc13c014040 tx=0x7fc13c004750 comp rx=0 tx=0).stop 2023-11-24T06:12:18.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 shutdown_connections 2023-11-24T06:12:18.025 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc13806b470 0x7fc13806d930 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc14c15efb0 0x7fc14c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc14c0cfd40 0x7fc14c13d420 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 --2- 172.21.15.97:0/3832938128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc14c13e000 0x7fc14c06db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 >> 172.21.15.97:0/3832938128 conn(0x7fc14c072990 msgr2=0x7fc14c0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 shutdown_connections 2023-11-24T06:12:18.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.021+0000 7fc151b5e700 1 -- 172.21.15.97:0/3832938128 wait complete. 2023-11-24T06:12:18.037 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:12:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.156+0000 7f22f0cfa700 1 Processor -- start 2023-11-24T06:12:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.156+0000 7f22f0cfa700 1 -- start start 2023-11-24T06:12:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22f0cfa700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06e370 0x7f22ec15b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22f0cfa700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f22ec06b0f0 0x7f22ec06b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22f0cfa700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec15e9f0 con 0x7f22ec0b49a0 2023-11-24T06:12:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22f0cfa700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec15ec90 con 0x7f22ec06b0f0 2023-11-24T06:12:18.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22f0cfa700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec15ef70 con 0x7f22ec06e370 2023-11-24T06:12:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ead9d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f22ec06b0f0 0x7f22ec06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22e9d9b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06e370 0x7f22ec15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ead9d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f22ec06b0f0 0x7f22ec06b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:45476/0 (socket says 172.21.15.97:45476) 2023-11-24T06:12:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ead9d700 1 -- 172.21.15.97:0/617990272 learned_addr learned my addr 172.21.15.97:0/617990272 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ea59c700 1 --1- 172.21.15.97:0/617990272 >> v1:172.21.15.183:6789/0 conn(0x7f22ec0b49a0 0x7f22ec0b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.183:6789/0 says I am v1:172.21.15.97:53086/0 (socket says 172.21.15.97:53086) 2023-11-24T06:12:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ead9d700 1 -- 172.21.15.97:0/617990272 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06e370 msgr2=0x7f22ec15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ead9d700 1 --2- 172.21.15.97:0/617990272 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06e370 0x7f22ec15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ead9d700 1 -- 172.21.15.97:0/617990272 >> v1:172.21.15.183:6789/0 conn(0x7f22ec0b49a0 legacy=0x7f22ec0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ead9d700 1 -- 172.21.15.97:0/617990272 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec15f250 con 0x7f22ec06b0f0 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22e959a700 1 -- 172.21.15.97:0/617990272 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649318546 0 0) 0x7f22ec15e9f0 con 0x7f22ec0b49a0 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22ead9d700 1 --2- 172.21.15.97:0/617990272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06b0f0 0x7f22ec06b580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009e50 tx=0x7f22e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8c74a4dc4a3dd37 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22e959a700 1 -- 172.21.15.97:0/617990272 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400e070 con 0x7f22ec06b0f0 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22e959a700 1 -- 172.21.15.97:0/617990272 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e400b910 con 0x7f22ec06b0f0 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.158+0000 7f22e959a700 1 -- 172.21.15.97:0/617990272 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400bcc0 con 0x7f22ec06b0f0 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.159+0000 7f22f0cfa700 1 -- 172.21.15.97:0/617990272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06b0f0 msgr2=0x7f22ec06b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.159+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/617990272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06b0f0 0x7f22ec06b580 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009e50 tx=0x7f22e400b330 comp rx=0 tx=0).stop 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.159+0000 7f22f0cfa700 1 -- 172.21.15.97:0/617990272 shutdown_connections 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.159+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/617990272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06b0f0 0x7f22ec06b580 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.159+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/617990272 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06e370 0x7f22ec15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.159+0000 7f22f0cfa700 1 -- 172.21.15.97:0/617990272 >> 172.21.15.97:0/617990272 conn(0x7f22ec0b28f0 msgr2=0x7f22ec0b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.159+0000 7f22f0cfa700 1 -- 172.21.15.97:0/617990272 shutdown_connections 2023-11-24T06:12:18.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.159+0000 7f22f0cfa700 1 -- 172.21.15.97:0/617990272 wait complete. 2023-11-24T06:12:18.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.160+0000 7f22f0cfa700 1 Processor -- start 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.160+0000 7f22f0cfa700 1 -- start start 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22f0cfa700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06b0f0 0x7f22ec14c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22f0cfa700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06e370 0x7f22ec0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22f0cfa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec0b49a0 0x7f22ec15a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22f0cfa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec074e70 con 0x7f22ec0b49a0 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22f0cfa700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0742f0 con 0x7f22ec06e370 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22f0cfa700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0745d0 con 0x7f22ec06b0f0 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22ead9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec0b49a0 0x7f22ec15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22ead9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec0b49a0 0x7f22ec15a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51322/0 (socket says 172.21.15.97:51322) 2023-11-24T06:12:18.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22ead9d700 1 -- 172.21.15.97:0/68511178 learned_addr learned my addr 172.21.15.97:0/68511178 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22ea59c700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06b0f0 0x7f22ec14c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.161+0000 7f22e9d9b700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06e370 0x7f22ec0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22ead9d700 1 -- 172.21.15.97:0/68511178 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06b0f0 msgr2=0x7f22ec14c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22ead9d700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06b0f0 0x7f22ec14c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22ead9d700 1 -- 172.21.15.97:0/68511178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06e370 msgr2=0x7f22ec0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22ead9d700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06e370 0x7f22ec0afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22ead9d700 1 -- 172.21.15.97:0/68511178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4009b30 con 0x7f22ec0b49a0 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22ead9d700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec0b49a0 0x7f22ec15a540 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f22ec072f70 tx=0x7f22e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22daffd700 1 -- 172.21.15.97:0/68511178 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4003ea0 con 0x7f22ec0b49a0 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22daffd700 1 -- 172.21.15.97:0/68511178 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e40048e0 con 0x7f22ec0b49a0 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec0748b0 con 0x7f22ec0b49a0 2023-11-24T06:12:18.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22daffd700 1 -- 172.21.15.97:0/68511178 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e401b670 con 0x7f22ec0b49a0 2023-11-24T06:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.162+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec0af5d0 con 0x7f22ec0b49a0 2023-11-24T06:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.163+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec0af790 con 0x7f22ec0b49a0 2023-11-24T06:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.163+0000 7f22daffd700 1 -- 172.21.15.97:0/68511178 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f22e401b7d0 con 0x7f22ec0b49a0 2023-11-24T06:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.164+0000 7f22daffd700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f22cc06b470 0x7f22cc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.164+0000 7f22daffd700 1 -- 172.21.15.97:0/68511178 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f22e408b860 con 0x7f22ec0b49a0 2023-11-24T06:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.164+0000 7f22ea59c700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f22cc06b470 0x7f22cc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.170 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.164+0000 7f22ea59c700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f22cc06b470 0x7f22cc06d930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f22e0000f60 tx=0x7f22e0007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.171 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.169+0000 7f22daffd700 1 -- 172.21.15.97:0/68511178 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f22e4056000 con 0x7f22ec0b49a0 2023-11-24T06:12:18.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.359+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f22ec068fb0 con 0x7f22ec0b49a0 2023-11-24T06:12:18.363 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.359+0000 7f22daffd700 1 -- 172.21.15.97:0/68511178 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f22e4059620 con 0x7f22ec0b49a0 2023-11-24T06:12:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f22cc06b470 msgr2=0x7f22cc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f22cc06b470 0x7f22cc06d930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f22e0000f60 tx=0x7f22e0007e70 comp rx=0 tx=0).stop 2023-11-24T06:12:18.365 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec0b49a0 msgr2=0x7f22ec15a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec0b49a0 0x7f22ec15a540 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f22ec072f70 tx=0x7f22e4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 shutdown_connections 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22ec0b49a0 0x7f22ec15a540 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f22cc06b470 0x7f22cc06d930 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f22ec06b0f0 0x7f22ec14c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 --2- 172.21.15.97:0/68511178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec06e370 0x7f22ec0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 >> 172.21.15.97:0/68511178 conn(0x7f22ec0b28f0 msgr2=0x7f22ec0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 shutdown_connections 2023-11-24T06:12:18.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.361+0000 7f22f0cfa700 1 -- 172.21.15.97:0/68511178 wait complete. 2023-11-24T06:12:18.377 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp6193 2023-11-24T06:12:18.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.496+0000 7f1a1eca6700 1 Processor -- start 2023-11-24T06:12:18.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.496+0000 7f1a1eca6700 1 -- start start 2023-11-24T06:12:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a1eca6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a180cc540 0x7f1a1806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a1eca6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a180b2f20 con 0x7f1a1806eb20 2023-11-24T06:12:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a1eca6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1815aa00 con 0x7f1a1815a620 2023-11-24T06:12:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a1eca6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1806efb0 con 0x7f1a180cc540 2023-11-24T06:12:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a1d243700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1a1806eb20 0x7f1a180b3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43782/0 (socket says 172.21.15.97:43782) 2023-11-24T06:12:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a1d243700 1 -- 172.21.15.97:0/1708211044 learned_addr learned my addr 172.21.15.97:0/1708211044 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:18.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a17fff700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a180cc540 0x7f1a1806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870962000 0 0) 0x7f1a180b2f20 con 0x7f1a1806eb20 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a080035b0 con 0x7f1a1806eb20 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3203256359 0 0) 0x7f1a080035b0 con 0x7f1a1806eb20 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a180cc540 msgr2=0x7f1a1806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a180cc540 0x7f1a1806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 >> v1:172.21.15.183:6789/0 conn(0x7f1a1815a620 legacy=0x7f1a180ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180ce960 con 0x7f1a1806eb20 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f1a100019f0 con 0x7f1a1806eb20 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 >> v1:172.21.15.97:6789/0 conn(0x7f1a1806eb20 legacy=0x7f1a180b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.497+0000 7f1a177fe700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a08003a30 0x7f1a08003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a177fe700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a08004850 0x7f1a08004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a177fe700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a180cc540 0x7f1a08005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1806efb0 con 0x7f1a08003a30 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0800cbd0 con 0x7f1a08004850 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08005a60 con 0x7f1a180cc540 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a1ca42700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a08003a30 0x7f1a08003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a17fff700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a08004850 0x7f1a08004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a1d243700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a180cc540 0x7f1a08005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a1ca42700 1 -- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a180cc540 msgr2=0x7f1a08005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a1ca42700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a180cc540 0x7f1a08005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a1ca42700 1 -- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a08004850 msgr2=0x7f1a08004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a1ca42700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a08004850 0x7f1a08004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a1ca42700 1 -- 172.21.15.97:0/1708211044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080035b0 con 0x7f1a08003a30 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a1ca42700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a08003a30 0x7f1a08003e50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007fd0 tx=0x7f1a08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa59ff394bd99b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c0052d0 con 0x7f1a08003a30 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c005470 con 0x7f1a08003a30 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.498+0000 7f1a177fe700 1 -- 172.21.15.97:0/1708211044 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c0038c0 con 0x7f1a08003a30 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.499+0000 7f1a1eca6700 1 -- 172.21.15.97:0/1708211044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a08003a30 msgr2=0x7f1a08003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.499+0000 7f1a1eca6700 1 --2- 172.21.15.97:0/1708211044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a08003a30 0x7f1a08003e50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007fd0 tx=0x7f1a08005880 comp rx=0 tx=0).stop 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.499+0000 7f1a1d243700 1 -- 172.21.15.97:0/1708211044 reap_dead start 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.499+0000 7f1a1eca6700 1 -- 172.21.15.97:0/1708211044 shutdown_connections 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.499+0000 7f1a1eca6700 1 -- 172.21.15.97:0/1708211044 >> 172.21.15.97:0/1708211044 conn(0x7f1a180b2050 msgr2=0x7f1a180b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.499+0000 7f1a1eca6700 1 -- 172.21.15.97:0/1708211044 shutdown_connections 2023-11-24T06:12:18.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.499+0000 7f1a1eca6700 1 -- 172.21.15.97:0/1708211044 wait complete. 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.500+0000 7f1a1eca6700 1 Processor -- start 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.500+0000 7f1a1eca6700 1 -- start start 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a1eca6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a08003a30 0x7f1a18146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a1eca6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18158bc0 0x7f1a181513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a1eca6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a181518f0 0x7f1a181457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a1eca6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a18145320 con 0x7f1a18158bc0 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a1eca6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a181473c0 con 0x7f1a181518f0 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a1eca6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a18144710 con 0x7f1a08003a30 2023-11-24T06:12:18.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18158bc0 0x7f1a181513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18158bc0 0x7f1a181513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51340/0 (socket says 172.21.15.97:51340) 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 -- 172.21.15.97:0/2028629556 learned_addr learned my addr 172.21.15.97:0/2028629556 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a1ca42700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a08003a30 0x7f1a18146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a1d243700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a181518f0 0x7f1a181457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 -- 172.21.15.97:0/2028629556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a08003a30 msgr2=0x7f1a18146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a08003a30 0x7f1a18146e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 -- 172.21.15.97:0/2028629556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a181518f0 msgr2=0x7f1a181457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a181518f0 0x7f1a181457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 -- 172.21.15.97:0/2028629556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a181449f0 con 0x7f1a18158bc0 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.501+0000 7f1a17fff700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18158bc0 0x7f1a181513b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1a040119d0 tx=0x7f1a04011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.502+0000 7f1a157fa700 1 -- 172.21.15.97:0/2028629556 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04011e70 con 0x7f1a18158bc0 2023-11-24T06:12:18.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.502+0000 7f1a157fa700 1 -- 172.21.15.97:0/2028629556 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0400f8c0 con 0x7f1a18158bc0 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.502+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a18144cd0 con 0x7f1a18158bc0 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.502+0000 7f1a157fa700 1 -- 172.21.15.97:0/2028629556 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400fc50 con 0x7f1a18158bc0 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.502+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a181443e0 con 0x7f1a18158bc0 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.503+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a18068d90 con 0x7f1a18158bc0 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.503+0000 7f1a157fa700 1 -- 172.21.15.97:0/2028629556 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1a04014440 con 0x7f1a18158bc0 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.503+0000 7f1a157fa700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19fc06b470 0x7f19fc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.504+0000 7f1a1ca42700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19fc06b470 0x7f19fc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.504+0000 7f1a157fa700 1 -- 172.21.15.97:0/2028629556 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1a0408a580 con 0x7f1a18158bc0 2023-11-24T06:12:18.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.504+0000 7f1a1ca42700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19fc06b470 0x7f19fc06d930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007fd0 tx=0x7f1a0c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.508+0000 7f1a157fa700 1 -- 172.21.15.97:0/2028629556 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a04054d60 con 0x7f1a18158bc0 2023-11-24T06:12:18.733 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.729+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp6193", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a180cf200 con 0x7f19fc06b470 2023-11-24T06:12:18.738 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:12:18.735+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:18.738 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:12:18.735+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:18.738 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:12:18.735+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:18.738 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:12:18.735+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:18.739 INFO:tasks.ceph.mgr.y.smithi097.stderr:2023-11-24T06:12:18.735+0000 7fc9c0830700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:12:18.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.750+0000 7f1a157fa700 1 -- 172.21.15.97:0/2028629556 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1a0c022070 con 0x7f19fc06b470 2023-11-24T06:12:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19fc06b470 msgr2=0x7f19fc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19fc06b470 0x7f19fc06d930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c007fd0 tx=0x7f1a0c009030 comp rx=0 tx=0).stop 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18158bc0 msgr2=0x7f1a181513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18158bc0 0x7f1a181513b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1a040119d0 tx=0x7f1a04011ce0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 shutdown_connections 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a181518f0 0x7f1a181457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19fc06b470 0x7f19fc06d930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1a08003a30 0x7f1a18146e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 --2- 172.21.15.97:0/2028629556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a18158bc0 0x7f1a181513b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 >> 172.21.15.97:0/2028629556 conn(0x7f1a180b2050 msgr2=0x7f1a180b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 shutdown_connections 2023-11-24T06:12:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.752+0000 7f1a1eca6700 1 -- 172.21.15.97:0/2028629556 wait complete. 2023-11-24T06:12:18.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol6879 --group_name subvol_grp6193 2023-11-24T06:12:18.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.890+0000 7fd46ef99700 1 Processor -- start 2023-11-24T06:12:18.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.890+0000 7fd46ef99700 1 -- start start 2023-11-24T06:12:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.890+0000 7fd46ef99700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd46806c710 0x7fd46806df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.890+0000 7fd46ef99700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd468145050 con 0x7fd468145280 2023-11-24T06:12:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.890+0000 7fd46ef99700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4680729e0 con 0x7fd468144c70 2023-11-24T06:12:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.890+0000 7fd46ef99700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4680cfcb0 con 0x7fd46806c710 2023-11-24T06:12:18.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd46d536700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd468145280 0x7fd46806e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43814/0 (socket says 172.21.15.97:43814) 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd46d536700 1 -- 172.21.15.97:0/4207419461 learned_addr learned my addr 172.21.15.97:0/4207419461 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd467fff700 1 --2- 172.21.15.97:0/4207419461 >> v2:172.21.15.183:3300/0 conn(0x7fd46806c710 0x7fd46806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915824057 0 0) 0x7fd468145050 con 0x7fd468145280 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4580035b0 con 0x7fd468145280 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3775662335 0 0) 0x7fd4580035b0 con 0x7fd468145280 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 >> v2:172.21.15.183:3300/0 conn(0x7fd46806c710 msgr2=0x7fd46806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 --2- 172.21.15.97:0/4207419461 >> v2:172.21.15.183:3300/0 conn(0x7fd46806c710 0x7fd46806df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 >> v1:172.21.15.183:6789/0 conn(0x7fd468144c70 legacy=0x7fd46806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680cff90 con 0x7fd468145280 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fd4600019f0 con 0x7fd468145280 2023-11-24T06:12:18.897 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 >> v1:172.21.15.97:6789/0 conn(0x7fd468145280 legacy=0x7fd46806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd458003a00 0x7fd458003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458004850 0x7fd458004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458005210 0x7fd45800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680cfcb0 con 0x7fd458005210 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45800c0b0 con 0x7fd458003a00 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45800c270 con 0x7fd458004850 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd46d536700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458005210 0x7fd45800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd46cd35700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd458003a00 0x7fd458003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.891+0000 7fd467fff700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458004850 0x7fd458004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46d536700 1 -- 172.21.15.97:0/4207419461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458004850 msgr2=0x7fd458004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46d536700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458004850 0x7fd458004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.898 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46d536700 1 -- 172.21.15.97:0/4207419461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd458003a00 msgr2=0x7fd458003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46d536700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd458003a00 0x7fd458003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46d536700 1 -- 172.21.15.97:0/4207419461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4580035b0 con 0x7fd458005210 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46d536700 1 -- 172.21.15.97:0/4207419461 reap_dead start 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46d536700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458005210 0x7fd45800ba10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd46000a830 tx=0x7fd458005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa4db488fddd970 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46000a2f0 con 0x7fd458005210 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd460012600 con 0x7fd458005210 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd4677fe700 1 -- 172.21.15.97:0/4207419461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600129b0 con 0x7fd458005210 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46ef99700 1 -- 172.21.15.97:0/4207419461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458005210 msgr2=0x7fd45800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46ef99700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458005210 0x7fd45800ba10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd46000a830 tx=0x7fd458005880 comp rx=0 tx=0).stop 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46ef99700 1 -- 172.21.15.97:0/4207419461 shutdown_connections 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.892+0000 7fd46ef99700 1 --2- 172.21.15.97:0/4207419461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd458005210 0x7fd45800ba10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.899 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.893+0000 7fd46ef99700 1 -- 172.21.15.97:0/4207419461 >> 172.21.15.97:0/4207419461 conn(0x7fd468071b50 msgr2=0x7fd468071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:18.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.893+0000 7fd46ef99700 1 -- 172.21.15.97:0/4207419461 shutdown_connections 2023-11-24T06:12:18.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.893+0000 7fd46ef99700 1 -- 172.21.15.97:0/4207419461 wait complete. 2023-11-24T06:12:18.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.894+0000 7fd46ef99700 1 Processor -- start 2023-11-24T06:12:18.900 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.894+0000 7fd46ef99700 1 -- start start 2023-11-24T06:12:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46ef99700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458005210 0x7fd46815bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46ef99700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4680b55a0 0x7fd4681e5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.902 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46ef99700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4680b5a20 0x7fd4681e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46ef99700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680b3cc0 con 0x7fd4680b5a20 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46ef99700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680b3f10 con 0x7fd4680b55a0 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46ef99700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4680b41f0 con 0x7fd458005210 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4680b5a20 0x7fd4681e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4680b5a20 0x7fd4681e5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51376/0 (socket says 172.21.15.97:51376) 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 -- 172.21.15.97:0/224365239 learned_addr learned my addr 172.21.15.97:0/224365239 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46cd35700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458005210 0x7fd46815bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd467fff700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4680b55a0 0x7fd4681e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 -- 172.21.15.97:0/224365239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458005210 msgr2=0x7fd46815bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458005210 0x7fd46815bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.903 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 -- 172.21.15.97:0/224365239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4680b55a0 msgr2=0x7fd4681e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4680b55a0 0x7fd4681e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 -- 172.21.15.97:0/224365239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680b44d0 con 0x7fd4680b5a20 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46d536700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4680b5a20 0x7fd4681e5ec0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd4680ceb90 tx=0x7fd460001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd4657fa700 1 -- 172.21.15.97:0/224365239 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460001250 con 0x7fd4680b5a20 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd4657fa700 1 -- 172.21.15.97:0/224365239 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4600017b0 con 0x7fd4680b5a20 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd4657fa700 1 -- 172.21.15.97:0/224365239 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46001d420 con 0x7fd4680b5a20 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4680b47b0 con 0x7fd4680b5a20 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.895+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd468140740 con 0x7fd4680b5a20 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.896+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4680b4330 con 0x7fd4680b5a20 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.897+0000 7fd4657fa700 1 -- 172.21.15.97:0/224365239 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd46001d580 con 0x7fd4680b5a20 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.897+0000 7fd4657fa700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd44c06b470 0x7fd44c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:18.904 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.897+0000 7fd4657fa700 1 -- 172.21.15.97:0/224365239 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd46008b2e0 con 0x7fd4680b5a20 2023-11-24T06:12:18.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.897+0000 7fd46cd35700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd44c06b470 0x7fd44c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:18.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.898+0000 7fd46cd35700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd44c06b470 0x7fd44c06d930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd4681410e0 tx=0x7fd45c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:18.905 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:18.902+0000 7fd4657fa700 1 -- 172.21.15.97:0/224365239 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4680b4330 con 0x7fd4680b5a20 2023-11-24T06:12:19.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.120+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol6879", "group_name": "subvol_grp6193", "target": ["mon-mgr", ""]}) v1 -- 0x7fd468072670 con 0x7fd44c06b470 2023-11-24T06:12:19.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.146+0000 7fd4657fa700 1 -- 172.21.15.97:0/224365239 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd45c001cc0 con 0x7fd44c06b470 2023-11-24T06:12:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd44c06b470 msgr2=0x7fd44c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd44c06b470 0x7fd44c06d930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd4681410e0 tx=0x7fd45c009040 comp rx=0 tx=0).stop 2023-11-24T06:12:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4680b5a20 msgr2=0x7fd4681e5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4680b5a20 0x7fd4681e5ec0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd4680ceb90 tx=0x7fd460001d80 comp rx=0 tx=0).stop 2023-11-24T06:12:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 shutdown_connections 2023-11-24T06:12:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4680b5a20 0x7fd4681e5ec0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd44c06b470 0x7fd44c06d930 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd458005210 0x7fd46815bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 --2- 172.21.15.97:0/224365239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4680b55a0 0x7fd4681e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 >> 172.21.15.97:0/224365239 conn(0x7fd468071b50 msgr2=0x7fd468071f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 shutdown_connections 2023-11-24T06:12:19.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.148+0000 7fd46ef99700 1 -- 172.21.15.97:0/224365239 wait complete. 2023-11-24T06:12:19.163 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:12:19.163 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:19.202 DEBUG:teuthology.orchestra.run.smithi183:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:12:19.257 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:12:19.257 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:12:19.258 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:19.275 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:12:19.275 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:19.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.489+0000 7f379e5d7700 1 Processor -- start 2023-11-24T06:12:19.492 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.489+0000 7f379e5d7700 1 -- start start 2023-11-24T06:12:19.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f379e5d7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f37980b0fa0 0x7f37980afc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f379e5d7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 0x7f37980ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f379e5d7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37980d10a0 con 0x7f37980b1380 2023-11-24T06:12:19.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f379e5d7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37980ca6d0 con 0x7f37980b0fa0 2023-11-24T06:12:19.496 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f379e5d7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37980622a0 con 0x7f37980d0c40 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f379cb74700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f37980b1380 0x7f37980ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43834/0 (socket says 172.21.15.97:43834) 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f379cb74700 1 -- 172.21.15.97:0/1225166077 learned_addr learned my addr 172.21.15.97:0/1225166077 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f3797fff700 1 --2- 172.21.15.97:0/1225166077 >> v2:172.21.15.183:3300/0 conn(0x7f37980b0fa0 0x7f37980afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070660570 0 0) 0x7f37980d10a0 con 0x7f37980b1380 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37800035b0 con 0x7f37980b1380 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.490+0000 7f37977fe700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 0x7f37980ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2298801881 0 0) 0x7f37800035b0 con 0x7f37980b1380 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 msgr2=0x7f37980ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 0x7f37980ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 >> v2:172.21.15.183:3300/0 conn(0x7f37980b0fa0 msgr2=0x7f37980afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 --2- 172.21.15.97:0/1225166077 >> v2:172.21.15.183:3300/0 conn(0x7f37980b0fa0 0x7f37980afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798062580 con 0x7f37980b1380 2023-11-24T06:12:19.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f37900019f0 con 0x7f37980b1380 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f37977fe700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 0x7f37980ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 >> v1:172.21.15.97:6789/0 conn(0x7f37980b1380 legacy=0x7f37980ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3780003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3780004850 0x7f3780004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 0x7f3780005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37980ca6d0 con 0x7f3780003a00 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37980622a0 con 0x7f3780004850 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378000ba90 con 0x7f37980d0c40 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3797fff700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3780003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f37977fe700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3780004850 0x7f3780004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.498 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f379cb74700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 0x7f3780005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3797fff700 1 -- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 msgr2=0x7f3780005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3797fff700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 0x7f3780005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3797fff700 1 -- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3780004850 msgr2=0x7f3780004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3797fff700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3780004850 0x7f3780004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3797fff700 1 -- 172.21.15.97:0/1225166077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37800035b0 con 0x7f3780003a00 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f379cb74700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f37980d0c40 0x7f3780005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3797fff700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3780003e20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f378c009b60 tx=0x7f3780005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a150a1d0e41f2a6d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00e6c0 con 0x7f3780003a00 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c00e820 con 0x7f3780003a00 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.491+0000 7f3796ffd700 1 -- 172.21.15.97:0/1225166077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00eb90 con 0x7f3780003a00 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.492+0000 7f379e5d7700 1 -- 172.21.15.97:0/1225166077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 msgr2=0x7f3780003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.492+0000 7f379e5d7700 1 --2- 172.21.15.97:0/1225166077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3780003e20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f378c009b60 tx=0x7f3780005880 comp rx=0 tx=0).stop 2023-11-24T06:12:19.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.492+0000 7f379cb74700 1 -- 172.21.15.97:0/1225166077 reap_dead start 2023-11-24T06:12:19.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.492+0000 7f379e5d7700 1 -- 172.21.15.97:0/1225166077 shutdown_connections 2023-11-24T06:12:19.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.492+0000 7f379e5d7700 1 -- 172.21.15.97:0/1225166077 >> 172.21.15.97:0/1225166077 conn(0x7f37980b57b0 msgr2=0x7f37980b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:19.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.492+0000 7f379e5d7700 1 -- 172.21.15.97:0/1225166077 shutdown_connections 2023-11-24T06:12:19.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.492+0000 7f379e5d7700 1 -- 172.21.15.97:0/1225166077 wait complete. 2023-11-24T06:12:19.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.493+0000 7f379e5d7700 1 Processor -- start 2023-11-24T06:12:19.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.493+0000 7f379e5d7700 1 -- start start 2023-11-24T06:12:19.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f379e5d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3798154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.502 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f379e5d7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37980cd580 0x7f3798157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f379e5d7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3798158340 0x7f37980b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f379e5d7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37981587c0 con 0x7f3780003a00 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f379e5d7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3798156200 con 0x7f37980cd580 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f379e5d7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3798159020 con 0x7f3798158340 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f3797fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3798154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f3797fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3798154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51396/0 (socket says 172.21.15.97:51396) 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f3797fff700 1 -- 172.21.15.97:0/3432547678 learned_addr learned my addr 172.21.15.97:0/3432547678 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f37977fe700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37980cd580 0x7f3798157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.494+0000 7f379cb74700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3798158340 0x7f37980b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3797fff700 1 -- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3798158340 msgr2=0x7f37980b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.503 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3797fff700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3798158340 0x7f37980b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3797fff700 1 -- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37980cd580 msgr2=0x7f3798157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3797fff700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37980cd580 0x7f3798157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3797fff700 1 -- 172.21.15.97:0/3432547678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798156450 con 0x7f3780003a00 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f379cb74700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3798158340 0x7f37980b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3797fff700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3798154cb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f378c00e4b0 tx=0x7f378c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3794ff9700 1 -- 172.21.15.97:0/3432547678 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00e6c0 con 0x7f3780003a00 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3794ff9700 1 -- 172.21.15.97:0/3432547678 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c0038b0 con 0x7f3780003a00 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37981589d0 con 0x7f3780003a00 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f3794ff9700 1 -- 172.21.15.97:0/3432547678 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c003c40 con 0x7f3780003a00 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.495+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37981594b0 con 0x7f3780003a00 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.496+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3798067f60 con 0x7f3780003a00 2023-11-24T06:12:19.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.496+0000 7f3794ff9700 1 -- 172.21.15.97:0/3432547678 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f378c013470 con 0x7f3780003a00 2023-11-24T06:12:19.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.497+0000 7f3794ff9700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f378406b470 0x7f378406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.497+0000 7f37977fe700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f378406b470 0x7f378406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.497+0000 7f3794ff9700 1 -- 172.21.15.97:0/3432547678 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f378c08b650 con 0x7f3780003a00 2023-11-24T06:12:19.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.497+0000 7f37977fe700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f378406b470 0x7f378406d930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f379806ed30 tx=0x7f3788008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.501+0000 7f3794ff9700 1 -- 172.21.15.97:0/3432547678 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f378c056000 con 0x7f3780003a00 2023-11-24T06:12:19.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.669+0000 7f3794ff9700 1 -- 172.21.15.97:0/3432547678 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f378c0096f0 con 0x7f3780003a00 2023-11-24T06:12:19.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.696+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3798158780 con 0x7f3780003a00 2023-11-24T06:12:19.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.697+0000 7f3794ff9700 1 -- 172.21.15.97:0/3432547678 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f378c0288c0 con 0x7f3780003a00 2023-11-24T06:12:19.700 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:19.700 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:09.577392+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:05.739467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:06.934719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f378406b470 msgr2=0x7f378406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f378406b470 0x7f378406d930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f379806ed30 tx=0x7f3788008040 comp rx=0 tx=0).stop 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 msgr2=0x7f3798154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3798154cb0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f378c00e4b0 tx=0x7f378c0047c0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 shutdown_connections 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f378406b470 0x7f378406d930 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3798158340 0x7f37980b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3780003a00 0x7f3798154cb0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 --2- 172.21.15.97:0/3432547678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37980cd580 0x7f3798157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 >> 172.21.15.97:0/3432547678 conn(0x7f37980b57b0 msgr2=0x7f37980d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:19.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 shutdown_connections 2023-11-24T06:12:19.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.700+0000 7f379e5d7700 1 -- 172.21.15.97:0/3432547678 wait complete. 2023-11-24T06:12:19.715 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:12:05.739467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:12:06.934719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:19.716 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:19.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.836+0000 7f88369c5700 1 Processor -- start 2023-11-24T06:12:19.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.836+0000 7f88369c5700 1 -- start start 2023-11-24T06:12:19.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.837+0000 7f88369c5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f883015a620 0x7f88300ce150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.837+0000 7f88369c5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f883006ecf0 0x7f88300b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.837+0000 7f88369c5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88300b2f20 con 0x7f88300cc540 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.837+0000 7f88369c5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88300ce690 con 0x7f883015a620 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.837+0000 7f88369c5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883015aa00 con 0x7f883006ecf0 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.837+0000 7f882ffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f883015a620 0x7f88300ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.837+0000 7f882f7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f88300cc540 0x7f883006e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43856/0 (socket says 172.21.15.97:43856) 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.837+0000 7f882f7fe700 1 -- 172.21.15.97:0/3157090824 learned_addr learned my addr 172.21.15.97:0/3157090824 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882ffff700 1 -- 172.21.15.97:0/3157090824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f883006ecf0 msgr2=0x7f88300b3350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882ffff700 1 --2- 172.21.15.97:0/3157090824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f883006ecf0 0x7f88300b3350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882ffff700 1 -- 172.21.15.97:0/3157090824 >> v1:172.21.15.97:6789/0 conn(0x7f88300cc540 legacy=0x7f883006e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882effd700 1 -- 172.21.15.97:0/3157090824 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103294279 0 0) 0x7f88300b2f20 con 0x7f88300cc540 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882ffff700 1 -- 172.21.15.97:0/3157090824 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88300ce960 con 0x7f883015a620 2023-11-24T06:12:19.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882ffff700 1 --2- 172.21.15.97:0/3157090824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f88300ce150 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8824009840 tx=0x7f8824009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f70331740948faa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882effd700 1 -- 172.21.15.97:0/3157090824 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824011830 con 0x7f883015a620 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882effd700 1 -- 172.21.15.97:0/3157090824 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8824011990 con 0x7f883015a620 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f882effd700 1 -- 172.21.15.97:0/3157090824 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824011d20 con 0x7f883015a620 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f88369c5700 1 -- 172.21.15.97:0/3157090824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 msgr2=0x7f88300ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.838+0000 7f88369c5700 1 --2- 172.21.15.97:0/3157090824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f88300ce150 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8824009840 tx=0x7f8824009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.839+0000 7f88369c5700 1 -- 172.21.15.97:0/3157090824 shutdown_connections 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.839+0000 7f88369c5700 1 --2- 172.21.15.97:0/3157090824 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f883006ecf0 0x7f88300b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.839+0000 7f88369c5700 1 --2- 172.21.15.97:0/3157090824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f88300ce150 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.839+0000 7f88369c5700 1 -- 172.21.15.97:0/3157090824 >> 172.21.15.97:0/3157090824 conn(0x7f88300b2050 msgr2=0x7f88300b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.839+0000 7f88369c5700 1 -- 172.21.15.97:0/3157090824 shutdown_connections 2023-11-24T06:12:19.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.839+0000 7f88369c5700 1 -- 172.21.15.97:0/3157090824 wait complete. 2023-11-24T06:12:19.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.840+0000 7f88369c5700 1 Processor -- start 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.840+0000 7f88369c5700 1 -- start start 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f88369c5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f883006ecf0 0x7f883014e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f88369c5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88300cc540 0x7f8830149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f88369c5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f8830149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f88369c5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883014cfa0 con 0x7f883015a620 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f88369c5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883014d280 con 0x7f88300cc540 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f88369c5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883014d560 con 0x7f883006ecf0 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f8830149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f8830149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51412/0 (socket says 172.21.15.97:51412) 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 -- 172.21.15.97:0/2673792495 learned_addr learned my addr 172.21.15.97:0/2673792495 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 -- 172.21.15.97:0/2673792495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f883006ecf0 msgr2=0x7f883014e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f883006ecf0 0x7f883014e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 -- 172.21.15.97:0/2673792495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88300cc540 msgr2=0x7f8830149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88300cc540 0x7f8830149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 -- 172.21.15.97:0/2673792495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88240094a0 con 0x7f883015a620 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f8834f62700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f8830149dc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f882800b7b0 tx=0x7f882800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f882cff9700 1 -- 172.21.15.97:0/2673792495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800f840 con 0x7f883015a620 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f882cff9700 1 -- 172.21.15.97:0/2673792495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882800f9a0 con 0x7f883015a620 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883014d840 con 0x7f883015a620 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f882cff9700 1 -- 172.21.15.97:0/2673792495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800fd50 con 0x7f883015a620 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.841+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883014e580 con 0x7f883015a620 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.842+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8830068d90 con 0x7f883015a620 2023-11-24T06:12:19.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.843+0000 7f882cff9700 1 -- 172.21.15.97:0/2673792495 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f88280203f0 con 0x7f883015a620 2023-11-24T06:12:19.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.843+0000 7f882cff9700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f881c06b500 0x7f881c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:19.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.843+0000 7f882cff9700 1 -- 172.21.15.97:0/2673792495 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f882808a510 con 0x7f883015a620 2023-11-24T06:12:19.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.843+0000 7f882ffff700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f881c06b500 0x7f881c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:19.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.844+0000 7f882ffff700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f881c06b500 0x7f881c06d9c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8824000c00 tx=0x7f8824010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:19.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:19.848+0000 7f882cff9700 1 -- 172.21.15.97:0/2673792495 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8828054be0 con 0x7f883015a620 2023-11-24T06:12:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.038+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f883014e7a0 con 0x7f883015a620 2023-11-24T06:12:20.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.038+0000 7f882cff9700 1 -- 172.21.15.97:0/2673792495 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8828058200 con 0x7f883015a620 2023-11-24T06:12:20.041 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:12:20.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f881c06b500 msgr2=0x7f881c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f881c06b500 0x7f881c06d9c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8824000c00 tx=0x7f8824010000 comp rx=0 tx=0).stop 2023-11-24T06:12:20.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 msgr2=0x7f8830149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f8830149dc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f882800b7b0 tx=0x7f882800bb70 comp rx=0 tx=0).stop 2023-11-24T06:12:20.043 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 shutdown_connections 2023-11-24T06:12:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f883015a620 0x7f8830149dc0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f881c06b500 0x7f881c06d9c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f883006ecf0 0x7f883014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 --2- 172.21.15.97:0/2673792495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88300cc540 0x7f8830149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 >> 172.21.15.97:0/2673792495 conn(0x7f88300b2050 msgr2=0x7f8830153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 shutdown_connections 2023-11-24T06:12:20.044 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.040+0000 7f88369c5700 1 -- 172.21.15.97:0/2673792495 wait complete. 2023-11-24T06:12:20.055 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.175+0000 7f459360f700 1 Processor -- start 2023-11-24T06:12:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.175+0000 7f459360f700 1 -- start start 2023-11-24T06:12:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f459360f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f458c071ac0 0x7f458c0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f459360f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c0d2a60 con 0x7f458c071610 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f459360f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c0d2d40 con 0x7f458c15e090 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f459360f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0d3020 con 0x7f458c071ac0 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f4590baa700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f458c071610 0x7f458c0b6fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43870/0 (socket says 172.21.15.97:43870) 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f4590baa700 1 -- 172.21.15.97:0/413020341 learned_addr learned my addr 172.21.15.97:0/413020341 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f4591bac700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f458c071ac0 0x7f458c0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449090995 0 0) 0x7f458c0d2a60 con 0x7f458c071610 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.176+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45780035b0 con 0x7f458c071610 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2024930359 0 0) 0x7f45780035b0 con 0x7f458c071610 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f458c071ac0 msgr2=0x7f458c0b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f458c071ac0 0x7f458c0b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 >> v1:172.21.15.183:6789/0 conn(0x7f458c15e090 legacy=0x7f458c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0d3300 con 0x7f458c071610 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f4588001670 con 0x7f458c071610 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 >> v1:172.21.15.97:6789/0 conn(0x7f458c071610 legacy=0x7f458c0b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f458c071ac0 0x7f4578003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578003f70 0x7f4578004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4578004d90 0x7f4578005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0d3020 con 0x7f4578003f70 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457800ba70 con 0x7f4578004d90 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457800bc30 con 0x7f458c071ac0 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4590baa700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578003f70 0x7f4578004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f45913ab700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f458c071ac0 0x7f4578003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4591bac700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4578004d90 0x7f4578005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4590baa700 1 -- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f458c071ac0 msgr2=0x7f4578003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4590baa700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f458c071ac0 0x7f4578003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4590baa700 1 -- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4578004d90 msgr2=0x7f4578005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4590baa700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4578004d90 0x7f4578005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4590baa700 1 -- 172.21.15.97:0/413020341 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45780035b0 con 0x7f4578003f70 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4591bac700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4578004d90 0x7f4578005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4590baa700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578003f70 0x7f4578004850 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f458800c2f0 tx=0x7f4578005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1842a23d1c81238 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458800cd70 con 0x7f4578003f70 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4588005960 con 0x7f4578003f70 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.177+0000 7f4583fff700 1 -- 172.21.15.97:0/413020341 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588005d10 con 0x7f4578003f70 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.178+0000 7f459360f700 1 -- 172.21.15.97:0/413020341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578003f70 msgr2=0x7f4578004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.178+0000 7f459360f700 1 --2- 172.21.15.97:0/413020341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4578003f70 0x7f4578004850 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f458800c2f0 tx=0x7f4578005880 comp rx=0 tx=0).stop 2023-11-24T06:12:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.178+0000 7f4591bac700 1 -- 172.21.15.97:0/413020341 reap_dead start 2023-11-24T06:12:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.178+0000 7f459360f700 1 -- 172.21.15.97:0/413020341 shutdown_connections 2023-11-24T06:12:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.178+0000 7f459360f700 1 -- 172.21.15.97:0/413020341 >> 172.21.15.97:0/413020341 conn(0x7f458c06b410 msgr2=0x7f458c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.178+0000 7f459360f700 1 -- 172.21.15.97:0/413020341 shutdown_connections 2023-11-24T06:12:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.178+0000 7f459360f700 1 -- 172.21.15.97:0/413020341 wait complete. 2023-11-24T06:12:20.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.179+0000 7f459360f700 1 Processor -- start 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.179+0000 7f459360f700 1 -- start start 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f459360f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4578003f70 0x7f458c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f459360f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f458c155300 0x7f458c151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f459360f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f458c155780 0x7f458c151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f459360f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c151cb0 con 0x7f458c155300 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f459360f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c160f20 con 0x7f458c155780 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f459360f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c160410 con 0x7f4578003f70 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4590baa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f458c155300 0x7f458c151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4590baa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f458c155300 0x7f458c151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51430/0 (socket says 172.21.15.97:51430) 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4590baa700 1 -- 172.21.15.97:0/597761256 learned_addr learned my addr 172.21.15.97:0/597761256 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f45913ab700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4578003f70 0x7f458c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4591bac700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f458c155780 0x7f458c151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4590baa700 1 -- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4578003f70 msgr2=0x7f458c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4590baa700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4578003f70 0x7f458c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4590baa700 1 -- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f458c155780 msgr2=0x7f458c151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4590baa700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f458c155780 0x7f458c151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4590baa700 1 -- 172.21.15.97:0/597761256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c160680 con 0x7f458c155300 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f45913ab700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4578003f70 0x7f458c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.180+0000 7f4591bac700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f458c155780 0x7f458c151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.181+0000 7f4590baa700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f458c155300 0x7f458c151230 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f457800ba40 tx=0x7f4588001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.181+0000 7f4581ffb700 1 -- 172.21.15.97:0/597761256 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588004a80 con 0x7f458c155300 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.181+0000 7f4581ffb700 1 -- 172.21.15.97:0/597761256 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4588004c00 con 0x7f458c155300 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.181+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c160960 con 0x7f458c155300 2023-11-24T06:12:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.181+0000 7f4581ffb700 1 -- 172.21.15.97:0/597761256 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880070d0 con 0x7f458c155300 2023-11-24T06:12:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.181+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c15f690 con 0x7f458c155300 2023-11-24T06:12:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.182+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c0b6f70 con 0x7f458c155300 2023-11-24T06:12:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.182+0000 7f4581ffb700 1 -- 172.21.15.97:0/597761256 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f4588019070 con 0x7f458c155300 2023-11-24T06:12:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.183+0000 7f4581ffb700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f457006b500 0x7f457006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.183+0000 7f4581ffb700 1 -- 172.21.15.97:0/597761256 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f458808a600 con 0x7f458c155300 2023-11-24T06:12:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.183+0000 7f45913ab700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f457006b500 0x7f457006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.183+0000 7f45913ab700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f457006b500 0x7f457006d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4584002230 tx=0x7f4584009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.187+0000 7f4581ffb700 1 -- 172.21.15.97:0/597761256 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4588054de0 con 0x7f458c155300 2023-11-24T06:12:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.378+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f458c068fb0 con 0x7f458c155300 2023-11-24T06:12:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.378+0000 7f4581ffb700 1 -- 172.21.15.97:0/597761256 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4588007da0 con 0x7f458c155300 2023-11-24T06:12:20.381 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:12:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.380+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f457006b500 msgr2=0x7f457006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.380+0000 7f459360f700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f457006b500 0x7f457006d9c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4584002230 tx=0x7f4584009040 comp rx=0 tx=0).stop 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.380+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f458c155300 msgr2=0x7f458c151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.380+0000 7f459360f700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f458c155300 0x7f458c151230 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f457800ba40 tx=0x7f4588001a00 comp rx=0 tx=0).stop 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.381+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 shutdown_connections 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.381+0000 7f459360f700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f458c155780 0x7f458c151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.381+0000 7f459360f700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f457006b500 0x7f457006d9c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.381+0000 7f459360f700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4578003f70 0x7f458c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.381+0000 7f459360f700 1 --2- 172.21.15.97:0/597761256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f458c155300 0x7f458c151230 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.381+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 >> 172.21.15.97:0/597761256 conn(0x7f458c06b410 msgr2=0x7f458c06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.381+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 shutdown_connections 2023-11-24T06:12:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.381+0000 7f459360f700 1 -- 172.21.15.97:0/597761256 wait complete. 2023-11-24T06:12:20.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.516+0000 7f4901357700 1 Processor -- start 2023-11-24T06:12:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.516+0000 7f4901357700 1 -- start start 2023-11-24T06:12:20.521 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f4901357700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc0aedc0 0x7f48fc0ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f4901357700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f48fc15e290 0x7f48fc0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f4901357700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc06e400 con 0x7f48fc0ad870 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f4901357700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc06fde0 con 0x7f48fc15e290 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f4901357700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0af6e0 con 0x7f48fc0aedc0 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f48fb7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f48fc0ad870 0x7f48fc06f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43892/0 (socket says 172.21.15.97:43892) 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f48fb7fe700 1 -- 172.21.15.97:0/740150194 learned_addr learned my addr 172.21.15.97:0/740150194 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f48faffd700 1 --2- 172.21.15.97:0/740150194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc0aedc0 0x7f48fc0ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f48fa7fc700 1 --2- 172.21.15.97:0/740150194 >> v2:172.21.15.183:3300/0 conn(0x7f48fc15e290 0x7f48fc0af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.517+0000 7f48f9ffb700 1 -- 172.21.15.97:0/740150194 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692037192 0 0) 0x7f48fc06e400 con 0x7f48fc0ad870 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48f9ffb700 1 -- 172.21.15.97:0/740150194 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e40035b0 con 0x7f48fc0ad870 2023-11-24T06:12:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48faffd700 1 -- 172.21.15.97:0/740150194 >> v2:172.21.15.183:3300/0 conn(0x7f48fc15e290 msgr2=0x7f48fc0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48faffd700 1 --2- 172.21.15.97:0/740150194 >> v2:172.21.15.183:3300/0 conn(0x7f48fc15e290 0x7f48fc0af1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48faffd700 1 -- 172.21.15.97:0/740150194 >> v1:172.21.15.97:6789/0 conn(0x7f48fc0ad870 legacy=0x7f48fc06f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48faffd700 1 -- 172.21.15.97:0/740150194 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc06e630 con 0x7f48fc0aedc0 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48faffd700 1 --2- 172.21.15.97:0/740150194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc0aedc0 0x7f48fc0ae790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009840 tx=0x7f48f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=782e213f585ade60 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48f9ffb700 1 -- 172.21.15.97:0/740150194 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0011830 con 0x7f48fc0aedc0 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48f9ffb700 1 -- 172.21.15.97:0/740150194 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f0011990 con 0x7f48fc0aedc0 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.518+0000 7f48f9ffb700 1 -- 172.21.15.97:0/740150194 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0011d00 con 0x7f48fc0aedc0 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.519+0000 7f4901357700 1 -- 172.21.15.97:0/740150194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc0aedc0 msgr2=0x7f48fc0ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.519+0000 7f4901357700 1 --2- 172.21.15.97:0/740150194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc0aedc0 0x7f48fc0ae790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009840 tx=0x7f48f0009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.519+0000 7f4901357700 1 -- 172.21.15.97:0/740150194 shutdown_connections 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.519+0000 7f4901357700 1 --2- 172.21.15.97:0/740150194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc0aedc0 0x7f48fc0ae790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.519+0000 7f4901357700 1 --2- 172.21.15.97:0/740150194 >> v2:172.21.15.183:3300/0 conn(0x7f48fc15e290 0x7f48fc0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.519+0000 7f4901357700 1 -- 172.21.15.97:0/740150194 >> 172.21.15.97:0/740150194 conn(0x7f48fc0b2050 msgr2=0x7f48fc0b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.519+0000 7f4901357700 1 -- 172.21.15.97:0/740150194 shutdown_connections 2023-11-24T06:12:20.524 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.519+0000 7f4901357700 1 -- 172.21.15.97:0/740150194 wait complete. 2023-11-24T06:12:20.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.520+0000 7f4901357700 1 Processor -- start 2023-11-24T06:12:20.526 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.520+0000 7f4901357700 1 -- start start 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f4901357700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48fc0ad870 0x7f48fc0cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f4901357700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48fc0aedc0 0x7f48fc0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f4901357700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc15e290 0x7f48fc0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f4901357700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0cc100 con 0x7f48fc0aedc0 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f4901357700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0c5460 con 0x7f48fc0ad870 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f4901357700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc0c5740 con 0x7f48fc15e290 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48fc0aedc0 0x7f48fc0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48fc0aedc0 0x7f48fc0cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51444/0 (socket says 172.21.15.97:51444) 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 -- 172.21.15.97:0/2644866285 learned_addr learned my addr 172.21.15.97:0/2644866285 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48faffd700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48fc0ad870 0x7f48fc0cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fb7fe700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc15e290 0x7f48fc0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 -- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc15e290 msgr2=0x7f48fc0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc15e290 0x7f48fc0cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.527 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 -- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48fc0ad870 msgr2=0x7f48fc0cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48fc0ad870 0x7f48fc0cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 -- 172.21.15.97:0/2644866285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f00094a0 con 0x7f48fc0aedc0 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48faffd700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48fc0ad870 0x7f48fc0cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48fa7fc700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48fc0aedc0 0x7f48fc0cef00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00e840 tx=0x7f48ec00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48e37fe700 1 -- 172.21.15.97:0/2644866285 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec009590 con 0x7f48fc0aedc0 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48e37fe700 1 -- 172.21.15.97:0/2644866285 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec013680 con 0x7f48fc0aedc0 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.521+0000 7f48e37fe700 1 -- 172.21.15.97:0/2644866285 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec013a30 con 0x7f48fc0aedc0 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.522+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc0c5a20 con 0x7f48fc0aedc0 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.522+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc0c5fe0 con 0x7f48fc0aedc0 2023-11-24T06:12:20.528 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.522+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc068d90 con 0x7f48fc0aedc0 2023-11-24T06:12:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.523+0000 7f48e37fe700 1 -- 172.21.15.97:0/2644866285 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f48ec013b90 con 0x7f48fc0aedc0 2023-11-24T06:12:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.523+0000 7f48e37fe700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f48e806b530 0x7f48e806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.523+0000 7f48e37fe700 1 -- 172.21.15.97:0/2644866285 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f48ec08a0e0 con 0x7f48fc0aedc0 2023-11-24T06:12:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.523+0000 7f48faffd700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f48e806b530 0x7f48e806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.529 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.524+0000 7f48faffd700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f48e806b530 0x7f48e806d9f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f48f0000c00 tx=0x7f48f0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.530 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.528+0000 7f48e37fe700 1 -- 172.21.15.97:0/2644866285 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48ec0bce90 con 0x7f48fc0aedc0 2023-11-24T06:12:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.720+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48fc0c6200 con 0x7f48fc0aedc0 2023-11-24T06:12:20.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.720+0000 7f48e37fe700 1 -- 172.21.15.97:0/2644866285 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48ec0210a0 con 0x7f48fc0aedc0 2023-11-24T06:12:20.723 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f48e806b530 msgr2=0x7f48e806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f48e806b530 0x7f48e806d9f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f48f0000c00 tx=0x7f48f0008030 comp rx=0 tx=0).stop 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48fc0aedc0 msgr2=0x7f48fc0cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48fc0aedc0 0x7f48fc0cef00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00e840 tx=0x7f48ec00ec00 comp rx=0 tx=0).stop 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 shutdown_connections 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f48e806b530 0x7f48e806d9f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f48fc15e290 0x7f48fc0cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48fc0ad870 0x7f48fc0cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.726 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 --2- 172.21.15.97:0/2644866285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48fc0aedc0 0x7f48fc0cef00 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 >> 172.21.15.97:0/2644866285 conn(0x7f48fc0b2050 msgr2=0x7f48fc0b4540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 shutdown_connections 2023-11-24T06:12:20.727 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.723+0000 7f4901357700 1 -- 172.21.15.97:0/2644866285 wait complete. 2023-11-24T06:12:20.738 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:12:20.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.856+0000 7f9a07e06700 1 Processor -- start 2023-11-24T06:12:20.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.856+0000 7f9a07e06700 1 -- start start 2023-11-24T06:12:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a07e06700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a00158670 0x7f9a000622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a07e06700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f9a000c9fc0 0x7f9a000ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a07e06700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a000cfff0 con 0x7f9a00158a50 2023-11-24T06:12:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a07e06700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000cfd10 con 0x7f9a00158670 2023-11-24T06:12:20.861 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a07e06700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a00161f50 con 0x7f9a000c9fc0 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a05ba2700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9a00158670 0x7f9a000622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a053a1700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f9a00158a50 0x7f9a000c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43904/0 (socket says 172.21.15.97:43904) 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a053a1700 1 -- 172.21.15.97:0/284156185 learned_addr learned my addr 172.21.15.97:0/284156185 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a063a3700 1 --2- 172.21.15.97:0/284156185 >> v2:172.21.15.183:3300/0 conn(0x7f9a000c9fc0 0x7f9a000ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a05ba2700 1 -- 172.21.15.97:0/284156185 >> v2:172.21.15.183:3300/0 conn(0x7f9a000c9fc0 msgr2=0x7f9a000ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a05ba2700 1 --2- 172.21.15.97:0/284156185 >> v2:172.21.15.183:3300/0 conn(0x7f9a000c9fc0 0x7f9a000ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a05ba2700 1 -- 172.21.15.97:0/284156185 >> v1:172.21.15.97:6789/0 conn(0x7f9a00158a50 legacy=0x7f9a000c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a05ba2700 1 -- 172.21.15.97:0/284156185 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00162230 con 0x7f9a00158670 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a04ba0700 1 -- 172.21.15.97:0/284156185 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534352291 0 0) 0x7f9a000cfff0 con 0x7f9a00158a50 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a063a3700 1 --2- 172.21.15.97:0/284156185 >> v2:172.21.15.183:3300/0 conn(0x7f9a000c9fc0 0x7f9a000ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:20.862 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a05ba2700 1 --2- 172.21.15.97:0/284156185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a000622a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9a000b0070 tx=0x7f99fc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84ff692524d5a29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a04ba0700 1 -- 172.21.15.97:0/284156185 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc011830 con 0x7f9a00158670 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a04ba0700 1 -- 172.21.15.97:0/284156185 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc011990 con 0x7f9a00158670 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.857+0000 7f9a04ba0700 1 -- 172.21.15.97:0/284156185 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc011d20 con 0x7f9a00158670 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.858+0000 7f9a07e06700 1 -- 172.21.15.97:0/284156185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 msgr2=0x7f9a000622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.858+0000 7f9a07e06700 1 --2- 172.21.15.97:0/284156185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a000622a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f9a000b0070 tx=0x7f99fc009b20 comp rx=0 tx=0).stop 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.858+0000 7f9a07e06700 1 -- 172.21.15.97:0/284156185 shutdown_connections 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.858+0000 7f9a07e06700 1 --2- 172.21.15.97:0/284156185 >> v2:172.21.15.183:3300/0 conn(0x7f9a000c9fc0 0x7f9a000ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.858+0000 7f9a07e06700 1 --2- 172.21.15.97:0/284156185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a000622a0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.858+0000 7f9a07e06700 1 -- 172.21.15.97:0/284156185 >> 172.21.15.97:0/284156185 conn(0x7f9a000b57b0 msgr2=0x7f9a000b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.858+0000 7f9a07e06700 1 -- 172.21.15.97:0/284156185 shutdown_connections 2023-11-24T06:12:20.863 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.858+0000 7f9a07e06700 1 -- 172.21.15.97:0/284156185 wait complete. 2023-11-24T06:12:20.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.860+0000 7f9a07e06700 1 Processor -- start 2023-11-24T06:12:20.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.860+0000 7f9a07e06700 1 -- start start 2023-11-24T06:12:20.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a07e06700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a000c9fc0 0x7f9a00153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.866 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a07e06700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a0015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a07e06700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a00158a50 0x7f9a0015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a07e06700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000b3b00 con 0x7f9a00158670 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a07e06700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000b4a30 con 0x7f9a000c9fc0 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a07e06700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000b4d10 con 0x7f9a00158a50 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a0015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a0015f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51458/0 (socket says 172.21.15.97:51458) 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 -- 172.21.15.97:0/2072388610 learned_addr learned my addr 172.21.15.97:0/2072388610 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a063a3700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a00158a50 0x7f9a0015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a05ba2700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a000c9fc0 0x7f9a00153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 -- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a00158a50 msgr2=0x7f9a0015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a00158a50 0x7f9a0015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.867 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 -- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a000c9fc0 msgr2=0x7f9a00153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a000c9fc0 0x7f9a00153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 -- 172.21.15.97:0/2072388610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc0094a0 con 0x7f9a00158670 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a05ba2700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a000c9fc0 0x7f9a00153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a063a3700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a00158a50 0x7f9a0015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a053a1700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a0015f4d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c460 tx=0x7f99f000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f99f67fc700 1 -- 172.21.15.97:0/2072388610 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0004c80 con 0x7f9a00158670 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f99f67fc700 1 -- 172.21.15.97:0/2072388610 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f0004e00 con 0x7f9a00158670 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a000b4ff0 con 0x7f9a00158670 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f99f67fc700 1 -- 172.21.15.97:0/2072388610 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0007310 con 0x7f9a00158670 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.861+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a000b4720 con 0x7f9a00158670 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.862+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a00067f60 con 0x7f9a00158670 2023-11-24T06:12:20.868 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.863+0000 7f99f67fc700 1 -- 172.21.15.97:0/2072388610 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f99f0007470 con 0x7f9a00158670 2023-11-24T06:12:20.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.863+0000 7f99f67fc700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:20.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.863+0000 7f99f67fc700 1 -- 172.21.15.97:0/2072388610 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f99f0089d70 con 0x7f9a00158670 2023-11-24T06:12:20.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.863+0000 7f9a05ba2700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:20.869 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.864+0000 7f9a05ba2700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99e406b500 0x7f99e406d9c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009b60 tx=0x7f99fc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:20.870 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:20.868+0000 7f99f67fc700 1 -- 172.21.15.97:0/2072388610 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99f0054550 con 0x7f9a00158670 2023-11-24T06:12:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.076+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9a00074a20 con 0x7f9a00158670 2023-11-24T06:12:21.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.076+0000 7f99f67fc700 1 -- 172.21.15.97:0/2072388610 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) v1 ==== 80+0+0 (secure 0 0 0) 0x7f99f0057b70 con 0x7f9a00158670 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99e406b500 msgr2=0x7f99e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99e406b500 0x7f99e406d9c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009b60 tx=0x7f99fc008030 comp rx=0 tx=0).stop 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 msgr2=0x7f9a0015f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a0015f4d0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c460 tx=0x7f99f000c820 comp rx=0 tx=0).stop 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 shutdown_connections 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9a00158a50 0x7f9a0015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a000c9fc0 0x7f9a00153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 --2- 172.21.15.97:0/2072388610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a00158670 0x7f9a0015f4d0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 >> 172.21.15.97:0/2072388610 conn(0x7f9a000b57b0 msgr2=0x7f9a000d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 shutdown_connections 2023-11-24T06:12:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.078+0000 7f9a07e06700 1 -- 172.21.15.97:0/2072388610 wait complete. 2023-11-24T06:12:21.082 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:12:21.092 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T06:12:21.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:21.093 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T06:12:21.135 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:12:21.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:21.135 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:12:21.192 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:12:21.192 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:21.213 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:12:21.213 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:21.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.392+0000 7f19c85c4700 1 Processor -- start 2023-11-24T06:12:21.395 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.393+0000 7f19c85c4700 1 -- start start 2023-11-24T06:12:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.393+0000 7f19c85c4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c01446a0 0x7f19c00b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.393+0000 7f19c85c4700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f19c006c710 0x7f19c00b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.393+0000 7f19c85c4700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c00af950 con 0x7f19c00adf00 2023-11-24T06:12:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.393+0000 7f19c85c4700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c0144a80 con 0x7f19c006c710 2023-11-24T06:12:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.393+0000 7f19c85c4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00afc30 con 0x7f19c01446a0 2023-11-24T06:12:21.397 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.394+0000 7f19c6b61700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f19c006c710 0x7f19c00b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.394+0000 7f19c6b61700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f19c006c710 0x7f19c00b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:45628/0 (socket says 172.21.15.97:45628) 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.394+0000 7f19c6b61700 1 -- 172.21.15.97:0/2101918922 learned_addr learned my addr 172.21.15.97:0/2101918922 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.394+0000 7f19c6b61700 1 -- 172.21.15.97:0/2101918922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c01446a0 msgr2=0x7f19c00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.394+0000 7f19c6b61700 1 --2- 172.21.15.97:0/2101918922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c01446a0 0x7f19c00b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.394+0000 7f19c6b61700 1 -- 172.21.15.97:0/2101918922 >> v1:172.21.15.183:6789/0 conn(0x7f19c00adf00 legacy=0x7f19c006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.394+0000 7f19c6b61700 1 -- 172.21.15.97:0/2101918922 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00aff10 con 0x7f19c006c710 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.394+0000 7f19c6b61700 1 --2- 172.21.15.97:0/2101918922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c006c710 0x7f19c00b3d40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f19b800d150 tx=0x7f19b800d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90096295702b5f01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c535e700 1 -- 172.21.15.97:0/2101918922 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b8010070 con 0x7f19c006c710 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c535e700 1 -- 172.21.15.97:0/2101918922 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800ddc0 con 0x7f19c006c710 2023-11-24T06:12:21.398 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c535e700 1 -- 172.21.15.97:0/2101918922 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800b310 con 0x7f19c006c710 2023-11-24T06:12:21.401 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c85c4700 1 -- 172.21.15.97:0/2101918922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c006c710 msgr2=0x7f19c00b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c85c4700 1 --2- 172.21.15.97:0/2101918922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c006c710 0x7f19c00b3d40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f19b800d150 tx=0x7f19b800d460 comp rx=0 tx=0).stop 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c85c4700 1 -- 172.21.15.97:0/2101918922 shutdown_connections 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c85c4700 1 --2- 172.21.15.97:0/2101918922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c006c710 0x7f19c00b3d40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c85c4700 1 --2- 172.21.15.97:0/2101918922 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c01446a0 0x7f19c00b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c85c4700 1 -- 172.21.15.97:0/2101918922 >> 172.21.15.97:0/2101918922 conn(0x7f19c0071b90 msgr2=0x7f19c0071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c85c4700 1 -- 172.21.15.97:0/2101918922 shutdown_connections 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.395+0000 7f19c85c4700 1 -- 172.21.15.97:0/2101918922 wait complete. 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.396+0000 7f19c85c4700 1 Processor -- start 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c85c4700 1 -- start start 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c85c4700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c006c710 0x7f19c013b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.402 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c85c4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c00adf00 0x7f19c0159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c85c4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c01446a0 0x7f19c013c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c85c4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c015ab50 con 0x7f19c01446a0 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c85c4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c013c480 con 0x7f19c00adf00 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c85c4700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c013b870 con 0x7f19c006c710 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c6b61700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c01446a0 0x7f19c013c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c6b61700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c01446a0 0x7f19c013c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51474/0 (socket says 172.21.15.97:51474) 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.397+0000 7f19c6b61700 1 -- 172.21.15.97:0/40439159 learned_addr learned my addr 172.21.15.97:0/40439159 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c5b5f700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c00adf00 0x7f19c0159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c6360700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c006c710 0x7f19c013b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c6b61700 1 -- 172.21.15.97:0/40439159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c006c710 msgr2=0x7f19c013b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c6b61700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c006c710 0x7f19c013b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c6b61700 1 -- 172.21.15.97:0/40439159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c00adf00 msgr2=0x7f19c0159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c6b61700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c00adf00 0x7f19c0159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c6b61700 1 -- 172.21.15.97:0/40439159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b800bd40 con 0x7f19c01446a0 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c6b61700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c01446a0 0x7f19c013c910 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f19c00729d0 tx=0x7f19b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19b6ffd700 1 -- 172.21.15.97:0/40439159 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800bbc0 con 0x7f19c01446a0 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19b6ffd700 1 -- 172.21.15.97:0/40439159 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800b4c0 con 0x7f19c01446a0 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c013bb50 con 0x7f19c01446a0 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19b6ffd700 1 -- 172.21.15.97:0/40439159 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800ae70 con 0x7f19c01446a0 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.398+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c013c110 con 0x7f19c01446a0 2023-11-24T06:12:21.404 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.399+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c013c2d0 con 0x7f19c01446a0 2023-11-24T06:12:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.400+0000 7f19b6ffd700 1 -- 172.21.15.97:0/40439159 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f19b8004040 con 0x7f19c01446a0 2023-11-24T06:12:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.400+0000 7f19b6ffd700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19a406b500 0x7f19a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.400+0000 7f19b6ffd700 1 -- 172.21.15.97:0/40439159 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f19b808d800 con 0x7f19c01446a0 2023-11-24T06:12:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.400+0000 7f19c6360700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19a406b500 0x7f19a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.405 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.400+0000 7f19c6360700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19a406b500 0x7f19a406d9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009970 tx=0x7f19b0009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.404+0000 7f19b6ffd700 1 -- 172.21.15.97:0/40439159 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19b8057f00 con 0x7f19c01446a0 2023-11-24T06:12:21.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.597+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19c00753f0 con 0x7f19c01446a0 2023-11-24T06:12:21.601 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.598+0000 7f19b6ffd700 1 -- 172.21.15.97:0/40439159 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f19b805b520 con 0x7f19c01446a0 2023-11-24T06:12:21.601 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:21.601 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:09.577392+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:05.739467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:06.934719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19a406b500 msgr2=0x7f19a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19a406b500 0x7f19a406d9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009970 tx=0x7f19b0009110 comp rx=0 tx=0).stop 2023-11-24T06:12:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c01446a0 msgr2=0x7f19c013c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c01446a0 0x7f19c013c910 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f19c00729d0 tx=0x7f19b8004ad0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 shutdown_connections 2023-11-24T06:12:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19c01446a0 0x7f19c013c910 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f19a406b500 0x7f19a406d9c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f19c006c710 0x7f19c013b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 --2- 172.21.15.97:0/40439159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19c00adf00 0x7f19c0159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 >> 172.21.15.97:0/40439159 conn(0x7f19c0071b90 msgr2=0x7f19c0071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 shutdown_connections 2023-11-24T06:12:21.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.600+0000 7f19c85c4700 1 -- 172.21.15.97:0/40439159 wait complete. 2023-11-24T06:12:21.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:12:05.739467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:12:06.934719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:21.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.736+0000 7f0250ca0700 1 Processor -- start 2023-11-24T06:12:21.739 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.736+0000 7f0250ca0700 1 -- start start 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.737+0000 7f0250ca0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f024c0b93c0 0x7f024c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.737+0000 7f0250ca0700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c0b6fb0 con 0x7f024c15e320 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.737+0000 7f0250ca0700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c0d2c10 con 0x7f024c0b3250 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.737+0000 7f0250ca0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d4da0 con 0x7f024c0b93c0 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.737+0000 7f024a59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f024c15e320 0x7f024c0d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43930/0 (socket says 172.21.15.97:43930) 2023-11-24T06:12:21.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.737+0000 7f024a59c700 1 -- 172.21.15.97:0/2898837830 learned_addr learned my addr 172.21.15.97:0/2898837830 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f0249d9b700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f024c0b93c0 0x7f024c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502350230 0 0) 0x7f024c0b6fb0 con 0x7f024c15e320 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02340035b0 con 0x7f024c15e320 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 83673018 0 0) 0x7f02340035b0 con 0x7f024c15e320 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f024c0b93c0 msgr2=0x7f024c06aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f024c0b93c0 0x7f024c06aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 >> v1:172.21.15.183:6789/0 conn(0x7f024c0b3250 legacy=0x7f024c167d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0d2ef0 con 0x7f024c15e320 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f02440019f0 con 0x7f024c15e320 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 >> v1:172.21.15.97:6789/0 conn(0x7f024c15e320 legacy=0x7f024c0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f024c0b93c0 0x7f0234003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.744 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0234003fb0 0x7f0234004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0234004d90 0x7f0234005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0d4da0 con 0x7f0234003fb0 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023400bb80 con 0x7f0234004d90 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023400bd40 con 0x7f024c0b93c0 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f0249d9b700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0234003fb0 0x7f0234004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024a59c700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f024c0b93c0 0x7f0234003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f024ad9d700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0234004d90 0x7f0234005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f0249d9b700 1 -- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f024c0b93c0 msgr2=0x7f0234003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f0249d9b700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f024c0b93c0 0x7f0234003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f0249d9b700 1 -- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0234004d90 msgr2=0x7f0234005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f0249d9b700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0234004d90 0x7f0234005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.738+0000 7f0249d9b700 1 -- 172.21.15.97:0/2898837830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02340035b0 con 0x7f0234003fb0 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f0249d9b700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0234003fb0 0x7f0234004850 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0240009b30 tx=0x7f0234005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43bc79bf06ff148 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024000e6c0 con 0x7f0234003fb0 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024000e820 con 0x7f0234003fb0 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f024959a700 1 -- 172.21.15.97:0/2898837830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024000eb90 con 0x7f0234003fb0 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f0250ca0700 1 -- 172.21.15.97:0/2898837830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0234003fb0 msgr2=0x7f0234004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f0250ca0700 1 --2- 172.21.15.97:0/2898837830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0234003fb0 0x7f0234004850 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0240009b30 tx=0x7f0234005880 comp rx=0 tx=0).stop 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f024ad9d700 1 -- 172.21.15.97:0/2898837830 reap_dead start 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f0250ca0700 1 -- 172.21.15.97:0/2898837830 shutdown_connections 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f0250ca0700 1 -- 172.21.15.97:0/2898837830 >> 172.21.15.97:0/2898837830 conn(0x7f024c0d3e50 msgr2=0x7f024c0d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f0250ca0700 1 -- 172.21.15.97:0/2898837830 shutdown_connections 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.739+0000 7f0250ca0700 1 -- 172.21.15.97:0/2898837830 wait complete. 2023-11-24T06:12:21.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0250ca0700 1 Processor -- start 2023-11-24T06:12:21.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0250ca0700 1 -- start start 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0250ca0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0234003fb0 0x7f024c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0250ca0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f024c1e7650 0x7f024c1e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0250ca0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c1f6cb0 0x7f024c1f7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0250ca0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c1f7680 con 0x7f024c1e7650 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0250ca0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0caa70 con 0x7f024c1f6cb0 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0250ca0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c1f7b70 con 0x7f0234003fb0 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0249d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f024c1e7650 0x7f024c1e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0249d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f024c1e7650 0x7f024c1e7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51492/0 (socket says 172.21.15.97:51492) 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.741+0000 7f0249d9b700 1 -- 172.21.15.97:0/2709018399 learned_addr learned my addr 172.21.15.97:0/2709018399 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:21.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f024a59c700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0234003fb0 0x7f024c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f024ad9d700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c1f6cb0 0x7f024c1f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f0249d9b700 1 -- 172.21.15.97:0/2709018399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0234003fb0 msgr2=0x7f024c0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f0249d9b700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0234003fb0 0x7f024c0b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f0249d9b700 1 -- 172.21.15.97:0/2709018399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c1f6cb0 msgr2=0x7f024c1f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f0249d9b700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c1f6cb0 0x7f024c1f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f0249d9b700 1 -- 172.21.15.97:0/2709018399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c1f7890 con 0x7f024c1e7650 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f0249d9b700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f024c1e7650 0x7f024c1e7ad0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f02400090b0 tx=0x7f02400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f023affd700 1 -- 172.21.15.97:0/2709018399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024000e6c0 con 0x7f024c1e7650 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f023affd700 1 -- 172.21.15.97:0/2709018399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240003ad0 con 0x7f024c1e7650 2023-11-24T06:12:21.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f023affd700 1 -- 172.21.15.97:0/2709018399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400045d0 con 0x7f024c1e7650 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c158e30 con 0x7f024c1e7650 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.742+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024c1593f0 con 0x7f024c1e7650 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.743+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024c1595b0 con 0x7f024c1e7650 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.743+0000 7f023affd700 1 -- 172.21.15.97:0/2709018399 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f0240013470 con 0x7f024c1e7650 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.744+0000 7f023affd700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f022c06b500 0x7f022c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.744+0000 7f023affd700 1 -- 172.21.15.97:0/2709018399 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f024008b940 con 0x7f024c1e7650 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.744+0000 7f024a59c700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f022c06b500 0x7f022c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.744+0000 7f024a59c700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f022c06b500 0x7f022c06d9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f02440013d0 tx=0x7f0244006040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:21.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.748+0000 7f023affd700 1 -- 172.21.15.97:0/2709018399 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0240056240 con 0x7f024c1e7650 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.937+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f024c068fb0 con 0x7f024c1e7650 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.937+0000 7f023affd700 1 -- 172.21.15.97:0/2709018399 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0240059860 con 0x7f024c1e7650 2023-11-24T06:12:21.940 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:12:21.942 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f022c06b500 msgr2=0x7f022c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f022c06b500 0x7f022c06d9c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f02440013d0 tx=0x7f0244006040 comp rx=0 tx=0).stop 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f024c1e7650 msgr2=0x7f024c1e7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f024c1e7650 0x7f024c1e7ad0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f02400090b0 tx=0x7f02400047c0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 shutdown_connections 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f024c1f6cb0 0x7f024c1f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f022c06b500 0x7f022c06d9c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0234003fb0 0x7f024c0b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 --2- 172.21.15.97:0/2709018399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f024c1e7650 0x7f024c1e7ad0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.939+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 >> 172.21.15.97:0/2709018399 conn(0x7f024c0d3e50 msgr2=0x7f024c15ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.940+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 shutdown_connections 2023-11-24T06:12:21.943 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:21.940+0000 7f0250ca0700 1 -- 172.21.15.97:0/2709018399 wait complete. 2023-11-24T06:12:21.954 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.074+0000 7f88faaf6700 1 Processor -- start 2023-11-24T06:12:22.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.074+0000 7f88faaf6700 1 -- start start 2023-11-24T06:12:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88faaf6700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f88f40746a0 0x7f88f40b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88faaf6700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40b3c60 con 0x7f88f415e160 2023-11-24T06:12:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88faaf6700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40b58c0 con 0x7f88f40afc90 2023-11-24T06:12:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88faaf6700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40b4500 con 0x7f88f40746a0 2023-11-24T06:12:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f8892700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f88f415e160 0x7f88f4073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43950/0 (socket says 172.21.15.97:43950) 2023-11-24T06:12:22.079 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f8892700 1 -- 172.21.15.97:0/3238275707 learned_addr learned my addr 172.21.15.97:0/3238275707 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f3fff700 1 --2- 172.21.15.97:0/3238275707 >> v2:172.21.15.183:3300/0 conn(0x7f88f40746a0 0x7f88f40b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f37fe700 1 -- 172.21.15.97:0/3238275707 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653313193 0 0) 0x7f88f40b3c60 con 0x7f88f415e160 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f37fe700 1 -- 172.21.15.97:0/3238275707 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e40035b0 con 0x7f88f415e160 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f37fe700 1 -- 172.21.15.97:0/3238275707 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570346952 0 0) 0x7f88e40035b0 con 0x7f88f415e160 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f3fff700 1 -- 172.21.15.97:0/3238275707 >> v1:172.21.15.183:6789/0 conn(0x7f88f40afc90 legacy=0x7f88f406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f3fff700 1 -- 172.21.15.97:0/3238275707 >> v1:172.21.15.97:6789/0 conn(0x7f88f415e160 legacy=0x7f88f4073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.075+0000 7f88f3fff700 1 -- 172.21.15.97:0/3238275707 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f406bdc0 con 0x7f88f40746a0 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88f3fff700 1 --2- 172.21.15.97:0/3238275707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88f40746a0 0x7f88f40b5da0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009840 tx=0x7f88e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874756aa1bfa6172 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88f37fe700 1 -- 172.21.15.97:0/3238275707 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8001670 con 0x7f88f40746a0 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88f37fe700 1 -- 172.21.15.97:0/3238275707 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e0004030 con 0x7f88f40746a0 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88f37fe700 1 -- 172.21.15.97:0/3238275707 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00043c0 con 0x7f88f40746a0 2023-11-24T06:12:22.080 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88faaf6700 1 -- 172.21.15.97:0/3238275707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88f40746a0 msgr2=0x7f88f40b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88faaf6700 1 --2- 172.21.15.97:0/3238275707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88f40746a0 0x7f88f40b5da0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f88e0009840 tx=0x7f88e0009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88faaf6700 1 -- 172.21.15.97:0/3238275707 shutdown_connections 2023-11-24T06:12:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88faaf6700 1 --2- 172.21.15.97:0/3238275707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88f40746a0 0x7f88f40b5da0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88faaf6700 1 -- 172.21.15.97:0/3238275707 >> 172.21.15.97:0/3238275707 conn(0x7f88f40d0bf0 msgr2=0x7f88f40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88faaf6700 1 -- 172.21.15.97:0/3238275707 shutdown_connections 2023-11-24T06:12:22.081 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.076+0000 7f88faaf6700 1 -- 172.21.15.97:0/3238275707 wait complete. 2023-11-24T06:12:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.078+0000 7f88faaf6700 1 Processor -- start 2023-11-24T06:12:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.078+0000 7f88faaf6700 1 -- start start 2023-11-24T06:12:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.078+0000 7f88faaf6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f40746a0 0x7f88f40b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.078+0000 7f88faaf6700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88f40afc90 0x7f88f415c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.078+0000 7f88faaf6700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88f415e160 0x7f88f415c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.078+0000 7f88faaf6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4160e30 con 0x7f88f40746a0 2023-11-24T06:12:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.078+0000 7f88faaf6700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4153470 con 0x7f88f415e160 2023-11-24T06:12:22.083 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.078+0000 7f88faaf6700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f41610f0 con 0x7f88f40afc90 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f40746a0 0x7f88f40b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f40746a0 0x7f88f40b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51506/0 (socket says 172.21.15.97:51506) 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 -- 172.21.15.97:0/3457457669 learned_addr learned my addr 172.21.15.97:0/3457457669 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 -- 172.21.15.97:0/3457457669 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88f40afc90 msgr2=0x7f88f415c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88f40afc90 0x7f88f415c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 -- 172.21.15.97:0/3457457669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88f415e160 msgr2=0x7f88f415c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88f415e160 0x7f88f415c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 -- 172.21.15.97:0/3457457669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00094a0 con 0x7f88f40746a0 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f8892700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f40746a0 0x7f88f40b23b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f88e800c520 tx=0x7f88e800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f17fa700 1 -- 172.21.15.97:0/3457457669 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8004c80 con 0x7f88f40746a0 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f17fa700 1 -- 172.21.15.97:0/3457457669 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8004e20 con 0x7f88f40746a0 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f41613d0 con 0x7f88f40746a0 2023-11-24T06:12:22.084 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88f17fa700 1 -- 172.21.15.97:0/3457457669 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8007310 con 0x7f88f40746a0 2023-11-24T06:12:22.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.079+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f41ea490 con 0x7f88f40746a0 2023-11-24T06:12:22.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.080+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f41ea6e0 con 0x7f88f40746a0 2023-11-24T06:12:22.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.080+0000 7f88f17fa700 1 -- 172.21.15.97:0/3457457669 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f88e8007470 con 0x7f88f40746a0 2023-11-24T06:12:22.085 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.081+0000 7f88f17fa700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f88d806b530 0x7f88d806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.081+0000 7f88f17fa700 1 -- 172.21.15.97:0/3457457669 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f88e8089ea0 con 0x7f88f40746a0 2023-11-24T06:12:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.081+0000 7f88f3fff700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f88d806b530 0x7f88d806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.086 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.081+0000 7f88f3fff700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f88d806b530 0x7f88d806d9f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f88f4160bf0 tx=0x7f88e0010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.088 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.085+0000 7f88f17fa700 1 -- 172.21.15.97:0/3457457669 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88e808d030 con 0x7f88f40746a0 2023-11-24T06:12:22.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.274+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88f41e8ba0 con 0x7f88f40746a0 2023-11-24T06:12:22.277 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.275+0000 7f88f17fa700 1 -- 172.21.15.97:0/3457457669 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88e8054680 con 0x7f88f40746a0 2023-11-24T06:12:22.277 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:12:22.279 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f88d806b530 msgr2=0x7f88d806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f88d806b530 0x7f88d806d9f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f88f4160bf0 tx=0x7f88e0010000 comp rx=0 tx=0).stop 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f40746a0 msgr2=0x7f88f40b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f40746a0 0x7f88f40b23b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f88e800c520 tx=0x7f88e800c8e0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 shutdown_connections 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88f415e160 0x7f88f415c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88f40746a0 0x7f88f40b23b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f88d806b530 0x7f88d806d9f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 --2- 172.21.15.97:0/3457457669 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f88f40afc90 0x7f88f415c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 >> 172.21.15.97:0/3457457669 conn(0x7f88f40d0bf0 msgr2=0x7f88f40b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 shutdown_connections 2023-11-24T06:12:22.280 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.277+0000 7f88faaf6700 1 -- 172.21.15.97:0/3457457669 wait complete. 2023-11-24T06:12:22.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:22.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.412+0000 7f3d8f6fb700 1 Processor -- start 2023-11-24T06:12:22.415 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.412+0000 7f3d8f6fb700 1 -- start start 2023-11-24T06:12:22.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8f6fb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88151660 0x7f3d880cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8f6fb700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d8806dec0 0x7f3d88070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8f6fb700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d880b2b00 con 0x7f3d88071080 2023-11-24T06:12:22.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8f6fb700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8806ee40 con 0x7f3d8806dec0 2023-11-24T06:12:22.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8f6fb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880d05b0 con 0x7f3d88151660 2023-11-24T06:12:22.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8cc96700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d8806dec0 0x7f3d88070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.417 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8cc96700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3d8806dec0 0x7f3d88070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51512/0 (socket says 172.21.15.97:51512) 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8cc96700 1 -- 172.21.15.97:0/567644164 learned_addr learned my addr 172.21.15.97:0/567644164 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8d497700 1 --2- 172.21.15.97:0/567644164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88151660 0x7f3d880cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8cc96700 1 -- 172.21.15.97:0/567644164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88151660 msgr2=0x7f3d880cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8cc96700 1 --2- 172.21.15.97:0/567644164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88151660 0x7f3d880cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8cc96700 1 -- 172.21.15.97:0/567644164 >> v1:172.21.15.183:6789/0 conn(0x7f3d88071080 legacy=0x7f3d880cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8cc96700 1 -- 172.21.15.97:0/567644164 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8806e3a0 con 0x7f3d8806dec0 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d8cc96700 1 --2- 172.21.15.97:0/567644164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d88070b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800bbc0 tx=0x7f3d7800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7325a91191ce6a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.413+0000 7f3d7ffff700 1 -- 172.21.15.97:0/567644164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800dd10 con 0x7f3d8806dec0 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d7ffff700 1 -- 172.21.15.97:0/567644164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7800de70 con 0x7f3d8806dec0 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d7ffff700 1 -- 172.21.15.97:0/567644164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78004230 con 0x7f3d8806dec0 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/567644164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 msgr2=0x7f3d88070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.418 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/567644164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d88070b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800bbc0 tx=0x7f3d7800bed0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/567644164 shutdown_connections 2023-11-24T06:12:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/567644164 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88151660 0x7f3d880cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/567644164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d88070b40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/567644164 >> 172.21.15.97:0/567644164 conn(0x7f3d8806b410 msgr2=0x7f3d8806b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/567644164 shutdown_connections 2023-11-24T06:12:22.419 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.414+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/567644164 wait complete. 2023-11-24T06:12:22.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.415+0000 7f3d8f6fb700 1 Processor -- start 2023-11-24T06:12:22.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.415+0000 7f3d8f6fb700 1 -- start start 2023-11-24T06:12:22.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8f6fb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d8815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8f6fb700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88071080 0x7f3d8814ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.421 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8f6fb700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151660 0x7f3d8814f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8f6fb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d88150c60 con 0x7f3d8806dec0 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8f6fb700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8815b360 con 0x7f3d88151660 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8f6fb700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8815b640 con 0x7f3d88071080 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8d497700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d8815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8d497700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d8815acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51516/0 (socket says 172.21.15.97:51516) 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8d497700 1 -- 172.21.15.97:0/4027169776 learned_addr learned my addr 172.21.15.97:0/4027169776 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8cc96700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88071080 0x7f3d8814ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8dc98700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151660 0x7f3d8814f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8d497700 1 -- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88071080 msgr2=0x7f3d8814ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8d497700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88071080 0x7f3d8814ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8d497700 1 -- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151660 msgr2=0x7f3d8814f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.422 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8d497700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151660 0x7f3d8814f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8d497700 1 -- 172.21.15.97:0/4027169776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7800b7f0 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.416+0000 7f3d8cc96700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88071080 0x7f3d8814ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.417+0000 7f3d8d497700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d8815acd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400c7f0 tx=0x7f3d8400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.417+0000 7f3d7dffb700 1 -- 172.21.15.97:0/4027169776 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84011800 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.417+0000 7f3d7dffb700 1 -- 172.21.15.97:0/4027169776 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d84011960 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.417+0000 7f3d7dffb700 1 -- 172.21.15.97:0/4027169776 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84011cf0 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.417+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d881f3ab0 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.417+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d881f3e40 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.418+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d88068480 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.418+0000 7f3d7dffb700 1 -- 172.21.15.97:0/4027169776 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f3d84018360 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.419+0000 7f3d7dffb700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06b500 0x7f3d6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.419+0000 7f3d7dffb700 1 -- 172.21.15.97:0/4027169776 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3d8408a250 con 0x7f3d8806dec0 2023-11-24T06:12:22.423 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.419+0000 7f3d8cc96700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06b500 0x7f3d6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.424 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.419+0000 7f3d8cc96700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06b500 0x7f3d6c06d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3d880cc5e0 tx=0x7f3d78013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.425 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.423+0000 7f3d7dffb700 1 -- 172.21.15.97:0/4027169776 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d840548a0 con 0x7f3d8806dec0 2023-11-24T06:12:22.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.612+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d881f3c40 con 0x7f3d8806dec0 2023-11-24T06:12:22.615 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.613+0000 7f3d7dffb700 1 -- 172.21.15.97:0/4027169776 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d84057ec0 con 0x7f3d8806dec0 2023-11-24T06:12:22.616 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06b500 msgr2=0x7f3d6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06b500 0x7f3d6c06d9c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3d880cc5e0 tx=0x7f3d78013000 comp rx=0 tx=0).stop 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 msgr2=0x7f3d8815acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d8815acd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400c7f0 tx=0x7f3d8400cbb0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 shutdown_connections 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d88151660 0x7f3d8814f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3d8806dec0 0x7f3d8815acd0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3d6c06b500 0x7f3d6c06d9c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 --2- 172.21.15.97:0/4027169776 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3d88071080 0x7f3d8814ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.618 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 >> 172.21.15.97:0/4027169776 conn(0x7f3d8806b410 msgr2=0x7f3d880b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:22.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 shutdown_connections 2023-11-24T06:12:22.619 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.615+0000 7f3d8f6fb700 1 -- 172.21.15.97:0/4027169776 wait complete. 2023-11-24T06:12:22.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:12:22.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.750+0000 7f5562b4b700 1 Processor -- start 2023-11-24T06:12:22.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.750+0000 7f5562b4b700 1 -- start start 2023-11-24T06:12:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f5562b4b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f555c15e4b0 0x7f555c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f5562b4b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c15eb00 con 0x7f555c147780 2023-11-24T06:12:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f5562b4b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c15ede0 con 0x7f555c1473a0 2023-11-24T06:12:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f5562b4b700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c15e910 con 0x7f555c15e4b0 2023-11-24T06:12:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f555c147780 0x7f555c06dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43978/0 (socket says 172.21.15.97:43978) 2023-11-24T06:12:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555bfff700 1 -- 172.21.15.97:0/1721363103 learned_addr learned my addr 172.21.15.97:0/1721363103 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f55610e8700 1 --2- 172.21.15.97:0/1721363103 >> v2:172.21.15.183:3300/0 conn(0x7f555c15e4b0 0x7f555c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049261669 0 0) 0x7f555c15eb00 con 0x7f555c147780 2023-11-24T06:12:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f554c0035b0 con 0x7f555c147780 2023-11-24T06:12:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 573778483 0 0) 0x7f554c0035b0 con 0x7f555c147780 2023-11-24T06:12:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 >> v2:172.21.15.183:3300/0 conn(0x7f555c15e4b0 msgr2=0x7f555c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 --2- 172.21.15.97:0/1721363103 >> v2:172.21.15.183:3300/0 conn(0x7f555c15e4b0 0x7f555c073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 >> v1:172.21.15.183:6789/0 conn(0x7f555c1473a0 legacy=0x7f555c06c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c15f0c0 con 0x7f555c147780 2023-11-24T06:12:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f5548001670 con 0x7f555c147780 2023-11-24T06:12:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 >> v1:172.21.15.97:6789/0 conn(0x7f555c147780 legacy=0x7f555c06dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f554c003a00 0x7f554c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.761 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f554c004850 0x7f554c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c005210 0x7f554c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c00c080 con 0x7f554c004850 2023-11-24T06:12:22.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c00c1f0 con 0x7f554c005210 2023-11-24T06:12:22.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c00c3b0 con 0x7f554c003a00 2023-11-24T06:12:22.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.751+0000 7f555bfff700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f554c004850 0x7f554c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.762 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f55608e7700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f554c003a00 0x7f554c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f55610e8700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c005210 0x7f554c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555bfff700 1 -- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f554c003a00 msgr2=0x7f554c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555bfff700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f554c003a00 0x7f554c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555bfff700 1 -- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c005210 msgr2=0x7f554c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:22.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555bfff700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c005210 0x7f554c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555bfff700 1 -- 172.21.15.97:0/1721363103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0035b0 con 0x7f554c004850 2023-11-24T06:12:22.763 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f55608e7700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f554c003a00 0x7f554c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f55610e8700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c005210 0x7f554c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f55610e8700 1 -- 172.21.15.97:0/1721363103 reap_dead start 2023-11-24T06:12:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555bfff700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f554c004850 0x7f554c004cd0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5548001050 tx=0x7f554c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2859c6809a87cf55 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554800cd50 con 0x7f554c004850 2023-11-24T06:12:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5548005960 con 0x7f554c004850 2023-11-24T06:12:22.764 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f555b7fe700 1 -- 172.21.15.97:0/1721363103 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548005d10 con 0x7f554c004850 2023-11-24T06:12:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f5562b4b700 1 -- 172.21.15.97:0/1721363103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f554c004850 msgr2=0x7f554c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.752+0000 7f5562b4b700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f554c004850 0x7f554c004cd0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5548001050 tx=0x7f554c005880 comp rx=0 tx=0).stop 2023-11-24T06:12:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.753+0000 7f5562b4b700 1 -- 172.21.15.97:0/1721363103 shutdown_connections 2023-11-24T06:12:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.753+0000 7f5562b4b700 1 --2- 172.21.15.97:0/1721363103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f554c004850 0x7f554c004cd0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.753+0000 7f5562b4b700 1 -- 172.21.15.97:0/1721363103 >> 172.21.15.97:0/1721363103 conn(0x7f555c0b3130 msgr2=0x7f555c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.753+0000 7f5562b4b700 1 -- 172.21.15.97:0/1721363103 shutdown_connections 2023-11-24T06:12:22.765 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.753+0000 7f5562b4b700 1 -- 172.21.15.97:0/1721363103 wait complete. 2023-11-24T06:12:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.754+0000 7f5562b4b700 1 Processor -- start 2023-11-24T06:12:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.754+0000 7f5562b4b700 1 -- start start 2023-11-24T06:12:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f5562b4b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c004850 0x7f555c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f5562b4b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c1f79f0 0x7f555c0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f5562b4b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f555c1f6e50 0x7f555c1f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.766 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f5562b4b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c1f38f0 con 0x7f555c1f79f0 2023-11-24T06:12:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f5562b4b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c142d30 con 0x7f554c004850 2023-11-24T06:12:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f5562b4b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c1f7810 con 0x7f555c1f6e50 2023-11-24T06:12:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c1f79f0 0x7f555c0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c1f79f0 0x7f555c0ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51538/0 (socket says 172.21.15.97:51538) 2023-11-24T06:12:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 -- 172.21.15.97:0/3319465395 learned_addr learned my addr 172.21.15.97:0/3319465395 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f55608e7700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c004850 0x7f555c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.767 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f55610e8700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f555c1f6e50 0x7f555c1f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 -- 172.21.15.97:0/3319465395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f555c1f6e50 msgr2=0x7f555c1f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f555c1f6e50 0x7f555c1f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 -- 172.21.15.97:0/3319465395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c004850 msgr2=0x7f555c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c004850 0x7f555c15d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 -- 172.21.15.97:0/3319465395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c142990 con 0x7f555c1f79f0 2023-11-24T06:12:22.768 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f555bfff700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c1f79f0 0x7f555c0ae280 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f554c00c1c0 tx=0x7f5548000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f55597fa700 1 -- 172.21.15.97:0/3319465395 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548007c00 con 0x7f555c1f79f0 2023-11-24T06:12:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f55597fa700 1 -- 172.21.15.97:0/3319465395 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55480070d0 con 0x7f555c1f79f0 2023-11-24T06:12:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f55597fa700 1 -- 172.21.15.97:0/3319465395 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548007480 con 0x7f555c1f79f0 2023-11-24T06:12:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c0aead0 con 0x7f555c1f79f0 2023-11-24T06:12:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.755+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c0af090 con 0x7f555c1f79f0 2023-11-24T06:12:22.769 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.756+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c067f60 con 0x7f555c1f79f0 2023-11-24T06:12:22.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.757+0000 7f55597fa700 1 -- 172.21.15.97:0/3319465395 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f55480075e0 con 0x7f555c1f79f0 2023-11-24T06:12:22.770 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.757+0000 7f55597fa700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f554006b530 0x7f554006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.757+0000 7f55608e7700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f554006b530 0x7f554006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.757+0000 7f55597fa700 1 -- 172.21.15.97:0/3319465395 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f554808afb0 con 0x7f555c1f79f0 2023-11-24T06:12:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.758+0000 7f55608e7700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f554006b530 0x7f554006d9f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f555c0505a0 tx=0x7f5550009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:22.771 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.762+0000 7f55597fa700 1 -- 172.21.15.97:0/3319465395 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5548055790 con 0x7f555c1f79f0 2023-11-24T06:12:22.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.965+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f555c142e70 con 0x7f555c1f79f0 2023-11-24T06:12:22.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.965+0000 7f55597fa700 1 -- 172.21.15.97:0/3319465395 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) v1 ==== 80+0+0 (secure 0 0 0) 0x7f555c142e70 con 0x7f555c1f79f0 2023-11-24T06:12:22.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f554006b530 msgr2=0x7f554006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f554006b530 0x7f554006d9f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f555c0505a0 tx=0x7f5550009040 comp rx=0 tx=0).stop 2023-11-24T06:12:22.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c1f79f0 msgr2=0x7f555c0ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:22.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c1f79f0 0x7f555c0ae280 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f554c00c1c0 tx=0x7f5548000fe0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 shutdown_connections 2023-11-24T06:12:22.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f554006b530 0x7f554006d9f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f555c1f6e50 0x7f555c1f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f554c004850 0x7f555c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 --2- 172.21.15.97:0/3319465395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555c1f79f0 0x7f555c0ae280 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 >> 172.21.15.97:0/3319465395 conn(0x7f555c0b3130 msgr2=0x7f555c146600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 shutdown_connections 2023-11-24T06:12:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:22.967+0000 7f5562b4b700 1 -- 172.21.15.97:0/3319465395 wait complete. 2023-11-24T06:12:22.972 INFO:teuthology.orchestra.run.smithi097.stderr:listed 0 entries 2023-11-24T06:12:22.982 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T06:12:22.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:22.987 DEBUG:teuthology.orchestra.run.smithi183:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T06:12:23.040 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:12:23.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:23.040 DEBUG:teuthology.orchestra.run.smithi183:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:12:23.059 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:12:23.059 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:23.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:12:23.077 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:12:23.077 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:23.118 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:12:23.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.236+0000 7f2eab1e5700 1 Processor -- start 2023-11-24T06:12:23.239 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.236+0000 7f2eab1e5700 1 -- start start 2023-11-24T06:12:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2eab1e5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 0x7f2ea406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2eab1e5700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f2ea406c540 0x7f2ea40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2eab1e5700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea40af950 con 0x7f2ea41446a0 2023-11-24T06:12:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2eab1e5700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea4144a80 con 0x7f2ea406c540 2023-11-24T06:12:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2eab1e5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea406c9b0 con 0x7f2ea40adf00 2023-11-24T06:12:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea8f81700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2ea41446a0 0x7f2ea40b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:43996/0 (socket says 172.21.15.97:43996) 2023-11-24T06:12:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea8f81700 1 -- 172.21.15.97:0/1986336788 learned_addr learned my addr 172.21.15.97:0/1986336788 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:23.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea3fff700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 0x7f2ea406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327606824 0 0) 0x7f2ea40af950 con 0x7f2ea41446a0 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e940035b0 con 0x7f2ea41446a0 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3620154225 0 0) 0x7f2e940035b0 con 0x7f2ea41446a0 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 msgr2=0x7f2ea406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 0x7f2ea406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 >> v2:172.21.15.183:3300/0 conn(0x7f2ea406c540 msgr2=0x7f2ea40b3d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 --2- 172.21.15.97:0/1986336788 >> v2:172.21.15.183:3300/0 conn(0x7f2ea406c540 0x7f2ea40b3d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea40afc30 con 0x7f2ea41446a0 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.237+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f2e98001670 con 0x7f2ea41446a0 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 >> v1:172.21.15.97:6789/0 conn(0x7f2ea41446a0 legacy=0x7f2ea40b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e94003a30 0x7f2e94003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94004850 0x7f2e94004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 0x7f2e94005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea4144a80 con 0x7f2e94004850 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea406c9b0 con 0x7f2e94003a30 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9400dd50 con 0x7f2ea40adf00 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea3fff700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94004850 0x7f2e94004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea9782700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 0x7f2e94005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea3fff700 1 -- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 msgr2=0x7f2e94005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea3fff700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 0x7f2e94005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea3fff700 1 -- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e94003a30 msgr2=0x7f2e94003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea3fff700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e94003a30 0x7f2e94003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea3fff700 1 -- 172.21.15.97:0/1986336788 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e940035b0 con 0x7f2e94004850 2023-11-24T06:12:23.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea9782700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea40adf00 0x7f2e94005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea3fff700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94004850 0x7f2e94004cd0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2ea406eef0 tx=0x7f2e94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e288a5cc685b0b78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000e6c0 con 0x7f2e94004850 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9000e820 con 0x7f2e94004850 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.238+0000 7f2ea37fe700 1 -- 172.21.15.97:0/1986336788 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000eb90 con 0x7f2e94004850 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.239+0000 7f2eab1e5700 1 -- 172.21.15.97:0/1986336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94004850 msgr2=0x7f2e94004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.239+0000 7f2eab1e5700 1 --2- 172.21.15.97:0/1986336788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e94004850 0x7f2e94004cd0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f2ea406eef0 tx=0x7f2e94005880 comp rx=0 tx=0).stop 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.239+0000 7f2ea9782700 1 -- 172.21.15.97:0/1986336788 reap_dead start 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.239+0000 7f2eab1e5700 1 -- 172.21.15.97:0/1986336788 shutdown_connections 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.239+0000 7f2eab1e5700 1 -- 172.21.15.97:0/1986336788 >> 172.21.15.97:0/1986336788 conn(0x7f2ea4071b90 msgr2=0x7f2ea4071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.239+0000 7f2eab1e5700 1 -- 172.21.15.97:0/1986336788 shutdown_connections 2023-11-24T06:12:23.246 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.239+0000 7f2eab1e5700 1 -- 172.21.15.97:0/1986336788 wait complete. 2023-11-24T06:12:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.240+0000 7f2eab1e5700 1 Processor -- start 2023-11-24T06:12:23.247 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.240+0000 7f2eab1e5700 1 -- start start 2023-11-24T06:12:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2eab1e5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e94004850 0x7f2ea41511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2eab1e5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea415a8a0 0x7f2ea415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2eab1e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea4140310 0x7f2ea4140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2eab1e5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea4140cd0 con 0x7f2ea4140310 2023-11-24T06:12:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2eab1e5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea415d420 con 0x7f2e94004850 2023-11-24T06:12:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2eab1e5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea415c150 con 0x7f2ea415a8a0 2023-11-24T06:12:23.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea4140310 0x7f2ea4140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea4140310 0x7f2ea4140790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51560/0 (socket says 172.21.15.97:51560) 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 -- 172.21.15.97:0/897157349 learned_addr learned my addr 172.21.15.97:0/897157349 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea8f81700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e94004850 0x7f2ea41511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea3fff700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea415a8a0 0x7f2ea415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 -- 172.21.15.97:0/897157349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea415a8a0 msgr2=0x7f2ea415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea415a8a0 0x7f2ea415c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 -- 172.21.15.97:0/897157349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e94004850 msgr2=0x7f2ea41511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e94004850 0x7f2ea41511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 -- 172.21.15.97:0/897157349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea415bb40 con 0x7f2ea4140310 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea9782700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea4140310 0x7f2ea4140790 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2ea406d440 tx=0x7f2e9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea17fa700 1 -- 172.21.15.97:0/897157349 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00bc00 con 0x7f2ea4140310 2023-11-24T06:12:23.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.241+0000 7f2ea17fa700 1 -- 172.21.15.97:0/897157349 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c012070 con 0x7f2ea4140310 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.242+0000 7f2ea17fa700 1 -- 172.21.15.97:0/897157349 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00dea0 con 0x7f2ea4140310 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.242+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea415cb20 con 0x7f2ea4140310 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.242+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea415d950 con 0x7f2ea4140310 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.243+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea40753f0 con 0x7f2ea4140310 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.243+0000 7f2ea17fa700 1 -- 172.21.15.97:0/897157349 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2e9c00e420 con 0x7f2ea4140310 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.243+0000 7f2ea17fa700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e8806b500 0x7f2e8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.243+0000 7f2ea17fa700 1 -- 172.21.15.97:0/897157349 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2e9c089bc0 con 0x7f2ea4140310 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.243+0000 7f2ea8f81700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e8806b500 0x7f2e8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.244+0000 7f2ea8f81700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e8806b500 0x7f2e8806d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000ff0 tx=0x7f2e98001ab0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.248+0000 7f2ea17fa700 1 -- 172.21.15.97:0/897157349 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e9c051e90 con 0x7f2ea4140310 2023-11-24T06:12:23.484 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.482+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ea40b3cd0 con 0x7f2ea4140310 2023-11-24T06:12:23.485 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.482+0000 7f2ea17fa700 1 -- 172.21.15.97:0/897157349 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f2e9c014020 con 0x7f2ea4140310 2023-11-24T06:12:23.485 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:23.485 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6903,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3848239046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3848239046},{"type":"v1","addr":"172.21.15.97:6836","nonce":3848239046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3181686967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3181686967},{"type":"v1","addr":"172.21.15.97:6837","nonce":3181686967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/4023096288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4023096288},{"type":"v1","addr":"172.21.15.183:6834","nonce":4023096288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:07.978316+0000","modified":"2023-11-24T06:12:11.003557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6921},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3868240464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3868240464},{"type":"v1","addr":"172.21.15.183:6835","nonce":3868240464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:12:23.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e8806b500 msgr2=0x7f2e8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e8806b500 0x7f2e8806d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000ff0 tx=0x7f2e98001ab0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.488 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea4140310 msgr2=0x7f2ea4140790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea4140310 0x7f2ea4140790 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f2ea406d440 tx=0x7f2e9c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 shutdown_connections 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ea4140310 0x7f2ea4140790 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e94004850 0x7f2ea41511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2e8806b500 0x7f2e8806d9c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 --2- 172.21.15.97:0/897157349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2ea415a8a0 0x7f2ea415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 >> 172.21.15.97:0/897157349 conn(0x7f2ea4071b90 msgr2=0x7f2ea40b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 shutdown_connections 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.484+0000 7f2eab1e5700 1 -- 172.21.15.97:0/897157349 wait complete. 2023-11-24T06:12:23.489 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 56 2023-11-24T06:12:23.499 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:23.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.618+0000 7f5e48c3b700 1 Processor -- start 2023-11-24T06:12:23.621 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.618+0000 7f5e48c3b700 1 -- start start 2023-11-24T06:12:23.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e48c3b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e4415e080 0x7f5e440b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.624 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e48c3b700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e440ae430 con 0x7f5e4406c710 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e48c3b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e440ae710 con 0x7f5e440af4d0 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e48c3b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440af0b0 con 0x7f5e4415e080 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e42d9d700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5e4406c710 0x7f5e4406e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44020/0 (socket says 172.21.15.97:44020) 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e42d9d700 1 -- 172.21.15.97:0/1686695884 learned_addr learned my addr 172.21.15.97:0/1686695884 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e4259c700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e4415e080 0x7f5e440b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098605584 0 0) 0x7f5e440ae430 con 0x7f5e4406c710 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e2c0035b0 con 0x7f5e4406c710 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.619+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608116253 0 0) 0x7f5e440ae710 con 0x7f5e440af4d0 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e440ae430 con 0x7f5e440af4d0 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362107036 0 0) 0x7f5e2c0035b0 con 0x7f5e4406c710 2023-11-24T06:12:23.625 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e4415e080 msgr2=0x7f5e440b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e4415e080 0x7f5e440b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 >> v1:172.21.15.183:6789/0 conn(0x7f5e440af4d0 legacy=0x7f5e4406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440ae9f0 con 0x7f5e4406c710 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f5e3c0019f0 con 0x7f5e4406c710 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 >> v1:172.21.15.97:6789/0 conn(0x7f5e4406c710 legacy=0x7f5e4406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e4415e080 0x7f5e2c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e2c004170 0x7f5e2c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c004d90 0x7f5e2c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440af0b0 con 0x7f5e2c004d90 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c00bb50 con 0x7f5e2c004170 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c00bd10 con 0x7f5e4415e080 2023-11-24T06:12:23.626 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e42d9d700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c004d90 0x7f5e2c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e41d9b700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e2c004170 0x7f5e2c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4259c700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e4415e080 0x7f5e2c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e42d9d700 1 -- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e4415e080 msgr2=0x7f5e2c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e42d9d700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e4415e080 0x7f5e2c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e42d9d700 1 -- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e2c004170 msgr2=0x7f5e2c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e42d9d700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e2c004170 0x7f5e2c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e42d9d700 1 -- 172.21.15.97:0/1686695884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0035b0 con 0x7f5e2c004d90 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e42d9d700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c004d90 0x7f5e2c005210 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00a710 tx=0x7f5e2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f97c6de4ca5c710 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0072d0 con 0x7f5e2c004d90 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c007430 con 0x7f5e2c004d90 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.620+0000 7f5e4159a700 1 -- 172.21.15.97:0/1686695884 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0077e0 con 0x7f5e2c004d90 2023-11-24T06:12:23.627 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.621+0000 7f5e48c3b700 1 -- 172.21.15.97:0/1686695884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c004d90 msgr2=0x7f5e2c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.621+0000 7f5e48c3b700 1 --2- 172.21.15.97:0/1686695884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e2c004d90 0x7f5e2c005210 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00a710 tx=0x7f5e2c005880 comp rx=0 tx=0).stop 2023-11-24T06:12:23.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.621+0000 7f5e42d9d700 1 -- 172.21.15.97:0/1686695884 reap_dead start 2023-11-24T06:12:23.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.621+0000 7f5e48c3b700 1 -- 172.21.15.97:0/1686695884 shutdown_connections 2023-11-24T06:12:23.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.621+0000 7f5e48c3b700 1 -- 172.21.15.97:0/1686695884 >> 172.21.15.97:0/1686695884 conn(0x7f5e44071b90 msgr2=0x7f5e44071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:23.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.621+0000 7f5e48c3b700 1 -- 172.21.15.97:0/1686695884 shutdown_connections 2023-11-24T06:12:23.628 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.621+0000 7f5e48c3b700 1 -- 172.21.15.97:0/1686695884 wait complete. 2023-11-24T06:12:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.622+0000 7f5e48c3b700 1 Processor -- start 2023-11-24T06:12:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.622+0000 7f5e48c3b700 1 -- start start 2023-11-24T06:12:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e48c3b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e2c004d90 0x7f5e441575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e48c3b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44156c40 0x7f5e4414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.630 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e48c3b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e44159b60 0x7f5e44159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e48c3b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e441586d0 con 0x7f5e44156c40 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e48c3b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e44157ec0 con 0x7f5e44159b60 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e48c3b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e44158110 con 0x7f5e2c004d90 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e41d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44156c40 0x7f5e4414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e41d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44156c40 0x7f5e4414bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51578/0 (socket says 172.21.15.97:51578) 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.623+0000 7f5e41d9b700 1 -- 172.21.15.97:0/3356941813 learned_addr learned my addr 172.21.15.97:0/3356941813 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e41d9b700 1 -- 172.21.15.97:0/3356941813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e2c004d90 msgr2=0x7f5e441575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e41d9b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e2c004d90 0x7f5e441575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e41d9b700 1 -- 172.21.15.97:0/3356941813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e44159b60 msgr2=0x7f5e44159fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e41d9b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e44159b60 0x7f5e44159fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e41d9b700 1 -- 172.21.15.97:0/3356941813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e441583f0 con 0x7f5e44156c40 2023-11-24T06:12:23.631 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e41d9b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44156c40 0x7f5e4414bc50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400f810 tx=0x7f5e3400fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e32ffd700 1 -- 172.21.15.97:0/3356941813 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340052e0 con 0x7f5e44156c40 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e32ffd700 1 -- 172.21.15.97:0/3356941813 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e34005460 con 0x7f5e44156c40 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4415f340 con 0x7f5e44156c40 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e32ffd700 1 -- 172.21.15.97:0/3356941813 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340038c0 con 0x7f5e44156c40 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.624+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4414c190 con 0x7f5e44156c40 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.625+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e440753f0 con 0x7f5e44156c40 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.625+0000 7f5e32ffd700 1 -- 172.21.15.97:0/3356941813 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f5e34003a20 con 0x7f5e44156c40 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.626+0000 7f5e32ffd700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e2406b500 0x7f5e2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.626+0000 7f5e32ffd700 1 -- 172.21.15.97:0/3356941813 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5e34089fe0 con 0x7f5e44156c40 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.626+0000 7f5e4259c700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e2406b500 0x7f5e2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.632 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.626+0000 7f5e4259c700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e2406b500 0x7f5e2406d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009b50 tx=0x7f5e38009670 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.633 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.630+0000 7f5e32ffd700 1 -- 172.21.15.97:0/3356941813 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e340547c0 con 0x7f5e44156c40 2023-11-24T06:12:23.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.824+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e4415f1a0 con 0x7f5e44156c40 2023-11-24T06:12:23.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.825+0000 7f5e32ffd700 1 -- 172.21.15.97:0/3356941813 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f5e4415f1a0 con 0x7f5e44156c40 2023-11-24T06:12:23.828 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:23.829 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:09.577392+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:05.739467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:06.934719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:23.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e2406b500 msgr2=0x7f5e2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e2406b500 0x7f5e2406d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009b50 tx=0x7f5e38009670 comp rx=0 tx=0).stop 2023-11-24T06:12:23.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44156c40 msgr2=0x7f5e4414bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44156c40 0x7f5e4414bc50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400f810 tx=0x7f5e3400fbd0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 shutdown_connections 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e44159b60 0x7f5e44159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5e2406b500 0x7f5e2406d9c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5e2c004d90 0x7f5e441575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 --2- 172.21.15.97:0/3356941813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44156c40 0x7f5e4414bc50 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 >> 172.21.15.97:0/3356941813 conn(0x7f5e44071b90 msgr2=0x7f5e440b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 shutdown_connections 2023-11-24T06:12:23.831 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.827+0000 7f5e48c3b700 1 -- 172.21.15.97:0/3356941813 wait complete. 2023-11-24T06:12:23.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:08:44.352292+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:12:05.739467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:12:06.934719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:12:23.843 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:12:23.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.963+0000 7f7a2594b700 1 Processor -- start 2023-11-24T06:12:23.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.963+0000 7f7a2594b700 1 -- start start 2023-11-24T06:12:23.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a2594b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a200b24a0 0x7f7a200b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a2594b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200b2ab0 0x7f7a200cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a2594b700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f7a200d0060 0x7f7a2006b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a2594b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a200d0c40 con 0x7f7a200b24a0 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a2594b700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a20161d20 con 0x7f7a200d0060 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a2594b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200b2880 con 0x7f7a200b2ab0 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a200b24a0 0x7f7a200b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a200b24a0 0x7f7a200b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51584/0 (socket says 172.21.15.97:51584) 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 -- 172.21.15.97:0/3373865274 learned_addr learned my addr 172.21.15.97:0/3373865274 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1e7fc700 1 --2- 172.21.15.97:0/3373865274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200b2ab0 0x7f7a200cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1f7fe700 1 --2- 172.21.15.97:0/3373865274 >> v2:172.21.15.183:3300/0 conn(0x7f7a200d0060 0x7f7a2006b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 -- 172.21.15.97:0/3373865274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200b2ab0 msgr2=0x7f7a200cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.969 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 --2- 172.21.15.97:0/3373865274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200b2ab0 0x7f7a200cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 -- 172.21.15.97:0/3373865274 >> v2:172.21.15.183:3300/0 conn(0x7f7a200d0060 msgr2=0x7f7a2006b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 --2- 172.21.15.97:0/3373865274 >> v2:172.21.15.183:3300/0 conn(0x7f7a200d0060 0x7f7a2006b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 -- 172.21.15.97:0/3373865274 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a200b31a0 con 0x7f7a200b24a0 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1e7fc700 1 --2- 172.21.15.97:0/3373865274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200b2ab0 0x7f7a200cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1f7fe700 1 --2- 172.21.15.97:0/3373865274 >> v2:172.21.15.183:3300/0 conn(0x7f7a200d0060 0x7f7a2006b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1effd700 1 --2- 172.21.15.97:0/3373865274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a200b4330 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7a20072c60 tx=0x7f7a14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50dbb4d1cd3f649 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.964+0000 7f7a1dffb700 1 -- 172.21.15.97:0/3373865274 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1400b860 con 0x7f7a200b24a0 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a1dffb700 1 -- 172.21.15.97:0/3373865274 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1400b9c0 con 0x7f7a200b24a0 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a1dffb700 1 -- 172.21.15.97:0/3373865274 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1400bd50 con 0x7f7a200b24a0 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 -- 172.21.15.97:0/3373865274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 msgr2=0x7f7a200b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 --2- 172.21.15.97:0/3373865274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a200b4330 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f7a20072c60 tx=0x7f7a14009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:23.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 -- 172.21.15.97:0/3373865274 shutdown_connections 2023-11-24T06:12:23.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 --2- 172.21.15.97:0/3373865274 >> v2:172.21.15.183:3300/0 conn(0x7f7a200d0060 0x7f7a2006b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 --2- 172.21.15.97:0/3373865274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a200b4330 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 --2- 172.21.15.97:0/3373865274 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200b2ab0 0x7f7a200cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 -- 172.21.15.97:0/3373865274 >> 172.21.15.97:0/3373865274 conn(0x7f7a200b3810 msgr2=0x7f7a200b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:23.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 -- 172.21.15.97:0/3373865274 shutdown_connections 2023-11-24T06:12:23.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.965+0000 7f7a2594b700 1 -- 172.21.15.97:0/3373865274 wait complete. 2023-11-24T06:12:23.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.966+0000 7f7a2594b700 1 Processor -- start 2023-11-24T06:12:23.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.966+0000 7f7a2594b700 1 -- start start 2023-11-24T06:12:23.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a2594b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a2006ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a2594b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a200b2ab0 0x7f7a2006f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a2594b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200d0060 0x7f7a200b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a2594b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200c94e0 con 0x7f7a200b24a0 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a2594b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200c5ca0 con 0x7f7a200b2ab0 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a2594b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200c5f80 con 0x7f7a200d0060 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a1effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a2006ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a1effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a2006ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51590/0 (socket says 172.21.15.97:51590) 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a1effd700 1 -- 172.21.15.97:0/197022010 learned_addr learned my addr 172.21.15.97:0/197022010 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a1e7fc700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a200b2ab0 0x7f7a2006f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.967+0000 7f7a1f7fe700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200d0060 0x7f7a200b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a1effd700 1 -- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200d0060 msgr2=0x7f7a200b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a1effd700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200d0060 0x7f7a200b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a1effd700 1 -- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a200b2ab0 msgr2=0x7f7a2006f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a1effd700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a200b2ab0 0x7f7a2006f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a1effd700 1 -- 172.21.15.97:0/197022010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a14009550 con 0x7f7a200b24a0 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a1f7fe700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200d0060 0x7f7a200b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a1e7fc700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a200b2ab0 0x7f7a2006f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a1effd700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a2006ed70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7a14014040 tx=0x7f7a14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a07fff700 1 -- 172.21.15.97:0/197022010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a140036a0 con 0x7f7a200b24a0 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a07fff700 1 -- 172.21.15.97:0/197022010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1401e070 con 0x7f7a200b24a0 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a200c6260 con 0x7f7a200b24a0 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a07fff700 1 -- 172.21.15.97:0/197022010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a14003a40 con 0x7f7a200b24a0 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.968+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2015e410 con 0x7f7a200b24a0 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.969+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a200c9620 con 0x7f7a200b24a0 2023-11-24T06:12:23.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.970+0000 7f7a07fff700 1 -- 172.21.15.97:0/197022010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7a14003ba0 con 0x7f7a200b24a0 2023-11-24T06:12:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.970+0000 7f7a07fff700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7a0c06b500 0x7f7a0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.970+0000 7f7a07fff700 1 -- 172.21.15.97:0/197022010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7a14020070 con 0x7f7a200b24a0 2023-11-24T06:12:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.970+0000 7f7a1e7fc700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7a0c06b500 0x7f7a0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.970+0000 7f7a1e7fc700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7a0c06b500 0x7f7a0c06d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7a200d26e0 tx=0x7f7a10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:23.977 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:23.974+0000 7f7a07fff700 1 -- 172.21.15.97:0/197022010 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a14055e10 con 0x7f7a200b24a0 2023-11-24T06:12:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.162+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a20068fb0 con 0x7f7a200b24a0 2023-11-24T06:12:24.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.163+0000 7f7a07fff700 1 -- 172.21.15.97:0/197022010 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a14059430 con 0x7f7a200b24a0 2023-11-24T06:12:24.166 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2023-11-24T06:12:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7a0c06b500 msgr2=0x7f7a0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7a0c06b500 0x7f7a0c06d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7a200d26e0 tx=0x7f7a10008040 comp rx=0 tx=0).stop 2023-11-24T06:12:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 msgr2=0x7f7a2006ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a2006ed70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7a14014040 tx=0x7f7a14004750 comp rx=0 tx=0).stop 2023-11-24T06:12:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 shutdown_connections 2023-11-24T06:12:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f7a0c06b500 0x7f7a0c06d9c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7a200d0060 0x7f7a200b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a200b24a0 0x7f7a2006ed70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 --2- 172.21.15.97:0/197022010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a200b2ab0 0x7f7a2006f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 >> 172.21.15.97:0/197022010 conn(0x7f7a200b3810 msgr2=0x7f7a200cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 shutdown_connections 2023-11-24T06:12:24.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.165+0000 7f7a2594b700 1 -- 172.21.15.97:0/197022010 wait complete. 2023-11-24T06:12:24.180 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:12:24.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.300+0000 7ff5c5c86700 1 Processor -- start 2023-11-24T06:12:24.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.300+0000 7ff5c5c86700 1 -- start start 2023-11-24T06:12:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5c5c86700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7ff5c015e160 0x7ff5c00b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5c5c86700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5c00746a0 0x7ff5c006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5c5c86700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c00b3c60 con 0x7ff5c00afc90 2023-11-24T06:12:24.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5c5c86700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00b4500 con 0x7ff5c00746a0 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5c5c86700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c00b58c0 con 0x7ff5c015e160 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5c00746a0 0x7ff5c006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bffff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff5c00746a0 0x7ff5c006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51602/0 (socket says 172.21.15.97:51602) 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5beffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff5c00afc90 0x7ff5c0073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44050/0 (socket says 172.21.15.97:44050) 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bffff700 1 -- 172.21.15.97:0/3160453179 learned_addr learned my addr 172.21.15.97:0/3160453179 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bf7fe700 1 --2- 172.21.15.97:0/3160453179 >> v2:172.21.15.183:3300/0 conn(0x7ff5c015e160 0x7ff5c00b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bffff700 1 -- 172.21.15.97:0/3160453179 >> v2:172.21.15.183:3300/0 conn(0x7ff5c015e160 msgr2=0x7ff5c00b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bffff700 1 --2- 172.21.15.97:0/3160453179 >> v2:172.21.15.183:3300/0 conn(0x7ff5c015e160 0x7ff5c00b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bffff700 1 -- 172.21.15.97:0/3160453179 >> v1:172.21.15.97:6789/0 conn(0x7ff5c00afc90 legacy=0x7ff5c0073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bffff700 1 -- 172.21.15.97:0/3160453179 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00b0140 con 0x7ff5c00746a0 2023-11-24T06:12:24.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5be7fc700 1 -- 172.21.15.97:0/3160453179 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485786975 0 0) 0x7ff5c00b3c60 con 0x7ff5c00afc90 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.301+0000 7ff5bffff700 1 --2- 172.21.15.97:0/3160453179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c006d930 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff5c009efa0 tx=0x7ff5b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0c751bc58aa153 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5be7fc700 1 -- 172.21.15.97:0/3160453179 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800e070 con 0x7ff5c00746a0 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5be7fc700 1 -- 172.21.15.97:0/3160453179 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b800b910 con 0x7ff5c00746a0 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5be7fc700 1 -- 172.21.15.97:0/3160453179 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b800bcc0 con 0x7ff5c00746a0 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5c5c86700 1 -- 172.21.15.97:0/3160453179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 msgr2=0x7ff5c006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/3160453179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c006d930 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff5c009efa0 tx=0x7ff5b800b330 comp rx=0 tx=0).stop 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5c5c86700 1 -- 172.21.15.97:0/3160453179 shutdown_connections 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/3160453179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c006d930 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/3160453179 >> v2:172.21.15.183:3300/0 conn(0x7ff5c015e160 0x7ff5c00b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5c5c86700 1 -- 172.21.15.97:0/3160453179 >> 172.21.15.97:0/3160453179 conn(0x7ff5c00d0bf0 msgr2=0x7ff5c00d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5c5c86700 1 -- 172.21.15.97:0/3160453179 shutdown_connections 2023-11-24T06:12:24.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.302+0000 7ff5c5c86700 1 -- 172.21.15.97:0/3160453179 wait complete. 2023-11-24T06:12:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.303+0000 7ff5c5c86700 1 Processor -- start 2023-11-24T06:12:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5c5c86700 1 -- start start 2023-11-24T06:12:24.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5c5c86700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c01f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5c5c86700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00afc90 0x7ff5c01f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5c5c86700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c015e160 0x7ff5c01f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5c5c86700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c014b210 con 0x7ff5c00746a0 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5c5c86700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01f3990 con 0x7ff5c015e160 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5c5c86700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00c75f0 con 0x7ff5c00afc90 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5bf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c01f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5bf7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c01f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51604/0 (socket says 172.21.15.97:51604) 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.304+0000 7ff5bf7fe700 1 -- 172.21.15.97:0/1193505796 learned_addr learned my addr 172.21.15.97:0/1193505796 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5bf7fe700 1 -- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00afc90 msgr2=0x7ff5c01f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5beffd700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00afc90 0x7ff5c01f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5bffff700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c015e160 0x7ff5c01f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5bf7fe700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00afc90 0x7ff5c01f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5bf7fe700 1 -- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c015e160 msgr2=0x7ff5c01f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5bf7fe700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c015e160 0x7ff5c01f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5bf7fe700 1 -- 172.21.15.97:0/1193505796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8009b30 con 0x7ff5c00746a0 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5beffd700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00afc90 0x7ff5c01f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5bf7fe700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c01f3450 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400e870 tx=0x7ff5b400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5a7fff700 1 -- 172.21.15.97:0/1193505796 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4009590 con 0x7ff5c00746a0 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5a7fff700 1 -- 172.21.15.97:0/1193505796 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400ee70 con 0x7ff5c00746a0 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c014b540 con 0x7ff5c00746a0 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5a7fff700 1 -- 172.21.15.97:0/1193505796 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400ca50 con 0x7ff5c00746a0 2023-11-24T06:12:24.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.305+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c014bb00 con 0x7ff5c00746a0 2023-11-24T06:12:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.306+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c014bcc0 con 0x7ff5c00746a0 2023-11-24T06:12:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.306+0000 7ff5a7fff700 1 -- 172.21.15.97:0/1193505796 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff5b400cbb0 con 0x7ff5c00746a0 2023-11-24T06:12:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.307+0000 7ff5a7fff700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff5ac06b500 0x7ff5ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.307+0000 7ff5a7fff700 1 -- 172.21.15.97:0/1193505796 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff5b408a0c0 con 0x7ff5c00746a0 2023-11-24T06:12:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.307+0000 7ff5beffd700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff5ac06b500 0x7ff5ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.307+0000 7ff5beffd700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff5ac06b500 0x7ff5ac06d9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff5c015dec0 tx=0x7ff5b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.311+0000 7ff5a7fff700 1 -- 172.21.15.97:0/1193505796 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5b40bce90 con 0x7ff5c00746a0 2023-11-24T06:12:24.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.506+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5c0068fb0 con 0x7ff5c00746a0 2023-11-24T06:12:24.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.506+0000 7ff5a7fff700 1 -- 172.21.15.97:0/1193505796 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5b4054810 con 0x7ff5c00746a0 2023-11-24T06:12:24.509 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2023-11-24T06:12:24.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff5ac06b500 msgr2=0x7ff5ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff5ac06b500 0x7ff5ac06d9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff5c015dec0 tx=0x7ff5b0009040 comp rx=0 tx=0).stop 2023-11-24T06:12:24.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 msgr2=0x7ff5c01f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c01f3450 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400e870 tx=0x7ff5b400ec30 comp rx=0 tx=0).stop 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 shutdown_connections 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5c015e160 0x7ff5c01f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c00746a0 0x7ff5c01f3450 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff5ac06b500 0x7ff5ac06d9c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 --2- 172.21.15.97:0/1193505796 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff5c00afc90 0x7ff5c01f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.508+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 >> 172.21.15.97:0/1193505796 conn(0x7ff5c00d0bf0 msgr2=0x7ff5c006e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.509+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 shutdown_connections 2023-11-24T06:12:24.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.509+0000 7ff5c5c86700 1 -- 172.21.15.97:0/1193505796 wait complete. 2023-11-24T06:12:24.518 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:12:24.520 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:12:24.523 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:12:24.689 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.687+0000 7ff7eb289700 1 Processor -- start 2023-11-24T06:12:24.690 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.687+0000 7ff7eb289700 1 -- start start 2023-11-24T06:12:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7eb289700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40b4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7eb289700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7e406c000 0x7ff7e406c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.691 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7eb289700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e40b4300 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7eb289700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e406c9b0 con 0x7ff7e406c000 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7eb289700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e4144b40 con 0x7ff7e4144760 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9826700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7e406c000 0x7ff7e406c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9826700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7e406c000 0x7ff7e406c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51616/0 (socket says 172.21.15.97:51616) 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9826700 1 -- 172.21.15.97:0/356560693 learned_addr learned my addr 172.21.15.97:0/356560693 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9025700 1 --2- 172.21.15.97:0/356560693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9826700 1 -- 172.21.15.97:0/356560693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 msgr2=0x7ff7e40b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9826700 1 --2- 172.21.15.97:0/356560693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40b4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9826700 1 -- 172.21.15.97:0/356560693 >> v1:172.21.15.183:6789/0 conn(0x7ff7e40b4cd0 legacy=0x7ff7e406db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9826700 1 -- 172.21.15.97:0/356560693 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e415ec80 con 0x7ff7e406c000 2023-11-24T06:12:24.692 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9025700 1 --2- 172.21.15.97:0/356560693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.688+0000 7ff7e9826700 1 --2- 172.21.15.97:0/356560693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e406c000 0x7ff7e406c470 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009e50 tx=0x7ff7dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d804612caf018c14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.690+0000 7ff7dbfff700 1 -- 172.21.15.97:0/356560693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00e070 con 0x7ff7e406c000 2023-11-24T06:12:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7dbfff700 1 -- 172.21.15.97:0/356560693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc00b9c0 con 0x7ff7e406c000 2023-11-24T06:12:24.693 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7dbfff700 1 -- 172.21.15.97:0/356560693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00bd70 con 0x7ff7e406c000 2023-11-24T06:12:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7eb289700 1 -- 172.21.15.97:0/356560693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e406c000 msgr2=0x7ff7e406c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7eb289700 1 --2- 172.21.15.97:0/356560693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e406c000 0x7ff7e406c470 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009e50 tx=0x7ff7dc00b330 comp rx=0 tx=0).stop 2023-11-24T06:12:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7eb289700 1 -- 172.21.15.97:0/356560693 shutdown_connections 2023-11-24T06:12:24.694 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7eb289700 1 --2- 172.21.15.97:0/356560693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e406c000 0x7ff7e406c470 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7eb289700 1 --2- 172.21.15.97:0/356560693 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7eb289700 1 -- 172.21.15.97:0/356560693 >> 172.21.15.97:0/356560693 conn(0x7ff7e4071a70 msgr2=0x7ff7e4071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7eb289700 1 -- 172.21.15.97:0/356560693 shutdown_connections 2023-11-24T06:12:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.691+0000 7ff7eb289700 1 -- 172.21.15.97:0/356560693 wait complete. 2023-11-24T06:12:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.692+0000 7ff7eb289700 1 Processor -- start 2023-11-24T06:12:24.695 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7eb289700 1 -- start start 2023-11-24T06:12:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7eb289700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7e406c000 0x7ff7e413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7eb289700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40b4cd0 0x7ff7e40c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7eb289700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.697 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7eb289700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415e3c0 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7eb289700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415cd70 con 0x7ff7e406c000 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7eb289700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415bfb0 con 0x7ff7e4144760 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7e8824700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40b4cd0 0x7ff7e40c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7e8824700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40b4cd0 0x7ff7e40c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51620/0 (socket says 172.21.15.97:51620) 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.693+0000 7ff7e8824700 1 -- 172.21.15.97:0/1114412967 learned_addr learned my addr 172.21.15.97:0/1114412967 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7e9826700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7e9025700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7e406c000 0x7ff7e413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7e8824700 1 -- 172.21.15.97:0/1114412967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 msgr2=0x7ff7e40c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7e8824700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7e8824700 1 -- 172.21.15.97:0/1114412967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7e406c000 msgr2=0x7ff7e413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.698 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7e8824700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7e406c000 0x7ff7e413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7e8824700 1 -- 172.21.15.97:0/1114412967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc009b30 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7e8824700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40b4cd0 0x7ff7e40c60b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400c4e0 tx=0x7ff7d400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7d9ffb700 1 -- 172.21.15.97:0/1114412967 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40052d0 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7d9ffb700 1 -- 172.21.15.97:0/1114412967 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d4005430 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e415c290 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7d9ffb700 1 -- 172.21.15.97:0/1114412967 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40038c0 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.699 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.694+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e415c850 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.695+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e4068fb0 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.695+0000 7ff7d9ffb700 1 -- 172.21.15.97:0/1114412967 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff7d4003a20 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.696+0000 7ff7d9ffb700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b500 0x7ff7c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.696+0000 7ff7d9ffb700 1 -- 172.21.15.97:0/1114412967 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff7d4089440 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.696+0000 7ff7e9025700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b500 0x7ff7c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:24.700 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.696+0000 7ff7e9025700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b500 0x7ff7c806d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff7e41fb9f0 tx=0x7ff7e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:24.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.700+0000 7ff7d9ffb700 1 -- 172.21.15.97:0/1114412967 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff7d4053c20 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.889+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7e415ca70 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.892 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.889+0000 7ff7d9ffb700 1 -- 172.21.15.97:0/1114412967 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7d4057240 con 0x7ff7e40b4cd0 2023-11-24T06:12:24.892 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b500 msgr2=0x7ff7c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b500 0x7ff7c806d9c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff7e41fb9f0 tx=0x7ff7e0008040 comp rx=0 tx=0).stop 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40b4cd0 msgr2=0x7ff7e40c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40b4cd0 0x7ff7e40c60b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff7d400c4e0 tx=0x7ff7d400c7f0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 shutdown_connections 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b500 0x7ff7c806d9c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7e4144760 0x7ff7e40c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7e406c000 0x7ff7e413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 --2- 172.21.15.97:0/1114412967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7e40b4cd0 0x7ff7e40c60b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 >> 172.21.15.97:0/1114412967 conn(0x7ff7e4071a70 msgr2=0x7ff7e40b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:24.895 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 shutdown_connections 2023-11-24T06:12:24.896 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:24.892+0000 7ff7eb289700 1 -- 172.21.15.97:0/1114412967 wait complete. 2023-11-24T06:12:24.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:25.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.024+0000 7f236eca7700 1 Processor -- start 2023-11-24T06:12:25.026 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.024+0000 7f236eca7700 1 -- start start 2023-11-24T06:12:25.028 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.024+0000 7f236eca7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f236815e080 0x7f23680b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.024+0000 7f236eca7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f236806c710 0x7f23680afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.024+0000 7f236eca7700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23680ae430 con 0x7f23680af4d0 2023-11-24T06:12:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.024+0000 7f236eca7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23680af0b0 con 0x7f236815e080 2023-11-24T06:12:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.024+0000 7f236eca7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23680b05a0 con 0x7f236806c710 2023-11-24T06:12:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236ca43700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f236815e080 0x7f23680b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.029 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236ca43700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f236815e080 0x7f23680b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51626/0 (socket says 172.21.15.97:51626) 2023-11-24T06:12:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236ca43700 1 -- 172.21.15.97:0/3770539960 learned_addr learned my addr 172.21.15.97:0/3770539960 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236d244700 1 --2- 172.21.15.97:0/3770539960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f236806c710 0x7f23680afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236ca43700 1 -- 172.21.15.97:0/3770539960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f236806c710 msgr2=0x7f23680afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:25.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236ca43700 1 --2- 172.21.15.97:0/3770539960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f236806c710 0x7f23680afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236ca43700 1 -- 172.21.15.97:0/3770539960 >> v1:172.21.15.183:6789/0 conn(0x7f23680af4d0 legacy=0x7f236806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:25.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236ca43700 1 -- 172.21.15.97:0/3770539960 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23680ae710 con 0x7f236815e080 2023-11-24T06:12:25.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236d244700 1 --2- 172.21.15.97:0/3770539960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f236806c710 0x7f23680afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:25.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f236ca43700 1 --2- 172.21.15.97:0/3770539960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23680b5050 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f235c009920 tx=0x7f235c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc0f2d2e20bedf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f23677fe700 1 -- 172.21.15.97:0/3770539960 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c00ba20 con 0x7f236815e080 2023-11-24T06:12:25.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f23677fe700 1 -- 172.21.15.97:0/3770539960 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00bb80 con 0x7f236815e080 2023-11-24T06:12:25.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.025+0000 7f23677fe700 1 -- 172.21.15.97:0/3770539960 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c009e70 con 0x7f236815e080 2023-11-24T06:12:25.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.026+0000 7f236eca7700 1 -- 172.21.15.97:0/3770539960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 msgr2=0x7f23680b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.026+0000 7f236eca7700 1 --2- 172.21.15.97:0/3770539960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23680b5050 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f235c009920 tx=0x7f235c009c30 comp rx=0 tx=0).stop 2023-11-24T06:12:25.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.026+0000 7f236eca7700 1 -- 172.21.15.97:0/3770539960 shutdown_connections 2023-11-24T06:12:25.033 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.026+0000 7f236eca7700 1 --2- 172.21.15.97:0/3770539960 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f236806c710 0x7f23680afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.026+0000 7f236eca7700 1 --2- 172.21.15.97:0/3770539960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23680b5050 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.026+0000 7f236eca7700 1 -- 172.21.15.97:0/3770539960 >> 172.21.15.97:0/3770539960 conn(0x7f2368071b90 msgr2=0x7f2368071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:25.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.026+0000 7f236eca7700 1 -- 172.21.15.97:0/3770539960 shutdown_connections 2023-11-24T06:12:25.034 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.026+0000 7f236eca7700 1 -- 172.21.15.97:0/3770539960 wait complete. 2023-11-24T06:12:25.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.027+0000 7f236eca7700 1 Processor -- start 2023-11-24T06:12:25.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.027+0000 7f236eca7700 1 -- start start 2023-11-24T06:12:25.036 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236eca7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f236806c710 0x7f2368153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236eca7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23680af4d0 0x7f2368158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236eca7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23681572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236eca7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368159230 con 0x7f236815e080 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236eca7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368157c50 con 0x7f236806c710 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236eca7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368157f30 con 0x7f23680af4d0 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23681572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23681572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51636/0 (socket says 172.21.15.97:51636) 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 -- 172.21.15.97:0/553586574 learned_addr learned my addr 172.21.15.97:0/553586574 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236ca43700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f236806c710 0x7f2368153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.037 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f2367fff700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23680af4d0 0x7f2368158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 -- 172.21.15.97:0/553586574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23680af4d0 msgr2=0x7f2368158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23680af4d0 0x7f2368158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 -- 172.21.15.97:0/553586574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f236806c710 msgr2=0x7f2368153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f236806c710 0x7f2368153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 -- 172.21.15.97:0/553586574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c009550 con 0x7f236815e080 2023-11-24T06:12:25.038 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f236d244700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23681572e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f236000cea0 tx=0x7f236000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.028+0000 7f23657fa700 1 -- 172.21.15.97:0/553586574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000ad40 con 0x7f236815e080 2023-11-24T06:12:25.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.029+0000 7f23657fa700 1 -- 172.21.15.97:0/553586574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360003a20 con 0x7f236815e080 2023-11-24T06:12:25.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.029+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2368158210 con 0x7f236815e080 2023-11-24T06:12:25.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.029+0000 7f23657fa700 1 -- 172.21.15.97:0/553586574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360003dd0 con 0x7f236815e080 2023-11-24T06:12:25.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.029+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23681587d0 con 0x7f236815e080 2023-11-24T06:12:25.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.030+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2368158990 con 0x7f236815e080 2023-11-24T06:12:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.030+0000 7f23657fa700 1 -- 172.21.15.97:0/553586574 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2360003f30 con 0x7f236815e080 2023-11-24T06:12:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.030+0000 7f23657fa700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.030+0000 7f23657fa700 1 -- 172.21.15.97:0/553586574 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f236008a440 con 0x7f236815e080 2023-11-24T06:12:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.031+0000 7f236ca43700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.040 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.031+0000 7f236ca43700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f234c06b500 0x7f234c06d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f235c000c00 tx=0x7f235c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.035+0000 7f23657fa700 1 -- 172.21.15.97:0/553586574 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2360054b90 con 0x7f236815e080 2023-11-24T06:12:25.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.265+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23680753f0 con 0x7f236815e080 2023-11-24T06:12:25.268 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.266+0000 7f23657fa700 1 -- 172.21.15.97:0/553586574 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f23600581b0 con 0x7f236815e080 2023-11-24T06:12:25.269 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:25.269 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6903,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3848239046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3848239046},{"type":"v1","addr":"172.21.15.97:6836","nonce":3848239046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3181686967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3181686967},{"type":"v1","addr":"172.21.15.97:6837","nonce":3181686967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/4023096288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4023096288},{"type":"v1","addr":"172.21.15.183:6834","nonce":4023096288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:07.978316+0000","modified":"2023-11-24T06:12:11.003557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6921},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3868240464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3868240464},{"type":"v1","addr":"172.21.15.183:6835","nonce":3868240464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:12:25.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f234c06b500 msgr2=0x7f234c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f234c06b500 0x7f234c06d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f235c000c00 tx=0x7f235c008030 comp rx=0 tx=0).stop 2023-11-24T06:12:25.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 msgr2=0x7f23681572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23681572e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f236000cea0 tx=0x7f236000ced0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.271 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 shutdown_connections 2023-11-24T06:12:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f236815e080 0x7f23681572e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f236806c710 0x7f2368153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.268+0000 7f236eca7700 1 --2- 172.21.15.97:0/553586574 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f23680af4d0 0x7f2368158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.269+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 >> 172.21.15.97:0/553586574 conn(0x7f2368071b90 msgr2=0x7f23680b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.269+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 shutdown_connections 2023-11-24T06:12:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.269+0000 7f236eca7700 1 -- 172.21.15.97:0/553586574 wait complete. 2023-11-24T06:12:25.272 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 56 2023-11-24T06:12:25.283 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.401+0000 7f82cdf85700 1 Processor -- start 2023-11-24T06:12:25.403 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.401+0000 7f82cdf85700 1 -- start start 2023-11-24T06:12:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.401+0000 7f82cdf85700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f82c81446a0 0x7f82c80b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82cdf85700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f82c806c710 0x7f82c80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82cdf85700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c80af950 con 0x7f82c80adf00 2023-11-24T06:12:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82cdf85700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c8144a80 con 0x7f82c806c710 2023-11-24T06:12:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82cdf85700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c80afc30 con 0x7f82c81446a0 2023-11-24T06:12:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c7fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f82c806c710 0x7f82c80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.406 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c6ffd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f82c80adf00 0x7f82c806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44094/0 (socket says 172.21.15.97:44094) 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c6ffd700 1 -- 172.21.15.97:0/275337683 learned_addr learned my addr 172.21.15.97:0/275337683 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c77fe700 1 --2- 172.21.15.97:0/275337683 >> v2:172.21.15.183:3300/0 conn(0x7f82c81446a0 0x7f82c80b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c7fff700 1 -- 172.21.15.97:0/275337683 >> v2:172.21.15.183:3300/0 conn(0x7f82c81446a0 msgr2=0x7f82c80b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c7fff700 1 --2- 172.21.15.97:0/275337683 >> v2:172.21.15.183:3300/0 conn(0x7f82c81446a0 0x7f82c80b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c7fff700 1 -- 172.21.15.97:0/275337683 >> v1:172.21.15.97:6789/0 conn(0x7f82c80adf00 legacy=0x7f82c806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c7fff700 1 -- 172.21.15.97:0/275337683 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c80aff10 con 0x7f82c806c710 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c67fc700 1 -- 172.21.15.97:0/275337683 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315698804 0 0) 0x7f82c80af950 con 0x7f82c80adf00 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c77fe700 1 --2- 172.21.15.97:0/275337683 >> v2:172.21.15.183:3300/0 conn(0x7f82c81446a0 0x7f82c80b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c7fff700 1 --2- 172.21.15.97:0/275337683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c806c710 0x7f82c80b3d40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f82c80729a0 tx=0x7f82b8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f603a6acfb812f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c67fc700 1 -- 172.21.15.97:0/275337683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8011830 con 0x7f82c806c710 2023-11-24T06:12:25.407 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c67fc700 1 -- 172.21.15.97:0/275337683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b8011990 con 0x7f82c806c710 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.402+0000 7f82c67fc700 1 -- 172.21.15.97:0/275337683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8011d20 con 0x7f82c806c710 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.403+0000 7f82cdf85700 1 -- 172.21.15.97:0/275337683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c806c710 msgr2=0x7f82c80b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.403+0000 7f82cdf85700 1 --2- 172.21.15.97:0/275337683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c806c710 0x7f82c80b3d40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f82c80729a0 tx=0x7f82b8009b20 comp rx=0 tx=0).stop 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.403+0000 7f82cdf85700 1 -- 172.21.15.97:0/275337683 shutdown_connections 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.403+0000 7f82cdf85700 1 --2- 172.21.15.97:0/275337683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c806c710 0x7f82c80b3d40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.403+0000 7f82cdf85700 1 --2- 172.21.15.97:0/275337683 >> v2:172.21.15.183:3300/0 conn(0x7f82c81446a0 0x7f82c80b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.403+0000 7f82cdf85700 1 -- 172.21.15.97:0/275337683 >> 172.21.15.97:0/275337683 conn(0x7f82c8071b90 msgr2=0x7f82c8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.403+0000 7f82cdf85700 1 -- 172.21.15.97:0/275337683 shutdown_connections 2023-11-24T06:12:25.408 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.403+0000 7f82cdf85700 1 -- 172.21.15.97:0/275337683 wait complete. 2023-11-24T06:12:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.404+0000 7f82cdf85700 1 Processor -- start 2023-11-24T06:12:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.404+0000 7f82cdf85700 1 -- start start 2023-11-24T06:12:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82cdf85700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c806c710 0x7f82c813d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82cdf85700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82c80adf00 0x7f82c813f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82cdf85700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c81446a0 0x7f82c8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82cdf85700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c813c020 con 0x7f82c81446a0 2023-11-24T06:12:25.411 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82cdf85700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c813c270 con 0x7f82c806c710 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82cdf85700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c813c550 con 0x7f82c80adf00 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c81446a0 0x7f82c8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c81446a0 0x7f82c8159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51650/0 (socket says 172.21.15.97:51650) 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 -- 172.21.15.97:0/4123262250 learned_addr learned my addr 172.21.15.97:0/4123262250 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c77fe700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c806c710 0x7f82c813d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c6ffd700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82c80adf00 0x7f82c813f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 -- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82c80adf00 msgr2=0x7f82c813f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82c80adf00 0x7f82c813f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 -- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c806c710 msgr2=0x7f82c813d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c806c710 0x7f82c813d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 -- 172.21.15.97:0/4123262250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b80094a0 con 0x7f82c81446a0 2023-11-24T06:12:25.412 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c77fe700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c806c710 0x7f82c813d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.405+0000 7f82c7fff700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c81446a0 0x7f82c8159fd0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f82b8014040 tx=0x7f82b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.406+0000 7f82bffff700 1 -- 172.21.15.97:0/4123262250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80038a0 con 0x7f82c81446a0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.406+0000 7f82bffff700 1 -- 172.21.15.97:0/4123262250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b801e070 con 0x7f82c81446a0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.406+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c813c830 con 0x7f82c81446a0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.406+0000 7f82bffff700 1 -- 172.21.15.97:0/4123262250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8003c60 con 0x7f82c81446a0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.406+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c813cdf0 con 0x7f82c81446a0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.407+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c80753f0 con 0x7f82c81446a0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.407+0000 7f82bffff700 1 -- 172.21.15.97:0/4123262250 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f82b802d450 con 0x7f82c81446a0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.407+0000 7f82bffff700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f82a806b500 0x7f82a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.407+0000 7f82bffff700 1 -- 172.21.15.97:0/4123262250 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f82b808b250 con 0x7f82c81446a0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.408+0000 7f82c77fe700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f82a806b500 0x7f82a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.413 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.408+0000 7f82c77fe700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f82a806b500 0x7f82a806d9c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f82c000d430 tx=0x7f82c0007530 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.414 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.412+0000 7f82bffff700 1 -- 172.21.15.97:0/4123262250 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82b80559f0 con 0x7f82c81446a0 2023-11-24T06:12:25.604 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.601+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82c8067f60 con 0x7f82c81446a0 2023-11-24T06:12:25.605 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.603+0000 7f82bffff700 1 -- 172.21.15.97:0/4123262250 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16615 (secure 0 0 0) 0x7f82b8059010 con 0x7f82c81446a0 2023-11-24T06:12:25.605 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:25.606 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:09.577392+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:05.739467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:06.934719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f82a806b500 msgr2=0x7f82a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f82a806b500 0x7f82a806d9c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f82c000d430 tx=0x7f82c0007530 comp rx=0 tx=0).stop 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c81446a0 msgr2=0x7f82c8159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c81446a0 0x7f82c8159fd0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f82b8014040 tx=0x7f82b8004750 comp rx=0 tx=0).stop 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 shutdown_connections 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82c81446a0 0x7f82c8159fd0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82c806c710 0x7f82c813d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f82a806b500 0x7f82a806d9c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 --2- 172.21.15.97:0/4123262250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f82c80adf00 0x7f82c813f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.608 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 >> 172.21.15.97:0/4123262250 conn(0x7f82c8071b90 msgr2=0x7f82c80b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:25.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 shutdown_connections 2023-11-24T06:12:25.609 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.605+0000 7f82cdf85700 1 -- 172.21.15.97:0/4123262250 wait complete. 2023-11-24T06:12:25.620 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:12:25.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:25.742 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.740+0000 7eff21f5c700 1 Processor -- start 2023-11-24T06:12:25.743 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.740+0000 7eff21f5c700 1 -- start start 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.740+0000 7eff21f5c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c15c160 0x7eff1c0622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.740+0000 7eff21f5c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff1c0b2870 0x7eff1c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.740+0000 7eff21f5c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1c15c540 con 0x7eff1c0b02e0 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.740+0000 7eff21f5c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1c0b2cf0 con 0x7eff1c0b2870 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.740+0000 7eff21f5c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c0b0770 con 0x7eff1c15c160 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff13fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff1c0b2870 0x7eff1c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff13fff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff1c0b2870 0x7eff1c0afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51654/0 (socket says 172.21.15.97:51654) 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff13fff700 1 -- 172.21.15.97:0/708228787 learned_addr learned my addr 172.21.15.97:0/708228787 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:25.745 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff1b7fe700 1 --2- 172.21.15.97:0/708228787 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c15c160 0x7eff1c0622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff13fff700 1 -- 172.21.15.97:0/708228787 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c15c160 msgr2=0x7eff1c0622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff13fff700 1 --2- 172.21.15.97:0/708228787 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c15c160 0x7eff1c0622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff13fff700 1 -- 172.21.15.97:0/708228787 >> v1:172.21.15.183:6789/0 conn(0x7eff1c0b02e0 legacy=0x7eff1c0ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff13fff700 1 -- 172.21.15.97:0/708228787 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c0b2ee0 con 0x7eff1c0b2870 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff13fff700 1 --2- 172.21.15.97:0/708228787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b2870 0x7eff1c0afda0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7eff040098f0 tx=0x7eff04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82c69f2f711c9f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff1affd700 1 -- 172.21.15.97:0/708228787 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0400b8a0 con 0x7eff1c0b2870 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff1affd700 1 -- 172.21.15.97:0/708228787 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0400ba00 con 0x7eff1c0b2870 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.741+0000 7eff1affd700 1 -- 172.21.15.97:0/708228787 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0400bd90 con 0x7eff1c0b2870 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.742+0000 7eff21f5c700 1 -- 172.21.15.97:0/708228787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b2870 msgr2=0x7eff1c0afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.742+0000 7eff21f5c700 1 --2- 172.21.15.97:0/708228787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b2870 0x7eff1c0afda0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7eff040098f0 tx=0x7eff04009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.742+0000 7eff21f5c700 1 -- 172.21.15.97:0/708228787 shutdown_connections 2023-11-24T06:12:25.746 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.742+0000 7eff21f5c700 1 --2- 172.21.15.97:0/708228787 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c15c160 0x7eff1c0622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.742+0000 7eff21f5c700 1 --2- 172.21.15.97:0/708228787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b2870 0x7eff1c0afda0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.742+0000 7eff21f5c700 1 -- 172.21.15.97:0/708228787 >> 172.21.15.97:0/708228787 conn(0x7eff1c0b57b0 msgr2=0x7eff1c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:25.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.742+0000 7eff21f5c700 1 -- 172.21.15.97:0/708228787 shutdown_connections 2023-11-24T06:12:25.747 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.742+0000 7eff21f5c700 1 -- 172.21.15.97:0/708228787 wait complete. 2023-11-24T06:12:25.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.743+0000 7eff21f5c700 1 Processor -- start 2023-11-24T06:12:25.749 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.743+0000 7eff21f5c700 1 -- start start 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff21f5c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b02e0 0x7eff1c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff21f5c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c0b2870 0x7eff1c154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff21f5c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c15c160 0x7eff1c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff21f5c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c15c6d0 con 0x7eff1c0b02e0 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff21f5c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c15d250 con 0x7eff1c15c160 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff21f5c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c15c920 con 0x7eff1c0b2870 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b02e0 0x7eff1c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b02e0 0x7eff1c154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51658/0 (socket says 172.21.15.97:51658) 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 -- 172.21.15.97:0/1818962250 learned_addr learned my addr 172.21.15.97:0/1818962250 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:25.750 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1bfff700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c15c160 0x7eff1c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff13fff700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c0b2870 0x7eff1c154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 -- 172.21.15.97:0/1818962250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c0b2870 msgr2=0x7eff1c154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c0b2870 0x7eff1c154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 -- 172.21.15.97:0/1818962250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c15c160 msgr2=0x7eff1c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c15c160 0x7eff1c151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 -- 172.21.15.97:0/1818962250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff04009550 con 0x7eff1c0b02e0 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.744+0000 7eff1b7fe700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b02e0 0x7eff1c154020 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00d830 tx=0x7eff0c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.745+0000 7eff18ff9700 1 -- 172.21.15.97:0/1818962250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0125f0 con 0x7eff1c0b02e0 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.745+0000 7eff18ff9700 1 -- 172.21.15.97:0/1818962250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0c012750 con 0x7eff1c0b02e0 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.745+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c15cc00 con 0x7eff1c0b02e0 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.745+0000 7eff18ff9700 1 -- 172.21.15.97:0/1818962250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c012b00 con 0x7eff1c0b02e0 2023-11-24T06:12:25.751 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.745+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c155010 con 0x7eff1c0b02e0 2023-11-24T06:12:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.746+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c0b0ec0 con 0x7eff1c0b02e0 2023-11-24T06:12:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.746+0000 7eff18ff9700 1 -- 172.21.15.97:0/1818962250 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7eff0c012c60 con 0x7eff1c0b02e0 2023-11-24T06:12:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.746+0000 7eff18ff9700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efefc06b530 0x7efefc06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.747+0000 7eff18ff9700 1 -- 172.21.15.97:0/1818962250 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7eff0c08a080 con 0x7eff1c0b02e0 2023-11-24T06:12:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.747+0000 7eff13fff700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efefc06b530 0x7efefc06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:25.752 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.747+0000 7eff13fff700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efefc06b530 0x7efefc06d9f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7eff04009c40 tx=0x7eff04008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:25.753 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.751+0000 7eff18ff9700 1 -- 172.21.15.97:0/1818962250 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff1c0b0ec0 con 0x7eff1c0b02e0 2023-11-24T06:12:25.987 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.984+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff1c0b0ec0 con 0x7eff1c0b02e0 2023-11-24T06:12:25.988 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.985+0000 7eff18ff9700 1 -- 172.21.15.97:0/1818962250 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4921 (secure 0 0 0) 0x7eff1c0b0ec0 con 0x7eff1c0b02e0 2023-11-24T06:12:25.988 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:25.988 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6903,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3848239046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3848239046},{"type":"v1","addr":"172.21.15.97:6836","nonce":3848239046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3181686967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3181686967},{"type":"v1","addr":"172.21.15.97:6837","nonce":3181686967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/4023096288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4023096288},{"type":"v1","addr":"172.21.15.183:6834","nonce":4023096288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:07.978316+0000","modified":"2023-11-24T06:12:11.003557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6921},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3868240464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3868240464},{"type":"v1","addr":"172.21.15.183:6835","nonce":3868240464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:12:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efefc06b530 msgr2=0x7efefc06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efefc06b530 0x7efefc06d9f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7eff04009c40 tx=0x7eff04008030 comp rx=0 tx=0).stop 2023-11-24T06:12:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b02e0 msgr2=0x7eff1c154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:25.990 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b02e0 0x7eff1c154020 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00d830 tx=0x7eff0c00db40 comp rx=0 tx=0).stop 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 shutdown_connections 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff1c15c160 0x7eff1c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff1c0b02e0 0x7eff1c154020 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7efefc06b530 0x7efefc06d9f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 --2- 172.21.15.97:0/1818962250 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7eff1c0b2870 0x7eff1c154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 >> 172.21.15.97:0/1818962250 conn(0x7eff1c0b57b0 msgr2=0x7eff1c06e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 shutdown_connections 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:25.987+0000 7eff21f5c700 1 -- 172.21.15.97:0/1818962250 wait complete. 2023-11-24T06:12:25.991 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 56 2023-11-24T06:12:26.004 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:26.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.121+0000 7fbe7018f700 1 Processor -- start 2023-11-24T06:12:26.123 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.121+0000 7fbe7018f700 1 -- start start 2023-11-24T06:12:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe7018f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe680adf00 0x7fbe6806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe7018f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe6806c540 0x7fbe680b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe7018f700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe680af950 con 0x7fbe681446a0 2023-11-24T06:12:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe7018f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe68144a80 con 0x7fbe680adf00 2023-11-24T06:12:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe7018f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6806c9c0 con 0x7fbe6806c540 2023-11-24T06:12:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6d72a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe680adf00 0x7fbe6806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6d72a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbe680adf00 0x7fbe6806c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51668/0 (socket says 172.21.15.97:51668) 2023-11-24T06:12:26.126 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6d72a700 1 -- 172.21.15.97:0/1094765799 learned_addr learned my addr 172.21.15.97:0/1094765799 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6e72c700 1 --2- 172.21.15.97:0/1094765799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe6806c540 0x7fbe680b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6d72a700 1 -- 172.21.15.97:0/1094765799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe6806c540 msgr2=0x7fbe680b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6d72a700 1 --2- 172.21.15.97:0/1094765799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe6806c540 0x7fbe680b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6d72a700 1 -- 172.21.15.97:0/1094765799 >> v1:172.21.15.183:6789/0 conn(0x7fbe681446a0 legacy=0x7fbe680b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6d72a700 1 -- 172.21.15.97:0/1094765799 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe680afc30 con 0x7fbe680adf00 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6e72c700 1 --2- 172.21.15.97:0/1094765799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe6806c540 0x7fbe680b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6d72a700 1 --2- 172.21.15.97:0/1094765799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe6806c000 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800bb90 tx=0x7fbe5800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375d9fd4e9daeea9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6cf29700 1 -- 172.21.15.97:0/1094765799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5800dd10 con 0x7fbe680adf00 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6cf29700 1 -- 172.21.15.97:0/1094765799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5800de70 con 0x7fbe680adf00 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.122+0000 7fbe6cf29700 1 -- 172.21.15.97:0/1094765799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58004230 con 0x7fbe680adf00 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.123+0000 7fbe7018f700 1 -- 172.21.15.97:0/1094765799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 msgr2=0x7fbe6806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.127 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.123+0000 7fbe7018f700 1 --2- 172.21.15.97:0/1094765799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe6806c000 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800bb90 tx=0x7fbe5800bea0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.123+0000 7fbe7018f700 1 -- 172.21.15.97:0/1094765799 shutdown_connections 2023-11-24T06:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.123+0000 7fbe7018f700 1 --2- 172.21.15.97:0/1094765799 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe6806c540 0x7fbe680b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.123+0000 7fbe7018f700 1 --2- 172.21.15.97:0/1094765799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe6806c000 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.123+0000 7fbe7018f700 1 -- 172.21.15.97:0/1094765799 >> 172.21.15.97:0/1094765799 conn(0x7fbe68071b90 msgr2=0x7fbe68071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.123+0000 7fbe7018f700 1 -- 172.21.15.97:0/1094765799 shutdown_connections 2023-11-24T06:12:26.128 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.123+0000 7fbe7018f700 1 -- 172.21.15.97:0/1094765799 wait complete. 2023-11-24T06:12:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.124+0000 7fbe7018f700 1 Processor -- start 2023-11-24T06:12:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.124+0000 7fbe7018f700 1 -- start start 2023-11-24T06:12:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe7018f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6806c540 0x7fbe68159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe7018f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe680b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe7018f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe681446a0 0x7fbe680b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe7018f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6815aa40 con 0x7fbe680adf00 2023-11-24T06:12:26.131 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe7018f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6813c780 con 0x7fbe6806c540 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe7018f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe680ceaf0 con 0x7fbe681446a0 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6d72a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe680b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6d72a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe680b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51672/0 (socket says 172.21.15.97:51672) 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6d72a700 1 -- 172.21.15.97:0/984897846 learned_addr learned my addr 172.21.15.97:0/984897846 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6e72c700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe681446a0 0x7fbe680b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6df2b700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6806c540 0x7fbe68159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6d72a700 1 -- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe681446a0 msgr2=0x7fbe680b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6d72a700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe681446a0 0x7fbe680b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6d72a700 1 -- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6806c540 msgr2=0x7fbe68159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6d72a700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6806c540 0x7fbe68159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6d72a700 1 -- 172.21.15.97:0/984897846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5800b7f0 con 0x7fbe680adf00 2023-11-24T06:12:26.132 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.125+0000 7fbe6df2b700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6806c540 0x7fbe68159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.126+0000 7fbe6d72a700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe680b0840 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800b340 tx=0x7fbe58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.126+0000 7fbe5e7fc700 1 -- 172.21.15.97:0/984897846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58020070 con 0x7fbe680adf00 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.126+0000 7fbe5e7fc700 1 -- 172.21.15.97:0/984897846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe58003b20 con 0x7fbe680adf00 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.126+0000 7fbe5e7fc700 1 -- 172.21.15.97:0/984897846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58010470 con 0x7fbe680adf00 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.126+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe680cedd0 con 0x7fbe680adf00 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.126+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6813ca60 con 0x7fbe680adf00 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.127+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe680753f0 con 0x7fbe680adf00 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.127+0000 7fbe5e7fc700 1 -- 172.21.15.97:0/984897846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fbe5802d430 con 0x7fbe680adf00 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.128+0000 7fbe5e7fc700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbe4c06b500 0x7fbe4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.128+0000 7fbe5e7fc700 1 -- 172.21.15.97:0/984897846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbe5808d650 con 0x7fbe680adf00 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.128+0000 7fbe6df2b700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbe4c06b500 0x7fbe4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.133 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.128+0000 7fbe6df2b700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbe4c06b500 0x7fbe4c06d9c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbe680b1ca0 tx=0x7fbe64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.135 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.132+0000 7fbe5e7fc700 1 -- 172.21.15.97:0/984897846 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbe58057da0 con 0x7fbe680adf00 2023-11-24T06:12:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.363+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe680cf290 con 0x7fbe680adf00 2023-11-24T06:12:26.366 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.364+0000 7fbe5e7fc700 1 -- 172.21.15.97:0/984897846 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fbe5805b3c0 con 0x7fbe680adf00 2023-11-24T06:12:26.367 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:26.367 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6903,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3848239046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3848239046},{"type":"v1","addr":"172.21.15.97:6836","nonce":3848239046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3181686967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3181686967},{"type":"v1","addr":"172.21.15.97:6837","nonce":3181686967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/4023096288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4023096288},{"type":"v1","addr":"172.21.15.183:6834","nonce":4023096288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:12:07.978316+0000","modified":"2023-11-24T06:12:11.003557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6921},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6921":{"gid":6921,"name":"d","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6835/3868240464","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":3868240464},{"type":"v1","addr":"172.21.15.183:6835","nonce":3868240464}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T06:12:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbe4c06b500 msgr2=0x7fbe4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbe4c06b500 0x7fbe4c06d9c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbe680b1ca0 tx=0x7fbe64008040 comp rx=0 tx=0).stop 2023-11-24T06:12:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 msgr2=0x7fbe680b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe680b0840 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800b340 tx=0x7fbe58004750 comp rx=0 tx=0).stop 2023-11-24T06:12:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 shutdown_connections 2023-11-24T06:12:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbe4c06b500 0x7fbe4c06d9c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.369 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbe681446a0 0x7fbe680b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6806c540 0x7fbe68159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 --2- 172.21.15.97:0/984897846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe680adf00 0x7fbe680b0840 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 >> 172.21.15.97:0/984897846 conn(0x7fbe68071b90 msgr2=0x7fbe6815efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 shutdown_connections 2023-11-24T06:12:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.366+0000 7fbe7018f700 1 -- 172.21.15.97:0/984897846 wait complete. 2023-11-24T06:12:26.370 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 56 2023-11-24T06:12:26.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.498+0000 7fd191680700 1 Processor -- start 2023-11-24T06:12:26.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.498+0000 7fd191680700 1 -- start start 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.499+0000 7fd191680700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c0b3370 0x7fd18c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.499+0000 7fd191680700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd18c0d1c70 0x7fd18c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.499+0000 7fd191680700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0b22d0 con 0x7fd18c15bdd0 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.499+0000 7fd191680700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18c0b2f50 con 0x7fd18c0d1c70 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.499+0000 7fd191680700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0d2640 con 0x7fd18c0b3370 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.499+0000 7fd18b7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd18c0d1c70 0x7fd18c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.499+0000 7fd18affd700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd18c15bdd0 0x7fd18c0b1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44132/0 (socket says 172.21.15.97:44132) 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.499+0000 7fd18affd700 1 -- 172.21.15.97:0/2085960968 learned_addr learned my addr 172.21.15.97:0/2085960968 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd18a7fc700 1 --2- 172.21.15.97:0/2085960968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c0b3370 0x7fd18c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.504 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd18b7fe700 1 -- 172.21.15.97:0/2085960968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c0b3370 msgr2=0x7fd18c1604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd18b7fe700 1 --2- 172.21.15.97:0/2085960968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c0b3370 0x7fd18c1604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd18b7fe700 1 -- 172.21.15.97:0/2085960968 >> v1:172.21.15.97:6789/0 conn(0x7fd18c15bdd0 legacy=0x7fd18c0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd189ffb700 1 -- 172.21.15.97:0/2085960968 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373007139 0 0) 0x7fd18c0b22d0 con 0x7fd18c15bdd0 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd18b7fe700 1 -- 172.21.15.97:0/2085960968 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0b25b0 con 0x7fd18c0d1c70 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd18b7fe700 1 --2- 172.21.15.97:0/2085960968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c0d2100 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd184009d90 tx=0x7fd18400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60af275093c13a30 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd189ffb700 1 -- 172.21.15.97:0/2085960968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18400e070 con 0x7fd18c0d1c70 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd189ffb700 1 -- 172.21.15.97:0/2085960968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18400bb20 con 0x7fd18c0d1c70 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.500+0000 7fd189ffb700 1 -- 172.21.15.97:0/2085960968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184012430 con 0x7fd18c0d1c70 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.501+0000 7fd191680700 1 -- 172.21.15.97:0/2085960968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 msgr2=0x7fd18c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.501+0000 7fd191680700 1 --2- 172.21.15.97:0/2085960968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c0d2100 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd184009d90 tx=0x7fd18400b330 comp rx=0 tx=0).stop 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.501+0000 7fd191680700 1 -- 172.21.15.97:0/2085960968 shutdown_connections 2023-11-24T06:12:26.505 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.501+0000 7fd191680700 1 --2- 172.21.15.97:0/2085960968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c0d2100 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.501+0000 7fd191680700 1 --2- 172.21.15.97:0/2085960968 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c0b3370 0x7fd18c1604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.501+0000 7fd191680700 1 -- 172.21.15.97:0/2085960968 >> 172.21.15.97:0/2085960968 conn(0x7fd18c0731a0 msgr2=0x7fd18c09aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.501+0000 7fd191680700 1 -- 172.21.15.97:0/2085960968 shutdown_connections 2023-11-24T06:12:26.506 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.501+0000 7fd191680700 1 -- 172.21.15.97:0/2085960968 wait complete. 2023-11-24T06:12:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.502+0000 7fd191680700 1 Processor -- start 2023-11-24T06:12:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.502+0000 7fd191680700 1 -- start start 2023-11-24T06:12:26.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd191680700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18c0b3370 0x7fd18c148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd191680700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c1e5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd191680700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c15bdd0 0x7fd18c1f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd191680700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c06c9a0 con 0x7fd18c0d1c70 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd191680700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c1e5c50 con 0x7fd18c0b3370 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd191680700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c06bbe0 con 0x7fd18c15bdd0 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c1e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c1e5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51688/0 (socket says 172.21.15.97:51688) 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 -- 172.21.15.97:0/4173209166 learned_addr learned my addr 172.21.15.97:0/4173209166 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18affd700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18c0b3370 0x7fd18c148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18b7fe700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c15bdd0 0x7fd18c1f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 -- 172.21.15.97:0/4173209166 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c15bdd0 msgr2=0x7fd18c1f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c15bdd0 0x7fd18c1f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 -- 172.21.15.97:0/4173209166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18c0b3370 msgr2=0x7fd18c148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18c0b3370 0x7fd18c148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 -- 172.21.15.97:0/4173209166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184009a70 con 0x7fd18c0d1c70 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd18a7fc700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c1e5400 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e9b0 tx=0x7fd17c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd1737fe700 1 -- 172.21.15.97:0/4173209166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c009710 con 0x7fd18c0d1c70 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd1737fe700 1 -- 172.21.15.97:0/4173209166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c00ee50 con 0x7fd18c0d1c70 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd1737fe700 1 -- 172.21.15.97:0/4173209166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00ca50 con 0x7fd18c0d1c70 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.503+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c06bec0 con 0x7fd18c0d1c70 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.504+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c06c480 con 0x7fd18c0d1c70 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.504+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c06c7f0 con 0x7fd18c0d1c70 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.505+0000 7fd1737fe700 1 -- 172.21.15.97:0/4173209166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd17c00cbb0 con 0x7fd18c0d1c70 2023-11-24T06:12:26.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.505+0000 7fd1737fe700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd17806b500 0x7fd17806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.505+0000 7fd1737fe700 1 -- 172.21.15.97:0/4173209166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd17c08a100 con 0x7fd18c0d1c70 2023-11-24T06:12:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.505+0000 7fd18affd700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd17806b500 0x7fd17806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.506+0000 7fd18affd700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd17806b500 0x7fd17806d9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd180000f60 tx=0x7fd180007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.510+0000 7fd1737fe700 1 -- 172.21.15.97:0/4173209166 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd17c0bce90 con 0x7fd18c0d1c70 2023-11-24T06:12:26.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.707+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd18c067f60 con 0x7fd18c0d1c70 2023-11-24T06:12:26.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.708+0000 7fd1737fe700 1 -- 172.21.15.97:0/4173209166 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16615 (secure 0 0 0) 0x7fd17c054850 con 0x7fd18c0d1c70 2023-11-24T06:12:26.711 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:26.712 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:09.577392+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:12:05.739467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T06:12:06.934719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:26.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd17806b500 msgr2=0x7fd17806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd17806b500 0x7fd17806d9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd180000f60 tx=0x7fd180007e70 comp rx=0 tx=0).stop 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 msgr2=0x7fd18c1e5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c1e5400 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e9b0 tx=0x7fd17c00ecc0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 shutdown_connections 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd17806b500 0x7fd17806d9c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd18c15bdd0 0x7fd18c1f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd18c0b3370 0x7fd18c148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 --2- 172.21.15.97:0/4173209166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd18c0d1c70 0x7fd18c1e5400 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 >> 172.21.15.97:0/4173209166 conn(0x7fd18c0731a0 msgr2=0x7fd18c0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 shutdown_connections 2023-11-24T06:12:26.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.711+0000 7fd191680700 1 -- 172.21.15.97:0/4173209166 wait complete. 2023-11-24T06:12:26.726 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:12:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.845+0000 7f3842569700 1 Processor -- start 2023-11-24T06:12:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.845+0000 7f3842569700 1 -- start start 2023-11-24T06:12:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f3842569700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f383c15ae20 0x7f383c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f3842569700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f3842569700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383c0cfd40 con 0x7f383c0cf780 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f3842569700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383c0d0020 con 0x7f383c15ae20 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f3842569700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383c0affa0 con 0x7f383c15b200 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f383bfff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f383c15ae20 0x7f383c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f383b7fe700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f383bfff700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f383c15ae20 0x7f383c0b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:45854/0 (socket says 172.21.15.97:45854) 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f383bfff700 1 -- 172.21.15.97:0/3672587320 learned_addr learned my addr 172.21.15.97:0/3672587320 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f383b7fe700 1 -- 172.21.15.97:0/3672587320 >> v2:172.21.15.183:3300/0 conn(0x7f383c15ae20 msgr2=0x7f383c0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f383b7fe700 1 --2- 172.21.15.97:0/3672587320 >> v2:172.21.15.183:3300/0 conn(0x7f383c15ae20 0x7f383c0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.846+0000 7f383b7fe700 1 -- 172.21.15.97:0/3672587320 >> v1:172.21.15.183:6789/0 conn(0x7f383c0cf780 legacy=0x7f383c0b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f383b7fe700 1 -- 172.21.15.97:0/3672587320 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c0b0280 con 0x7f383c15b200 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f383bfff700 1 --2- 172.21.15.97:0/3672587320 >> v2:172.21.15.183:3300/0 conn(0x7f383c15ae20 0x7f383c0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f383b7fe700 1 --2- 172.21.15.97:0/3672587320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c0b3290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f382c00bae0 tx=0x7f382c00bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9e2e7cd7851ecb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f383affd700 1 -- 172.21.15.97:0/3672587320 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00dc70 con 0x7f383c15b200 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f383affd700 1 -- 172.21.15.97:0/3672587320 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c00ddd0 con 0x7f383c15b200 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f383affd700 1 -- 172.21.15.97:0/3672587320 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c010690 con 0x7f383c15b200 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f3842569700 1 -- 172.21.15.97:0/3672587320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 msgr2=0x7f383c0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f3842569700 1 --2- 172.21.15.97:0/3672587320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c0b3290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f382c00bae0 tx=0x7f382c00bdf0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f3842569700 1 -- 172.21.15.97:0/3672587320 shutdown_connections 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f3842569700 1 --2- 172.21.15.97:0/3672587320 >> v2:172.21.15.183:3300/0 conn(0x7f383c15ae20 0x7f383c0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f3842569700 1 --2- 172.21.15.97:0/3672587320 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c0b3290 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.847+0000 7f3842569700 1 -- 172.21.15.97:0/3672587320 >> 172.21.15.97:0/3672587320 conn(0x7f383c0b57b0 msgr2=0x7f383c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:26.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.848+0000 7f3842569700 1 -- 172.21.15.97:0/3672587320 shutdown_connections 2023-11-24T06:12:26.853 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.848+0000 7f3842569700 1 -- 172.21.15.97:0/3672587320 wait complete. 2023-11-24T06:12:26.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.849+0000 7f3842569700 1 Processor -- start 2023-11-24T06:12:26.855 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.849+0000 7f3842569700 1 -- start start 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3842569700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c0cf780 0x7f383c1f2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3842569700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383c15ae20 0x7f383c0726e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3842569700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3842569700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c1f39b0 con 0x7f383c15ae20 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3842569700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c1f2d30 con 0x7f383c0cf780 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3842569700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383c1f3010 con 0x7f383c15b200 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383c15ae20 0x7f383c0726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383c15ae20 0x7f383c0726e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51700/0 (socket says 172.21.15.97:51700) 2023-11-24T06:12:26.856 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 -- 172.21.15.97:0/4190122070 learned_addr learned my addr 172.21.15.97:0/4190122070 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383bfff700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c0cf780 0x7f383c1f2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3840b06700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 -- 172.21.15.97:0/4190122070 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 msgr2=0x7f383c072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c072c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 -- 172.21.15.97:0/4190122070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c0cf780 msgr2=0x7f383c1f2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c0cf780 0x7f383c1f2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 -- 172.21.15.97:0/4190122070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3830009550 con 0x7f383c15ae20 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f383b7fe700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383c15ae20 0x7f383c0726e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b2c0 tx=0x7f382c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3838ff9700 1 -- 172.21.15.97:0/4190122070 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c01f070 con 0x7f383c15ae20 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3838ff9700 1 -- 172.21.15.97:0/4190122070 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c003a70 con 0x7f383c15ae20 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c00b740 con 0x7f383c15ae20 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3838ff9700 1 -- 172.21.15.97:0/4190122070 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c003e20 con 0x7f383c15ae20 2023-11-24T06:12:26.857 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.850+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383c1f35d0 con 0x7f383c15ae20 2023-11-24T06:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.851+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383c067f60 con 0x7f383c15ae20 2023-11-24T06:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.852+0000 7f3838ff9700 1 -- 172.21.15.97:0/4190122070 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f382c019400 con 0x7f383c15ae20 2023-11-24T06:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.852+0000 7f3838ff9700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f382806b500 0x7f382806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.852+0000 7f3838ff9700 1 -- 172.21.15.97:0/4190122070 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f382c08d620 con 0x7f383c15ae20 2023-11-24T06:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.852+0000 7f383bfff700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f382806b500 0x7f382806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:26.858 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.853+0000 7f383bfff700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f382806b500 0x7f382806d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3830009cf0 tx=0x7f3830008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:26.859 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:26.857+0000 7f3838ff9700 1 -- 172.21.15.97:0/4190122070 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f382c057d70 con 0x7f383c15ae20 2023-11-24T06:12:27.092 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.089+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f383c0af990 con 0x7f383c15ae20 2023-11-24T06:12:27.092 INFO:tasks.ceph.mon.a.smithi097.stderr:2023-11-24T06:12:27.090+0000 7fbb14a83700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:12:27.094 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.091+0000 7f3838ff9700 1 -- 172.21.15.97:0/4190122070 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7f382c05b390 con 0x7f383c15ae20 2023-11-24T06:12:27.097 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.094+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f382806b500 msgr2=0x7f382806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.094+0000 7f3842569700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f382806b500 0x7f382806d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3830009cf0 tx=0x7f3830008040 comp rx=0 tx=0).stop 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.094+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383c15ae20 msgr2=0x7f383c0726e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.094+0000 7f3842569700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383c15ae20 0x7f383c0726e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b2c0 tx=0x7f382c004750 comp rx=0 tx=0).stop 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.094+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 shutdown_connections 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.094+0000 7f3842569700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f382806b500 0x7f382806d9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.095+0000 7f3842569700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f383c15b200 0x7f383c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.095+0000 7f3842569700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f383c0cf780 0x7f383c1f2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.095+0000 7f3842569700 1 --2- 172.21.15.97:0/4190122070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f383c15ae20 0x7f383c0726e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.095+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 >> 172.21.15.97:0/4190122070 conn(0x7f383c0b57b0 msgr2=0x7f383c06e350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.095+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 shutdown_connections 2023-11-24T06:12:27.098 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.095+0000 7f3842569700 1 -- 172.21.15.97:0/4190122070 wait complete. 2023-11-24T06:12:27.099 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:12:27.100 INFO:tasks.ceph.mds.d.smithi183.stderr: -6594> 2023-11-24T06:12:13.566+0000 7fd9c58ac700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:12:27.109 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:12:27.136 INFO:tasks.ceph.mds.d.smithi183.stdout:starting mds.d at 2023-11-24T06:12:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.229+0000 7f699262e700 1 Processor -- start 2023-11-24T06:12:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.229+0000 7f699262e700 1 -- start start 2023-11-24T06:12:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f699262e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f698c15e250 0x7f698c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f699262e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c0afea0 0x7f698c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.234 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f699262e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c0b48f0 con 0x7f698c0b0560 2023-11-24T06:12:27.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f699262e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698c15e630 con 0x7f698c15e250 2023-11-24T06:12:27.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f699262e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c0b4bd0 con 0x7f698c0afea0 2023-11-24T06:12:27.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f698bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f698c15e250 0x7f698c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f698bfff700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f698c15e250 0x7f698c073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51710/0 (socket says 172.21.15.97:51710) 2023-11-24T06:12:27.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f698bfff700 1 -- 172.21.15.97:0/2894464139 learned_addr learned my addr 172.21.15.97:0/2894464139 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:27.235 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f6990bcb700 1 --2- 172.21.15.97:0/2894464139 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c0afea0 0x7f698c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f698bfff700 1 -- 172.21.15.97:0/2894464139 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c0afea0 msgr2=0x7f698c0744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:27.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f698bfff700 1 --2- 172.21.15.97:0/2894464139 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c0afea0 0x7f698c0744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f698bfff700 1 -- 172.21.15.97:0/2894464139 >> v1:172.21.15.183:6789/0 conn(0x7f698c0b0560 legacy=0x7f698c161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f698bfff700 1 -- 172.21.15.97:0/2894464139 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c0b50d0 con 0x7f698c15e250 2023-11-24T06:12:27.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.230+0000 7f698bfff700 1 --2- 172.21.15.97:0/2894464139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c15e250 0x7f698c073f90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f69800098f0 tx=0x7f6980009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26049226d41c9cec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f698affd700 1 -- 172.21.15.97:0/2894464139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000b8a0 con 0x7f698c15e250 2023-11-24T06:12:27.236 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f698affd700 1 -- 172.21.15.97:0/2894464139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698000ba00 con 0x7f698c15e250 2023-11-24T06:12:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f698affd700 1 -- 172.21.15.97:0/2894464139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000bd90 con 0x7f698c15e250 2023-11-24T06:12:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f699262e700 1 -- 172.21.15.97:0/2894464139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c15e250 msgr2=0x7f698c073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f699262e700 1 --2- 172.21.15.97:0/2894464139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c15e250 0x7f698c073f90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f69800098f0 tx=0x7f6980009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f699262e700 1 -- 172.21.15.97:0/2894464139 shutdown_connections 2023-11-24T06:12:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f699262e700 1 --2- 172.21.15.97:0/2894464139 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c0afea0 0x7f698c0744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f699262e700 1 --2- 172.21.15.97:0/2894464139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c15e250 0x7f698c073f90 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.237 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f699262e700 1 -- 172.21.15.97:0/2894464139 >> 172.21.15.97:0/2894464139 conn(0x7f698c072960 msgr2=0x7f698c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:27.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f699262e700 1 -- 172.21.15.97:0/2894464139 shutdown_connections 2023-11-24T06:12:27.238 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.231+0000 7f699262e700 1 -- 172.21.15.97:0/2894464139 wait complete. 2023-11-24T06:12:27.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.233+0000 7f699262e700 1 Processor -- start 2023-11-24T06:12:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.233+0000 7f699262e700 1 -- start start 2023-11-24T06:12:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f699262e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f698c0afea0 0x7f698c1456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f699262e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c0b0560 0x7f698c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f699262e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c15e250 0x7f698c0aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.241 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f699262e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c145be0 con 0x7f698c0b0560 2023-11-24T06:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f699262e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c1e67a0 con 0x7f698c0afea0 2023-11-24T06:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f699262e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f698c1e6a80 con 0x7f698c15e250 2023-11-24T06:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c0b0560 0x7f698c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c0b0560 0x7f698c0cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51714/0 (socket says 172.21.15.97:51714) 2023-11-24T06:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 -- 172.21.15.97:0/1664323932 learned_addr learned my addr 172.21.15.97:0/1664323932 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698bfff700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f698c0afea0 0x7f698c1456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f6990bcb700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c15e250 0x7f698c0aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 -- 172.21.15.97:0/1664323932 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c15e250 msgr2=0x7f698c0aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c15e250 0x7f698c0aea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 -- 172.21.15.97:0/1664323932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f698c0afea0 msgr2=0x7f698c1456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f698c0afea0 0x7f698c1456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 -- 172.21.15.97:0/1664323932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980009550 con 0x7f698c0b0560 2023-11-24T06:12:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f698b7fe700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c0b0560 0x7f698c0cfb60 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f697c00c4b0 tx=0x7f697c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.243 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.234+0000 7f6988ff9700 1 -- 172.21.15.97:0/1664323932 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c0052d0 con 0x7f698c0b0560 2023-11-24T06:12:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.235+0000 7f6988ff9700 1 -- 172.21.15.97:0/1664323932 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c005430 con 0x7f698c0b0560 2023-11-24T06:12:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.235+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c1e6d60 con 0x7f698c0b0560 2023-11-24T06:12:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.235+0000 7f6988ff9700 1 -- 172.21.15.97:0/1664323932 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c0038c0 con 0x7f698c0b0560 2023-11-24T06:12:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.235+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c1f4840 con 0x7f698c0b0560 2023-11-24T06:12:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.236+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c068fb0 con 0x7f698c0b0560 2023-11-24T06:12:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.236+0000 7f6988ff9700 1 -- 172.21.15.97:0/1664323932 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f697c003a20 con 0x7f698c0b0560 2023-11-24T06:12:27.244 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.236+0000 7f6988ff9700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f697806b500 0x7f697806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:27.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.236+0000 7f6988ff9700 1 -- 172.21.15.97:0/1664323932 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f697c0894d0 con 0x7f698c0b0560 2023-11-24T06:12:27.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.236+0000 7f698bfff700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f697806b500 0x7f697806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:27.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.237+0000 7f698bfff700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f697806b500 0x7f697806d9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6980014040 tx=0x7f6980008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:27.245 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.241+0000 7f6988ff9700 1 -- 172.21.15.97:0/1664323932 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f697c053bd0 con 0x7f698c0b0560 2023-11-24T06:12:27.483 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:27.481+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f698c074ac0 con 0x7f698c0b0560 2023-11-24T06:12:28.095 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.092+0000 7f6988ff9700 1 -- 172.21.15.97:0/1664323932 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 104+0+0 (secure 0 0 0) 0x7f697c0571f0 con 0x7f698c0b0560 2023-11-24T06:12:28.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f697806b500 msgr2=0x7f697806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f697806b500 0x7f697806d9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6980014040 tx=0x7f6980008030 comp rx=0 tx=0).stop 2023-11-24T06:12:28.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c0b0560 msgr2=0x7f698c0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c0b0560 0x7f698c0cfb60 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f697c00c4b0 tx=0x7f697c00c7c0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 shutdown_connections 2023-11-24T06:12:28.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f697806b500 0x7f697806d9c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f698c15e250 0x7f698c0aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.099 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f698c0afea0 0x7f698c1456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 --2- 172.21.15.97:0/1664323932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f698c0b0560 0x7f698c0cfb60 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 >> 172.21.15.97:0/1664323932 conn(0x7f698c072960 msgr2=0x7f698c15e770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:28.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 shutdown_connections 2023-11-24T06:12:28.100 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.096+0000 7f699262e700 1 -- 172.21.15.97:0/1664323932 wait complete. 2023-11-24T06:12:28.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:12:28.240 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.237+0000 7f843facd700 1 Processor -- start 2023-11-24T06:12:28.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.237+0000 7f843facd700 1 -- start start 2023-11-24T06:12:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843facd700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f84380b13f0 0x7f843815e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843facd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f84380b19c0 0x7f843815e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843facd700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84380b0900 con 0x7f84380ce4c0 2023-11-24T06:12:28.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843facd700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84380cefb0 con 0x7f84380b19c0 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843facd700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84380b1160 con 0x7f84380b13f0 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843e06a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f84380b19c0 0x7f843815e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843e06a700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f84380b19c0 0x7f843815e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51724/0 (socket says 172.21.15.97:51724) 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843e06a700 1 -- 172.21.15.97:0/2404412244 learned_addr learned my addr 172.21.15.97:0/2404412244 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843d068700 1 --2- 172.21.15.97:0/2404412244 >> v2:172.21.15.183:3300/0 conn(0x7f84380b13f0 0x7f843815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843e06a700 1 -- 172.21.15.97:0/2404412244 >> v2:172.21.15.183:3300/0 conn(0x7f84380b13f0 msgr2=0x7f843815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843e06a700 1 --2- 172.21.15.97:0/2404412244 >> v2:172.21.15.183:3300/0 conn(0x7f84380b13f0 0x7f843815e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843e06a700 1 -- 172.21.15.97:0/2404412244 >> v1:172.21.15.183:6789/0 conn(0x7f84380ce4c0 legacy=0x7f84380ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843e06a700 1 -- 172.21.15.97:0/2404412244 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84380b0e80 con 0x7f84380b19c0 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.238+0000 7f843d068700 1 --2- 172.21.15.97:0/2404412244 >> v2:172.21.15.183:3300/0 conn(0x7f84380b13f0 0x7f843815e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843e06a700 1 --2- 172.21.15.97:0/2404412244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f843815e5b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f84380cd840 tx=0x7f843000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01522d2f6f16fec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843c867700 1 -- 172.21.15.97:0/2404412244 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430010070 con 0x7f84380b19c0 2023-11-24T06:12:28.255 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843c867700 1 -- 172.21.15.97:0/2404412244 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843000dd50 con 0x7f84380b19c0 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843c867700 1 -- 172.21.15.97:0/2404412244 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843000b310 con 0x7f84380b19c0 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843facd700 1 -- 172.21.15.97:0/2404412244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 msgr2=0x7f843815e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843facd700 1 --2- 172.21.15.97:0/2404412244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f843815e5b0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f84380cd840 tx=0x7f843000d430 comp rx=0 tx=0).stop 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843facd700 1 -- 172.21.15.97:0/2404412244 shutdown_connections 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843facd700 1 --2- 172.21.15.97:0/2404412244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f843815e5b0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843facd700 1 --2- 172.21.15.97:0/2404412244 >> v2:172.21.15.183:3300/0 conn(0x7f84380b13f0 0x7f843815e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843facd700 1 -- 172.21.15.97:0/2404412244 >> 172.21.15.97:0/2404412244 conn(0x7f8438072710 msgr2=0x7f8438072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843facd700 1 -- 172.21.15.97:0/2404412244 shutdown_connections 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.239+0000 7f843facd700 1 -- 172.21.15.97:0/2404412244 wait complete. 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.240+0000 7f843facd700 1 Processor -- start 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843facd700 1 -- start start 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843facd700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84380b13f0 0x7f8438157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.256 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843facd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f8438157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843facd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84380ce4c0 0x7f843806e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843facd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84381587b0 con 0x7f84380b19c0 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843facd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8438158a90 con 0x7f84380ce4c0 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843facd700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8438158d70 con 0x7f84380b13f0 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843d068700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f8438157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843d068700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f8438157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51728/0 (socket says 172.21.15.97:51728) 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.241+0000 7f843d068700 1 -- 172.21.15.97:0/108419752 learned_addr learned my addr 172.21.15.97:0/108419752 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843e06a700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84380ce4c0 0x7f843806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843d869700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84380b13f0 0x7f8438157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843d068700 1 -- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84380b13f0 msgr2=0x7f8438157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843d068700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84380b13f0 0x7f8438157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843d068700 1 -- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84380ce4c0 msgr2=0x7f843806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843d068700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84380ce4c0 0x7f843806e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843d068700 1 -- 172.21.15.97:0/108419752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843000bd40 con 0x7f84380b19c0 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843e06a700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84380ce4c0 0x7f843806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843d068700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f8438157940 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f843400e7e0 tx=0x7f843400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f842e7fc700 1 -- 172.21.15.97:0/108419752 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434004230 con 0x7f84380b19c0 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f842e7fc700 1 -- 172.21.15.97:0/108419752 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434004390 con 0x7f84380b19c0 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8438159050 con 0x7f84380b19c0 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f842e7fc700 1 -- 172.21.15.97:0/108419752 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340036a0 con 0x7f84380b19c0 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.242+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8438157690 con 0x7f84380b19c0 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.243+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8438067f60 con 0x7f84380b19c0 2023-11-24T06:12:28.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.243+0000 7f842e7fc700 1 -- 172.21.15.97:0/108419752 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8434003800 con 0x7f84380b19c0 2023-11-24T06:12:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.244+0000 7f842e7fc700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f841c06b500 0x7f841c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.244+0000 7f843d869700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f841c06b500 0x7f841c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.244+0000 7f842e7fc700 1 -- 172.21.15.97:0/108419752 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8434089cd0 con 0x7f84380b19c0 2023-11-24T06:12:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.244+0000 7f843d869700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f841c06b500 0x7f841c06d9c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f84380afe00 tx=0x7f8428008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:28.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.248+0000 7f842e7fc700 1 -- 172.21.15.97:0/108419752 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8434054350 con 0x7f84380b19c0 2023-11-24T06:12:28.443 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:28.441+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8438073450 con 0x7f84380b19c0 2023-11-24T06:12:29.102 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.099+0000 7f842e7fc700 1 -- 172.21.15.97:0/108419752 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v87) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8434057970 con 0x7f84380b19c0 2023-11-24T06:12:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f841c06b500 msgr2=0x7f841c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f841c06b500 0x7f841c06d9c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f84380afe00 tx=0x7f8428008040 comp rx=0 tx=0).stop 2023-11-24T06:12:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 msgr2=0x7f8438157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f8438157940 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f843400e7e0 tx=0x7f843400eba0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.106 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 shutdown_connections 2023-11-24T06:12:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84380ce4c0 0x7f843806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f841c06b500 0x7f841c06d9c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f84380b13f0 0x7f8438157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 --2- 172.21.15.97:0/108419752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84380b19c0 0x7f8438157940 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 >> 172.21.15.97:0/108419752 conn(0x7f8438072710 msgr2=0x7f84380cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 shutdown_connections 2023-11-24T06:12:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.103+0000 7f843facd700 1 -- 172.21.15.97:0/108419752 wait complete. 2023-11-24T06:12:29.107 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:12:29.122 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:12:29.250 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.248+0000 7f987c385700 1 Processor -- start 2023-11-24T06:12:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.248+0000 7f987c385700 1 -- start start 2023-11-24T06:12:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987c385700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98740746a0 0x7f987406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987c385700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f98740b59c0 0x7f98740b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987c385700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98740b5e00 con 0x7f98740b01e0 2023-11-24T06:12:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987c385700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98741470a0 con 0x7f98740b59c0 2023-11-24T06:12:29.251 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987c385700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740b05c0 con 0x7f98740746a0 2023-11-24T06:12:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987a121700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f98740b01e0 0x7f9874073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44186/0 (socket says 172.21.15.97:44186) 2023-11-24T06:12:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987a121700 1 -- 172.21.15.97:0/3200903379 learned_addr learned my addr 172.21.15.97:0/3200903379 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194199844 0 0) 0x7f98740b5e00 con 0x7f98740b01e0 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98600035b0 con 0x7f98740b01e0 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987a922700 1 --2- 172.21.15.97:0/3200903379 >> v2:172.21.15.183:3300/0 conn(0x7f98740b59c0 0x7f98740b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f9879920700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98740746a0 0x7f987406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 680781260 0 0) 0x7f98600035b0 con 0x7f98740b01e0 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98740746a0 msgr2=0x7f987406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98740746a0 0x7f987406e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 >> v2:172.21.15.183:3300/0 conn(0x7f98740b59c0 msgr2=0x7f98740b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 --2- 172.21.15.97:0/3200903379 >> v2:172.21.15.183:3300/0 conn(0x7f98740b59c0 0x7f98740b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740b4250 con 0x7f98740b01e0 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f9870001670 con 0x7f98740b01e0 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 >> v1:172.21.15.97:6789/0 conn(0x7f98740b01e0 legacy=0x7f9874073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98740746a0 0x7f9860003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9860003f80 0x7f9860004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860004d90 0x7f9860005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98741470a0 con 0x7f9860003f80 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740b05c0 con 0x7f9860004d90 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.249+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986000ba60 con 0x7f98740746a0 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f9879920700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9860003f80 0x7f9860004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f987a121700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98740746a0 0x7f9860003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f987a922700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860004d90 0x7f9860005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f9879920700 1 -- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98740746a0 msgr2=0x7f9860003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f9879920700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f98740746a0 0x7f9860003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f9879920700 1 -- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860004d90 msgr2=0x7f9860005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f9879920700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9860004d90 0x7f9860005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f9879920700 1 -- 172.21.15.97:0/3200903379 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600035b0 con 0x7f9860003f80 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f9879920700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9860003f80 0x7f9860004850 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9864009b90 tx=0x7f9860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94be3981d66c89b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400e6c0 con 0x7f9860003f80 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986400e820 con 0x7f9860003f80 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f987911f700 1 -- 172.21.15.97:0/3200903379 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400eb90 con 0x7f9860003f80 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f987c385700 1 -- 172.21.15.97:0/3200903379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9860003f80 msgr2=0x7f9860004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f987c385700 1 --2- 172.21.15.97:0/3200903379 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9860003f80 0x7f9860004850 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f9864009b90 tx=0x7f9860005880 comp rx=0 tx=0).stop 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.250+0000 7f987a922700 1 -- 172.21.15.97:0/3200903379 reap_dead start 2023-11-24T06:12:29.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.251+0000 7f987c385700 1 -- 172.21.15.97:0/3200903379 shutdown_connections 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.251+0000 7f987c385700 1 -- 172.21.15.97:0/3200903379 >> 172.21.15.97:0/3200903379 conn(0x7f98740d0b40 msgr2=0x7f98740d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.251+0000 7f987c385700 1 -- 172.21.15.97:0/3200903379 shutdown_connections 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.251+0000 7f987c385700 1 -- 172.21.15.97:0/3200903379 wait complete. 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.252+0000 7f987c385700 1 Processor -- start 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.252+0000 7f987c385700 1 -- start start 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987c385700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9860003f80 0x7f987415df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987c385700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f987415f3d0 0x7f987415f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987c385700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98740cf320 0x7f98740cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987c385700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98740cfce0 con 0x7f98740cf320 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987c385700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9874132710 con 0x7f987415f3d0 2023-11-24T06:12:29.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987c385700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987415fe90 con 0x7f9860003f80 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98740cf320 0x7f98740cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98740cf320 0x7f98740cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51748/0 (socket says 172.21.15.97:51748) 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 -- 172.21.15.97:0/3206740502 learned_addr learned my addr 172.21.15.97:0/3206740502 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a121700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9860003f80 0x7f987415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 -- 172.21.15.97:0/3206740502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9860003f80 msgr2=0x7f987415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9860003f80 0x7f987415df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 -- 172.21.15.97:0/3206740502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f987415f3d0 msgr2=0x7f987415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f987415f3d0 0x7f987415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 -- 172.21.15.97:0/3206740502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9874160170 con 0x7f98740cf320 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987a922700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98740cf320 0x7f98740cf7a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f98740b53c0 tx=0x7f986c0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f986affd700 1 -- 172.21.15.97:0/3206740502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c00e070 con 0x7f98740cf320 2023-11-24T06:12:29.261 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f986affd700 1 -- 172.21.15.97:0/3206740502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c003c30 con 0x7f98740cf320 2023-11-24T06:12:29.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98741323a0 con 0x7f98740cf320 2023-11-24T06:12:29.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f986affd700 1 -- 172.21.15.97:0/3206740502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c003de0 con 0x7f98740cf320 2023-11-24T06:12:29.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.253+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987415e480 con 0x7f98740cf320 2023-11-24T06:12:29.262 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.254+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9874132200 con 0x7f98740cf320 2023-11-24T06:12:29.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.255+0000 7f986affd700 1 -- 172.21.15.97:0/3206740502 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f986c011a10 con 0x7f98740cf320 2023-11-24T06:12:29.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.255+0000 7f986affd700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f985806b530 0x7f985806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:29.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.255+0000 7f986affd700 1 -- 172.21.15.97:0/3206740502 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5385+0+0 (secure 0 0 0) 0x7f986c08a7b0 con 0x7f98740cf320 2023-11-24T06:12:29.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.255+0000 7f987a121700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f985806b530 0x7f985806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:29.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.255+0000 7f987a121700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f985806b530 0x7f985806d9f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9870000fb0 tx=0x7f9870005cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:29.263 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.259+0000 7f986affd700 1 -- 172.21.15.97:0/3206740502 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9874132200 con 0x7f98740cf320 2023-11-24T06:12:29.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:29.451+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9874132200 con 0x7f98740cf320 2023-11-24T06:12:30.108 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.105+0000 7f986affd700 1 -- 172.21.15.97:0/3206740502 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9874132200 con 0x7f98740cf320 2023-11-24T06:12:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f985806b530 msgr2=0x7f985806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f985806b530 0x7f985806d9f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9870000fb0 tx=0x7f9870005cd0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98740cf320 msgr2=0x7f98740cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98740cf320 0x7f98740cf7a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f98740b53c0 tx=0x7f986c0114b0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 shutdown_connections 2023-11-24T06:12:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98740cf320 0x7f98740cf7a0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.112 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f985806b530 0x7f985806d9f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f9860003f80 0x7f987415df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 --2- 172.21.15.97:0/3206740502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f987415f3d0 0x7f987415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 >> 172.21.15.97:0/3206740502 conn(0x7f98740d0b40 msgr2=0x7f98740b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 shutdown_connections 2023-11-24T06:12:30.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.109+0000 7f987c385700 1 -- 172.21.15.97:0/3206740502 wait complete. 2023-11-24T06:12:30.114 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2023-11-24T06:12:30.127 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:12:30.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.247+0000 7fe00c3b7700 1 Processor -- start 2023-11-24T06:12:30.249 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.247+0000 7fe00c3b7700 1 -- start start 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe00c3b7700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe0040746a0 0x7fe0040b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe00c3b7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fe0040afc90 0x7fe00406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe00c3b7700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0040b3c60 con 0x7fe00415e160 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe00c3b7700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0040b4500 con 0x7fe0040746a0 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe00c3b7700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0040b58c0 con 0x7fe0040afc90 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009952700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe0040746a0 0x7fe0040b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe00a153700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe00415e160 0x7fe004073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44204/0 (socket says 172.21.15.97:44204) 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe00a153700 1 -- 172.21.15.97:0/1314887583 learned_addr learned my addr 172.21.15.97:0/1314887583 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe00a954700 1 --2- 172.21.15.97:0/1314887583 >> v2:172.21.15.183:3300/0 conn(0x7fe0040afc90 0x7fe00406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.252 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009952700 1 -- 172.21.15.97:0/1314887583 >> v2:172.21.15.183:3300/0 conn(0x7fe0040afc90 msgr2=0x7fe00406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009952700 1 --2- 172.21.15.97:0/1314887583 >> v2:172.21.15.183:3300/0 conn(0x7fe0040afc90 0x7fe00406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009952700 1 -- 172.21.15.97:0/1314887583 >> v1:172.21.15.97:6789/0 conn(0x7fe00415e160 legacy=0x7fe004073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009952700 1 -- 172.21.15.97:0/1314887583 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00406bdc0 con 0x7fe0040746a0 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009151700 1 -- 172.21.15.97:0/1314887583 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663528990 0 0) 0x7fe0040b3c60 con 0x7fe00415e160 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009952700 1 --2- 172.21.15.97:0/1314887583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0040b5da0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe004162db0 tx=0x7fdff4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe0447579df6098 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009151700 1 -- 172.21.15.97:0/1314887583 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4011830 con 0x7fe0040746a0 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009151700 1 -- 172.21.15.97:0/1314887583 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff4011990 con 0x7fe0040746a0 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.248+0000 7fe009151700 1 -- 172.21.15.97:0/1314887583 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4011d20 con 0x7fe0040746a0 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.249+0000 7fe00c3b7700 1 -- 172.21.15.97:0/1314887583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 msgr2=0x7fe0040b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.253 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.249+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/1314887583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0040b5da0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe004162db0 tx=0x7fdff4009b20 comp rx=0 tx=0).stop 2023-11-24T06:12:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.249+0000 7fe00c3b7700 1 -- 172.21.15.97:0/1314887583 shutdown_connections 2023-11-24T06:12:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.249+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/1314887583 >> v2:172.21.15.183:3300/0 conn(0x7fe0040afc90 0x7fe00406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.249+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/1314887583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0040b5da0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.249+0000 7fe00c3b7700 1 -- 172.21.15.97:0/1314887583 >> 172.21.15.97:0/1314887583 conn(0x7fe0040d0bf0 msgr2=0x7fe0040d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.249+0000 7fe00c3b7700 1 -- 172.21.15.97:0/1314887583 shutdown_connections 2023-11-24T06:12:30.254 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.249+0000 7fe00c3b7700 1 -- 172.21.15.97:0/1314887583 wait complete. 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.250+0000 7fe00c3b7700 1 Processor -- start 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.250+0000 7fe00c3b7700 1 -- start start 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00c3b7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0041528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00c3b7700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0040afc90 0x7fe0041f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00c3b7700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe00415e160 0x7fe00415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00c3b7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00415e750 con 0x7fe0040746a0 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00c3b7700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0041f46e0 con 0x7fe0040afc90 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00c3b7700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0041f49c0 con 0x7fe00415e160 2023-11-24T06:12:30.257 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0041528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0041528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51758/0 (socket says 172.21.15.97:51758) 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 -- 172.21.15.97:0/482086435 learned_addr learned my addr 172.21.15.97:0/482086435 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe009952700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0040afc90 0x7fe0041f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a954700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe00415e160 0x7fe00415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 -- 172.21.15.97:0/482086435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe00415e160 msgr2=0x7fe00415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe00415e160 0x7fe00415eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 -- 172.21.15.97:0/482086435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0040afc90 msgr2=0x7fe0041f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0040afc90 0x7fe0041f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 -- 172.21.15.97:0/482086435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff40094a0 con 0x7fe0040746a0 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.251+0000 7fe00a153700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0041528b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe00000c550 tx=0x7fe00000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.258 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.252+0000 7fdffaffd700 1 -- 172.21.15.97:0/482086435 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000004c80 con 0x7fe0040746a0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.252+0000 7fdffaffd700 1 -- 172.21.15.97:0/482086435 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000004e00 con 0x7fe0040746a0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.252+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0041f4ca0 con 0x7fe0040746a0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.252+0000 7fdffaffd700 1 -- 172.21.15.97:0/482086435 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000007310 con 0x7fe0040746a0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.252+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0041f43a0 con 0x7fe0040746a0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.253+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0040b51c0 con 0x7fe0040746a0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.253+0000 7fdffaffd700 1 -- 172.21.15.97:0/482086435 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe000007470 con 0x7fe0040746a0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.253+0000 7fdffaffd700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdfe806b500 0x7fdfe806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.254+0000 7fdffaffd700 1 -- 172.21.15.97:0/482086435 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fe000089a90 con 0x7fe0040746a0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.254+0000 7fe009952700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdfe806b500 0x7fdfe806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.259 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.254+0000 7fe009952700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdfe806b500 0x7fdfe806d9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdff40090b0 tx=0x7fdff4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.260 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.258+0000 7fdffaffd700 1 -- 172.21.15.97:0/482086435 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0000544e0 con 0x7fe0040746a0 2023-11-24T06:12:30.453 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.450+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe004068fb0 con 0x7fe0040746a0 2023-11-24T06:12:30.456 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.453+0000 7fdffaffd700 1 -- 172.21.15.97:0/482086435 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe000057b00 con 0x7fe0040746a0 2023-11-24T06:12:30.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.455+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdfe806b500 msgr2=0x7fdfe806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.458 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.455+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdfe806b500 0x7fdfe806d9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdff40090b0 tx=0x7fdff4008030 comp rx=0 tx=0).stop 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.455+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 msgr2=0x7fe0041528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.455+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0041528b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe00000c550 tx=0x7fe00000c860 comp rx=0 tx=0).stop 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.456+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 shutdown_connections 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.456+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fdfe806b500 0x7fdfe806d9c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.456+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fe00415e160 0x7fe00415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.456+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0040746a0 0x7fe0041528b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.456+0000 7fe00c3b7700 1 --2- 172.21.15.97:0/482086435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0040afc90 0x7fe0041f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.456+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 >> 172.21.15.97:0/482086435 conn(0x7fe0040d0bf0 msgr2=0x7fe004072470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.456+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 shutdown_connections 2023-11-24T06:12:30.459 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.456+0000 7fe00c3b7700 1 -- 172.21.15.97:0/482086435 wait complete. 2023-11-24T06:12:30.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T06:12:30.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.587+0000 7fb525301700 1 Processor -- start 2023-11-24T06:12:30.590 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.587+0000 7fb525301700 1 -- start start 2023-11-24T06:12:30.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.588+0000 7fb525301700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb52006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.588+0000 7fb525301700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5200b49a0 0x7fb52006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.588+0000 7fb525301700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52015c4b0 con 0x7fb5200cff20 2023-11-24T06:12:30.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.588+0000 7fb525301700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52015c790 con 0x7fb5200b49a0 2023-11-24T06:12:30.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.588+0000 7fb525301700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015ca70 con 0x7fb520152cc0 2023-11-24T06:12:30.593 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.588+0000 7fb51e7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5200b49a0 0x7fb52006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.588+0000 7fb51e7fc700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb5200b49a0 0x7fb52006c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51766/0 (socket says 172.21.15.97:51766) 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.588+0000 7fb51e7fc700 1 -- 172.21.15.97:0/2548952939 learned_addr learned my addr 172.21.15.97:0/2548952939 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51effd700 1 --2- 172.21.15.97:0/2548952939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb52006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51e7fc700 1 -- 172.21.15.97:0/2548952939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 msgr2=0x7fb52006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51e7fc700 1 --2- 172.21.15.97:0/2548952939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb52006c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51e7fc700 1 -- 172.21.15.97:0/2548952939 >> v1:172.21.15.183:6789/0 conn(0x7fb5200cff20 legacy=0x7fb5200d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51e7fc700 1 -- 172.21.15.97:0/2548952939 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52015cd50 con 0x7fb5200b49a0 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51effd700 1 --2- 172.21.15.97:0/2548952939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb52006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51e7fc700 1 --2- 172.21.15.97:0/2548952939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb52006c540 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb5100098f0 tx=0x7fb510009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1d25fa56de417a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51dffb700 1 -- 172.21.15.97:0/2548952939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510004230 con 0x7fb5200b49a0 2023-11-24T06:12:30.594 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51dffb700 1 -- 172.21.15.97:0/2548952939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510004390 con 0x7fb5200b49a0 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.589+0000 7fb51dffb700 1 -- 172.21.15.97:0/2548952939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100036a0 con 0x7fb5200b49a0 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.590+0000 7fb525301700 1 -- 172.21.15.97:0/2548952939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 msgr2=0x7fb52006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.590+0000 7fb525301700 1 --2- 172.21.15.97:0/2548952939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb52006c540 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb5100098f0 tx=0x7fb510009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.590+0000 7fb525301700 1 -- 172.21.15.97:0/2548952939 shutdown_connections 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.590+0000 7fb525301700 1 --2- 172.21.15.97:0/2548952939 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb52006c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.590+0000 7fb525301700 1 --2- 172.21.15.97:0/2548952939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb52006c540 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.590+0000 7fb525301700 1 -- 172.21.15.97:0/2548952939 >> 172.21.15.97:0/2548952939 conn(0x7fb520071b90 msgr2=0x7fb520071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.590+0000 7fb525301700 1 -- 172.21.15.97:0/2548952939 shutdown_connections 2023-11-24T06:12:30.595 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.590+0000 7fb525301700 1 -- 172.21.15.97:0/2548952939 wait complete. 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.591+0000 7fb525301700 1 Processor -- start 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.591+0000 7fb525301700 1 -- start start 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.592+0000 7fb525301700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb5201536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.592+0000 7fb525301700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5200cff20 0x7fb520154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.592+0000 7fb525301700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb520147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.592+0000 7fb525301700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb520155870 con 0x7fb5200b49a0 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.592+0000 7fb525301700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb520155b50 con 0x7fb5200cff20 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.592+0000 7fb525301700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb520155e30 con 0x7fb520152cc0 2023-11-24T06:12:30.597 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.592+0000 7fb51effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb5201536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb5201536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51768/0 (socket says 172.21.15.97:51768) 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51effd700 1 -- 172.21.15.97:0/2623927028 learned_addr learned my addr 172.21.15.97:0/2623927028 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51e7fc700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5200cff20 0x7fb520154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51f7fe700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb520147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51effd700 1 -- 172.21.15.97:0/2623927028 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 msgr2=0x7fb520147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51effd700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb520147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51effd700 1 -- 172.21.15.97:0/2623927028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5200cff20 msgr2=0x7fb520154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51effd700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5200cff20 0x7fb520154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51effd700 1 -- 172.21.15.97:0/2623927028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb510009550 con 0x7fb5200b49a0 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb51effd700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb5201536b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb51400c7f0 tx=0x7fb51400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.598 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb5077fe700 1 -- 172.21.15.97:0/2623927028 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514011800 con 0x7fb5200b49a0 2023-11-24T06:12:30.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb5077fe700 1 -- 172.21.15.97:0/2623927028 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb514011960 con 0x7fb5200b49a0 2023-11-24T06:12:30.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb520156110 con 0x7fb5200b49a0 2023-11-24T06:12:30.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb5077fe700 1 -- 172.21.15.97:0/2623927028 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514011cf0 con 0x7fb5200b49a0 2023-11-24T06:12:30.599 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.593+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5201554d0 con 0x7fb5200b49a0 2023-11-24T06:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.595+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5200753f0 con 0x7fb5200b49a0 2023-11-24T06:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.595+0000 7fb5077fe700 1 -- 172.21.15.97:0/2623927028 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fb514018360 con 0x7fb5200b49a0 2023-11-24T06:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.596+0000 7fb5077fe700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb50c06b500 0x7fb50c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.596+0000 7fb5077fe700 1 -- 172.21.15.97:0/2623927028 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fb514089f70 con 0x7fb5200b49a0 2023-11-24T06:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.596+0000 7fb51e7fc700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb50c06b500 0x7fb50c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:30.600 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.596+0000 7fb51e7fc700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb50c06b500 0x7fb50c06d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb510009130 tx=0x7fb510008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:30.603 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.601+0000 7fb5077fe700 1 -- 172.21.15.97:0/2623927028 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb514054870 con 0x7fb5200b49a0 2023-11-24T06:12:30.879 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:30.877+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7fb52006dd10 con 0x7fb5200b49a0 2023-11-24T06:12:31.113 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.110+0000 7fb5077fe700 1 -- 172.21.15.97:0/2623927028 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v227) v1 ==== 164+0+0 (secure 0 0 0) 0x7fb52006dd10 con 0x7fb5200b49a0 2023-11-24T06:12:31.115 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb50c06b500 msgr2=0x7fb50c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb50c06b500 0x7fb50c06d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb510009130 tx=0x7fb510008030 comp rx=0 tx=0).stop 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 msgr2=0x7fb5201536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb5201536b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb51400c7f0 tx=0x7fb51400cbb0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 shutdown_connections 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fb50c06b500 0x7fb50c06d9c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fb520152cc0 0x7fb520147540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5200b49a0 0x7fb5201536b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 --2- 172.21.15.97:0/2623927028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5200cff20 0x7fb520154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 >> 172.21.15.97:0/2623927028 conn(0x7fb520071b90 msgr2=0x7fb5200adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 shutdown_connections 2023-11-24T06:12:31.116 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:31.113+0000 7fb525301700 1 -- 172.21.15.97:0/2623927028 wait complete. 2023-11-24T06:12:31.129 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ERROR 2023-11-24T06:12:31.129 INFO:tasks.cephfs_test_runner: 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 4537, in test_subvolume_inherited_snapshot_ls 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner: 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner:Ran 10 tests in 180.823s 2023-11-24T06:12:31.130 INFO:tasks.cephfs_test_runner: 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=5) 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner: 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 4537, in test_subvolume_inherited_snapshot_ls 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T06:12:31.131 INFO:tasks.cephfs_test_runner: 2023-11-24T06:12:31.132 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:12:31.196 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b3422b105f6e4d53b8ecf42136d28439 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T06:12:31.197 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T06:12:31.207 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-24T06:12:31.219 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-24T06:12:31.220 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:12:31.220 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:31.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:12:31.241 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-24T06:12:31.241 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:31.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284264k,nr_inodes=4071066,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13000 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T06:12:31.299 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:12:31.299 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:31.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:12:31.320 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-11-24T06:12:31.321 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:12:31.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284224k,nr_inodes=4071056,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1413 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260544k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T06:12:31.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.380 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:12:31.397 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:12:31.397 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:12:31.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:12:31.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:12:31.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-11-24T06:12:31.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.674 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-24T06:12:31.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.691 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-11-24T06:12:31.762 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2023-11-24T06:12:31.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:12:31.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.764 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:12:31.784 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:12:31.784 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:12:31.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.785 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:12:31.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.871 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:12:31.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:31.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link delete ceph-brx 2023-11-24T06:12:32.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:32.076 DEBUG:teuthology.orchestra.run.smithi183:> ip netns list 2023-11-24T06:12:32.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:12:32.093 DEBUG:teuthology.orchestra.run.smithi183:> sudo ip link delete ceph-brx 2023-11-24T06:12:32.162 INFO:teuthology.orchestra.run.smithi183.stderr:Cannot find device "ceph-brx" 2023-11-24T06:12:32.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:12:32.164 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T06:12:32.173 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:12:32.173 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.293+0000 7fbf6189c700 1 Processor -- start 2023-11-24T06:12:32.296 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.293+0000 7fbf6189c700 1 -- start start 2023-11-24T06:12:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf6189c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c145760 0x7fbf5c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf6189c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fbf5c06df20 0x7fbf5c15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf6189c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c06ee70 con 0x7fbf5c15fde0 2023-11-24T06:12:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf6189c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c06d4d0 con 0x7fbf5c06df20 2023-11-24T06:12:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf6189c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0cc5e0 con 0x7fbf5c145760 2023-11-24T06:12:32.299 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5b7fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbf5c15fde0 0x7fbf5c070ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44232/0 (socket says 172.21.15.97:44232) 2023-11-24T06:12:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5b7fe700 1 -- 172.21.15.97:0/3667514410 learned_addr learned my addr 172.21.15.97:0/3667514410 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5a7fc700 1 --2- 172.21.15.97:0/3667514410 >> v2:172.21.15.183:3300/0 conn(0x7fbf5c06df20 0x7fbf5c15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5affd700 1 --2- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c145760 0x7fbf5c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf59ffb700 1 -- 172.21.15.97:0/3667514410 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488042760 0 0) 0x7fbf5c06ee70 con 0x7fbf5c15fde0 2023-11-24T06:12:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf59ffb700 1 -- 172.21.15.97:0/3667514410 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf480035b0 con 0x7fbf5c15fde0 2023-11-24T06:12:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5a7fc700 1 -- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c145760 msgr2=0x7fbf5c06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5a7fc700 1 --2- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c145760 0x7fbf5c06e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.300 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5a7fc700 1 -- 172.21.15.97:0/3667514410 >> v1:172.21.15.97:6789/0 conn(0x7fbf5c15fde0 legacy=0x7fbf5c070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5a7fc700 1 -- 172.21.15.97:0/3667514410 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0cc8c0 con 0x7fbf5c06df20 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.294+0000 7fbf5affd700 1 --2- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c145760 0x7fbf5c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf5a7fc700 1 --2- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c06df20 0x7fbf5c15f8a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009840 tx=0x7fbf4c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e33841ed781d7a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf59ffb700 1 -- 172.21.15.97:0/3667514410 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c011830 con 0x7fbf5c06df20 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf59ffb700 1 -- 172.21.15.97:0/3667514410 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c011990 con 0x7fbf5c06df20 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf59ffb700 1 -- 172.21.15.97:0/3667514410 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c011d00 con 0x7fbf5c06df20 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf6189c700 1 -- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c06df20 msgr2=0x7fbf5c15f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf6189c700 1 --2- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c06df20 0x7fbf5c15f8a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009840 tx=0x7fbf4c009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:32.301 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf6189c700 1 -- 172.21.15.97:0/3667514410 shutdown_connections 2023-11-24T06:12:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf6189c700 1 --2- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c145760 0x7fbf5c06e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf6189c700 1 --2- 172.21.15.97:0/3667514410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c06df20 0x7fbf5c15f8a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf6189c700 1 -- 172.21.15.97:0/3667514410 >> 172.21.15.97:0/3667514410 conn(0x7fbf5c0b1740 msgr2=0x7fbf5c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf6189c700 1 -- 172.21.15.97:0/3667514410 shutdown_connections 2023-11-24T06:12:32.302 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.295+0000 7fbf6189c700 1 -- 172.21.15.97:0/3667514410 wait complete. 2023-11-24T06:12:32.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.297+0000 7fbf6189c700 1 Processor -- start 2023-11-24T06:12:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.297+0000 7fbf6189c700 1 -- start start 2023-11-24T06:12:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.297+0000 7fbf6189c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c06df20 0x7fbf5c15b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.297+0000 7fbf6189c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf5c145760 0x7fbf5c06e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.297+0000 7fbf6189c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c15fde0 0x7fbf5c13fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.297+0000 7fbf6189c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c1f3ef0 con 0x7fbf5c145760 2023-11-24T06:12:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.297+0000 7fbf6189c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c06eb40 con 0x7fbf5c15fde0 2023-11-24T06:12:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.297+0000 7fbf6189c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c1f3550 con 0x7fbf5c06df20 2023-11-24T06:12:32.305 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf5c145760 0x7fbf5c06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf5c145760 0x7fbf5c06e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51782/0 (socket says 172.21.15.97:51782) 2023-11-24T06:12:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 -- 172.21.15.97:0/4105487412 learned_addr learned my addr 172.21.15.97:0/4105487412 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5b7fe700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c15fde0 0x7fbf5c13fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5affd700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c06df20 0x7fbf5c15b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 -- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c06df20 msgr2=0x7fbf5c15b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c06df20 0x7fbf5c15b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 -- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c15fde0 msgr2=0x7fbf5c13fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c15fde0 0x7fbf5c13fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 -- 172.21.15.97:0/4105487412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440094a0 con 0x7fbf5c145760 2023-11-24T06:12:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5b7fe700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c15fde0 0x7fbf5c13fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5affd700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c06df20 0x7fbf5c15b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf5a7fc700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf5c145760 0x7fbf5c06e600 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0090b0 tx=0x7fbf4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf53fff700 1 -- 172.21.15.97:0/4105487412 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0045d0 con 0x7fbf5c145760 2023-11-24T06:12:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf53fff700 1 -- 172.21.15.97:0/4105487412 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c01f070 con 0x7fbf5c145760 2023-11-24T06:12:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf53fff700 1 -- 172.21.15.97:0/4105487412 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0038a0 con 0x7fbf5c145760 2023-11-24T06:12:32.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c0094a0 con 0x7fbf5c145760 2023-11-24T06:12:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.298+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c1f3b10 con 0x7fbf5c145760 2023-11-24T06:12:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.299+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c0751e0 con 0x7fbf5c145760 2023-11-24T06:12:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.300+0000 7fbf53fff700 1 -- 172.21.15.97:0/4105487412 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fbf4c003a00 con 0x7fbf5c145760 2023-11-24T06:12:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.300+0000 7fbf53fff700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf3c06b500 0x7fbf3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.300+0000 7fbf53fff700 1 -- 172.21.15.97:0/4105487412 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fbf4c021070 con 0x7fbf5c145760 2023-11-24T06:12:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.300+0000 7fbf5affd700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf3c06b500 0x7fbf3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.300+0000 7fbf5affd700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf3c06b500 0x7fbf3c06d9c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009cf0 tx=0x7fbf44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.304+0000 7fbf53fff700 1 -- 172.21.15.97:0/4105487412 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf4c056140 con 0x7fbf5c145760 2023-11-24T06:12:32.494 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.492+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf5c06da00 con 0x7fbf3c06b500 2023-11-24T06:12:32.497 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.494+0000 7fbf53fff700 1 -- 172.21.15.97:0/4105487412 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50729 (secure 0 0 0) 0x7fbf44009e30 con 0x7fbf3c06b500 2023-11-24T06:12:32.498 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.496+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf3c06b500 msgr2=0x7fbf3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.496+0000 7fbf6189c700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf3c06b500 0x7fbf3c06d9c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009cf0 tx=0x7fbf44008040 comp rx=0 tx=0).stop 2023-11-24T06:12:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.496+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf5c145760 msgr2=0x7fbf5c06e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.496+0000 7fbf6189c700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf5c145760 0x7fbf5c06e600 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0090b0 tx=0x7fbf4c004750 comp rx=0 tx=0).stop 2023-11-24T06:12:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.497+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 shutdown_connections 2023-11-24T06:12:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.497+0000 7fbf6189c700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf5c15fde0 0x7fbf5c13fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.497+0000 7fbf6189c700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fbf3c06b500 0x7fbf3c06d9c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.497+0000 7fbf6189c700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fbf5c06df20 0x7fbf5c15b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.497+0000 7fbf6189c700 1 --2- 172.21.15.97:0/4105487412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf5c145760 0x7fbf5c06e600 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.497+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 >> 172.21.15.97:0/4105487412 conn(0x7fbf5c0b1740 msgr2=0x7fbf5c071350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.497+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 shutdown_connections 2023-11-24T06:12:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.497+0000 7fbf6189c700 1 -- 172.21.15.97:0/4105487412 wait complete. 2023-11-24T06:12:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:12:32.511 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2023-11-24T06:12:32.366608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225696,"kb_used_data":9536,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749024,"statfs":{"total":773094113280,"available":772863000576,"internally_reserved":0,"allocated":9764864,"data_stored":4872380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28350,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-60,"num_write_kb":-48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001030"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:12:27.102152+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:12:27.102152+0000","last_peered":"2023-11-24T06:12:27.102152+0000","last_clean":"2023-11-24T06:12:27.102152+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:12:27.102152+0000","last_undegraded":"2023-11-24T06:12:27.102152+0000","last_fullsized":"2023-11-24T06:12:27.102152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343204,"statfs":{"total":96636764160,"available":96607440896,"internally_reserved":0,"allocated":1654784,"data_stored":1044806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.755,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.82299999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.746}]},{"osd":5,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.88600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1069056,"data_stored":455062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.71299999999999997,"15min":0.79300000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.75,"15min":0.86199999999999999},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68999999999999995,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.79700000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":1.371,"15min":1.371},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69099999999999995,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.76400000000000001,"15min":0.85299999999999998},"min":{"1min":0.38800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.629,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69899999999999995,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":1.407,"15min":1.407},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.622}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.439,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80600000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1069056,"data_stored":454562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33900000000000002,"15min":0.34499999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"min":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.379},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.245}]},{"osd":4,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.374},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.624},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42699999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80600000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.44600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343200,"statfs":{"total":96636764160,"available":96607436800,"internally_reserved":0,"allocated":1658880,"data_stored":1044905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.624,"15min":0.75800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47899999999999998,"5min":12.378,"15min":12.378},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.45900000000000002,"15min":0.502},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.496,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.46700000000000003,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.48499999999999999,"15min":0.53000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.46300000000000002,"15min":0.50600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52000000000000002,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.499},{"interface":"front","average":{"1min":0.375,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56000000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.96099999999999997,"15min":1.125},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73299999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68300000000000005,"15min":0.72699999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82199999999999995,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69299999999999995,"15min":0.73299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.96799999999999997,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":15.356,"15min":15.356},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71499999999999997,"15min":0.76100000000000001},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":4.726,"15min":4.726},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.71999999999999997,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.997,"15min":1.165},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75,"5min":15.388,"15min":15.388},"last":0.73499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":454950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32700000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.34399999999999997,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.372}]},{"osd":3,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54200000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.173,"15min":1.173},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.36399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.83599999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.39700000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.65900000000000003,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49099999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.442}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":481785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.32700000000000001,"15min":0.33800000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.39200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.34999999999999998,"15min":0.36699999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.315}]},{"osd":1,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34499999999999997,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45700000000000002,"5min":1.032,"15min":1.032},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.252,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44400000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.48599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.622,"15min":0.65200000000000002},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64800000000000002,"5min":1.728,"15min":1.728},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.67100000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.495},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.626,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.504}]},{"osd":6,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69199999999999995,"5min":1.304,"15min":1.304},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.044,"5min":1.222,"15min":1.222},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69199999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":454503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.437,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.45500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.745,"15min":0.745},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":481807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.63200000000000001,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":3.335,"15min":3.335},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.622,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66900000000000004,"15min":0.72999999999999998},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.627},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71299999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.0419999999999998,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.76300000000000001},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":1.536,"15min":1.536},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81999999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93700000000000006,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.754},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90700000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:32.513 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:32.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.632+0000 7f351cb91700 1 Processor -- start 2023-11-24T06:12:32.635 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.632+0000 7f351cb91700 1 -- start start 2023-11-24T06:12:32.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.633+0000 7f351cb91700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.638 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.633+0000 7f351cb91700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35180ae430 con 0x7f35180af4d0 2023-11-24T06:12:32.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.633+0000 7f351cb91700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35180ae710 con 0x7f351806c710 2023-11-24T06:12:32.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.633+0000 7f351cb91700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35180af0b0 con 0x7f351815e080 2023-11-24T06:12:32.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.633+0000 7f3515d9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f35180af4d0 0x7f351806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44242/0 (socket says 172.21.15.97:44242) 2023-11-24T06:12:32.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.633+0000 7f3515d9b700 1 -- 172.21.15.97:0/2566783349 learned_addr learned my addr 172.21.15.97:0/2566783349 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:32.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.633+0000 7f351659c700 1 --2- 172.21.15.97:0/2566783349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351559a700 1 -- 172.21.15.97:0/2566783349 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050877843 0 0) 0x7f35180ae430 con 0x7f35180af4d0 2023-11-24T06:12:32.639 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351559a700 1 -- 172.21.15.97:0/2566783349 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35000035b0 con 0x7f35180af4d0 2023-11-24T06:12:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351559a700 1 -- 172.21.15.97:0/2566783349 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475080282 0 0) 0x7f35180ae710 con 0x7f351806c710 2023-11-24T06:12:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351559a700 1 -- 172.21.15.97:0/2566783349 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35180ae430 con 0x7f351806c710 2023-11-24T06:12:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351559a700 1 -- 172.21.15.97:0/2566783349 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2429026957 0 0) 0x7f35000035b0 con 0x7f35180af4d0 2023-11-24T06:12:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351659c700 1 -- 172.21.15.97:0/2566783349 >> v1:172.21.15.183:6789/0 conn(0x7f351806c710 legacy=0x7f351806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351659c700 1 -- 172.21.15.97:0/2566783349 >> v1:172.21.15.97:6789/0 conn(0x7f35180af4d0 legacy=0x7f351806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351659c700 1 -- 172.21.15.97:0/2566783349 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180ae9f0 con 0x7f351815e080 2023-11-24T06:12:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351659c700 1 --2- 172.21.15.97:0/2566783349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b5050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f350c009870 tx=0x7f350c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d686d5fd0280d28 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.640 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351559a700 1 -- 172.21.15.97:0/2566783349 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508001670 con 0x7f351815e080 2023-11-24T06:12:32.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351559a700 1 -- 172.21.15.97:0/2566783349 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c004030 con 0x7f351815e080 2023-11-24T06:12:32.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.634+0000 7f351559a700 1 -- 172.21.15.97:0/2566783349 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c0043c0 con 0x7f351815e080 2023-11-24T06:12:32.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.635+0000 7f351cb91700 1 -- 172.21.15.97:0/2566783349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 msgr2=0x7f35180b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.635+0000 7f351cb91700 1 --2- 172.21.15.97:0/2566783349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b5050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f350c009870 tx=0x7f350c009b80 comp rx=0 tx=0).stop 2023-11-24T06:12:32.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.635+0000 7f351cb91700 1 -- 172.21.15.97:0/2566783349 shutdown_connections 2023-11-24T06:12:32.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.635+0000 7f351cb91700 1 --2- 172.21.15.97:0/2566783349 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b5050 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.635+0000 7f351cb91700 1 -- 172.21.15.97:0/2566783349 >> 172.21.15.97:0/2566783349 conn(0x7f3518071b90 msgr2=0x7f3518071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:32.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.635+0000 7f351cb91700 1 -- 172.21.15.97:0/2566783349 shutdown_connections 2023-11-24T06:12:32.642 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.635+0000 7f351cb91700 1 -- 172.21.15.97:0/2566783349 wait complete. 2023-11-24T06:12:32.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.636+0000 7f351cb91700 1 Processor -- start 2023-11-24T06:12:32.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.636+0000 7f351cb91700 1 -- start start 2023-11-24T06:12:32.644 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351cb91700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351806c710 0x7f35181537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351cb91700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35180af4d0 0x7f35181566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351cb91700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351cb91700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518158270 con 0x7f351806c710 2023-11-24T06:12:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351cb91700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518156bf0 con 0x7f35180af4d0 2023-11-24T06:12:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351cb91700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35181575f0 con 0x7f351815e080 2023-11-24T06:12:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351806c710 0x7f35181537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.645 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351806c710 0x7f35181537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51796/0 (socket says 172.21.15.97:51796) 2023-11-24T06:12:32.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 -- 172.21.15.97:0/1916327416 learned_addr learned my addr 172.21.15.97:0/1916327416 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:32.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f3515d9b700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35180af4d0 0x7f35181566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f3516d9d700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 -- 172.21.15.97:0/1916327416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 msgr2=0x7f35180b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 -- 172.21.15.97:0/1916327416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35180af4d0 msgr2=0x7f35181566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.646 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35180af4d0 0x7f35181566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 -- 172.21.15.97:0/1916327416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c0094a0 con 0x7f351806c710 2023-11-24T06:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.637+0000 7f351659c700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351806c710 0x7f35181537b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f350c000c00 tx=0x7f350c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.638+0000 7f3506ffd700 1 -- 172.21.15.97:0/1916327416 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c003cd0 con 0x7f351806c710 2023-11-24T06:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.638+0000 7f3506ffd700 1 -- 172.21.15.97:0/1916327416 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c01f070 con 0x7f351806c710 2023-11-24T06:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.638+0000 7f3506ffd700 1 -- 172.21.15.97:0/1916327416 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c011870 con 0x7f351806c710 2023-11-24T06:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.638+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35181578d0 con 0x7f351806c710 2023-11-24T06:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.638+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3518157e90 con 0x7f351806c710 2023-11-24T06:12:32.647 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.639+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35180753f0 con 0x7f351806c710 2023-11-24T06:12:32.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.639+0000 7f3506ffd700 1 -- 172.21.15.97:0/1916327416 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f350c021030 con 0x7f351806c710 2023-11-24T06:12:32.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.639+0000 7f3506ffd700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f34f806b500 0x7f34f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.640+0000 7f3506ffd700 1 -- 172.21.15.97:0/1916327416 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f350c08b2d0 con 0x7f351806c710 2023-11-24T06:12:32.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.640+0000 7f3515d9b700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f34f806b500 0x7f34f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.640+0000 7f3515d9b700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f34f806b500 0x7f34f806d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f35180d0690 tx=0x7f3508001e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.644+0000 7f3506ffd700 1 -- 172.21.15.97:0/1916327416 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f350c055e50 con 0x7f351806c710 2023-11-24T06:12:32.833 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.830+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35180b05a0 con 0x7f34f806b500 2023-11-24T06:12:32.835 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.832+0000 7f3506ffd700 1 -- 172.21.15.97:0/1916327416 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50729 (secure 0 0 0) 0x7f350800f690 con 0x7f34f806b500 2023-11-24T06:12:32.836 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:32.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f34f806b500 msgr2=0x7f34f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f34f806b500 0x7f34f806d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f35180d0690 tx=0x7f3508001e00 comp rx=0 tx=0).stop 2023-11-24T06:12:32.837 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351806c710 msgr2=0x7f35181537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351806c710 0x7f35181537b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f350c000c00 tx=0x7f350c004750 comp rx=0 tx=0).stop 2023-11-24T06:12:32.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 shutdown_connections 2023-11-24T06:12:32.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f34f806b500 0x7f34f806d9c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f351815e080 0x7f35180b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.838 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f351806c710 0x7f35181537b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 --2- 172.21.15.97:0/1916327416 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35180af4d0 0x7f35181566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 >> 172.21.15.97:0/1916327416 conn(0x7f3518071b90 msgr2=0x7f35180b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 shutdown_connections 2023-11-24T06:12:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.834+0000 7f351cb91700 1 -- 172.21.15.97:0/1916327416 wait complete. 2023-11-24T06:12:32.839 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:12:32.849 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2023-11-24T06:12:32.366608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225696,"kb_used_data":9536,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749024,"statfs":{"total":773094113280,"available":772863000576,"internally_reserved":0,"allocated":9764864,"data_stored":4872380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28350,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-60,"num_write_kb":-48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001030"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:12:27.102152+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:12:27.102152+0000","last_peered":"2023-11-24T06:12:27.102152+0000","last_clean":"2023-11-24T06:12:27.102152+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:12:27.102152+0000","last_undegraded":"2023-11-24T06:12:27.102152+0000","last_fullsized":"2023-11-24T06:12:27.102152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343204,"statfs":{"total":96636764160,"available":96607440896,"internally_reserved":0,"allocated":1654784,"data_stored":1044806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.755,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.82299999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.746}]},{"osd":5,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.88600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1069056,"data_stored":455062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.71299999999999997,"15min":0.79300000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.75,"15min":0.86199999999999999},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68999999999999995,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.79700000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":1.371,"15min":1.371},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69099999999999995,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.76400000000000001,"15min":0.85299999999999998},"min":{"1min":0.38800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.629,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69899999999999995,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":1.407,"15min":1.407},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.622}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.439,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80600000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1069056,"data_stored":454562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33900000000000002,"15min":0.34499999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"min":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.379},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.245}]},{"osd":4,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.374},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.624},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42699999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80600000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.44600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343200,"statfs":{"total":96636764160,"available":96607436800,"internally_reserved":0,"allocated":1658880,"data_stored":1044905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.624,"15min":0.75800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47899999999999998,"5min":12.378,"15min":12.378},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.45900000000000002,"15min":0.502},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.496,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.46700000000000003,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.48499999999999999,"15min":0.53000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.46300000000000002,"15min":0.50600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52000000000000002,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.499},{"interface":"front","average":{"1min":0.375,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56000000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.96099999999999997,"15min":1.125},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73299999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68300000000000005,"15min":0.72699999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82199999999999995,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69299999999999995,"15min":0.73299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.96799999999999997,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":15.356,"15min":15.356},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71499999999999997,"15min":0.76100000000000001},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":4.726,"15min":4.726},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.71999999999999997,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.997,"15min":1.165},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75,"5min":15.388,"15min":15.388},"last":0.73499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":454950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32700000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.34399999999999997,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.372}]},{"osd":3,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54200000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.173,"15min":1.173},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.36399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.83599999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.39700000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.65900000000000003,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49099999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.442}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":481785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.32700000000000001,"15min":0.33800000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.39200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.34999999999999998,"15min":0.36699999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.315}]},{"osd":1,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34499999999999997,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45700000000000002,"5min":1.032,"15min":1.032},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.252,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44400000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.48599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.622,"15min":0.65200000000000002},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64800000000000002,"5min":1.728,"15min":1.728},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.67100000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.495},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.626,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.504}]},{"osd":6,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69199999999999995,"5min":1.304,"15min":1.304},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.044,"5min":1.222,"15min":1.222},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69199999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":454503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.437,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.45500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.745,"15min":0.745},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":481807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.63200000000000001,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":3.335,"15min":3.335},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.622,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66900000000000004,"15min":0.72999999999999998},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.627},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71299999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.0419999999999998,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.76300000000000001},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":1.536,"15min":1.536},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81999999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93700000000000006,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.754},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90700000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:32.851 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:12:32.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:32.970 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.968+0000 7f238d156700 1 Processor -- start 2023-11-24T06:12:32.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.968+0000 7f238d156700 1 -- start start 2023-11-24T06:12:32.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.968+0000 7f238d156700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f238815ae20 0x7f23880b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.968+0000 7f238d156700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815b200 0x7f23880b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.968+0000 7f238d156700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880cfd40 con 0x7f23880cf780 2023-11-24T06:12:32.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.968+0000 7f238d156700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880d0020 con 0x7f238815ae20 2023-11-24T06:12:32.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.968+0000 7f238d156700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23880affa0 con 0x7f238815b200 2023-11-24T06:12:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2386d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f238815ae20 0x7f23880b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2386d9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f238815ae20 0x7f23880b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51804/0 (socket says 172.21.15.97:51804) 2023-11-24T06:12:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2386d9d700 1 -- 172.21.15.97:0/2708364046 learned_addr learned my addr 172.21.15.97:0/2708364046 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f238659c700 1 --2- 172.21.15.97:0/2708364046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815b200 0x7f23880b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2386d9d700 1 -- 172.21.15.97:0/2708364046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815b200 msgr2=0x7f23880b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2386d9d700 1 --2- 172.21.15.97:0/2708364046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815b200 0x7f23880b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2386d9d700 1 -- 172.21.15.97:0/2708364046 >> v1:172.21.15.183:6789/0 conn(0x7f23880cf780 legacy=0x7f23880b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2386d9d700 1 -- 172.21.15.97:0/2708364046 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880b0280 con 0x7f238815ae20 2023-11-24T06:12:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f238659c700 1 --2- 172.21.15.97:0/2708364046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815b200 0x7f23880b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2386d9d700 1 --2- 172.21.15.97:0/2708364046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238815ae20 0x7f23880b29a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f23780098f0 tx=0x7f2378009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd015f8c964e38f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2385d9b700 1 -- 172.21.15.97:0/2708364046 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378004230 con 0x7f238815ae20 2023-11-24T06:12:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2385d9b700 1 -- 172.21.15.97:0/2708364046 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2378004390 con 0x7f238815ae20 2023-11-24T06:12:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.969+0000 7f2385d9b700 1 -- 172.21.15.97:0/2708364046 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23780036a0 con 0x7f238815ae20 2023-11-24T06:12:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.970+0000 7f238d156700 1 -- 172.21.15.97:0/2708364046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238815ae20 msgr2=0x7f23880b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.970+0000 7f238d156700 1 --2- 172.21.15.97:0/2708364046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238815ae20 0x7f23880b29a0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f23780098f0 tx=0x7f2378009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:32.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.970+0000 7f238d156700 1 -- 172.21.15.97:0/2708364046 shutdown_connections 2023-11-24T06:12:32.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.970+0000 7f238d156700 1 --2- 172.21.15.97:0/2708364046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f238815ae20 0x7f23880b29a0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.970+0000 7f238d156700 1 --2- 172.21.15.97:0/2708364046 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815b200 0x7f23880b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.970+0000 7f238d156700 1 -- 172.21.15.97:0/2708364046 >> 172.21.15.97:0/2708364046 conn(0x7f23880b57b0 msgr2=0x7f23880b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:32.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.970+0000 7f238d156700 1 -- 172.21.15.97:0/2708364046 shutdown_connections 2023-11-24T06:12:32.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.970+0000 7f238d156700 1 -- 172.21.15.97:0/2708364046 wait complete. 2023-11-24T06:12:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.971+0000 7f238d156700 1 Processor -- start 2023-11-24T06:12:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.971+0000 7f238d156700 1 -- start start 2023-11-24T06:12:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f238d156700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23880cf780 0x7f23880ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f238d156700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815ae20 0x7f2388148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f238d156700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238815b200 0x7f2388097040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.979 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f238d156700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23881f8b00 con 0x7f23880cf780 2023-11-24T06:12:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f238d156700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23880aef00 con 0x7f238815b200 2023-11-24T06:12:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f238d156700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23881f7d40 con 0x7f238815ae20 2023-11-24T06:12:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f2386d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23880cf780 0x7f23880ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f2386d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23880cf780 0x7f23880ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51810/0 (socket says 172.21.15.97:51810) 2023-11-24T06:12:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f2386d9d700 1 -- 172.21.15.97:0/189114183 learned_addr learned my addr 172.21.15.97:0/189114183 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f238659c700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815ae20 0x7f2388148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f238759e700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238815b200 0x7f2388097040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.980 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f2386d9d700 1 -- 172.21.15.97:0/189114183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815ae20 msgr2=0x7f2388148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f2386d9d700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815ae20 0x7f2388148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f2386d9d700 1 -- 172.21.15.97:0/189114183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238815b200 msgr2=0x7f2388097040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:32.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f2386d9d700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238815b200 0x7f2388097040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:32.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.972+0000 7f2386d9d700 1 -- 172.21.15.97:0/189114183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2378009550 con 0x7f23880cf780 2023-11-24T06:12:32.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.973+0000 7f2386d9d700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23880cf780 0x7f23880ae620 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2378009130 tx=0x7f2378004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.973+0000 7f236f7fe700 1 -- 172.21.15.97:0/189114183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378003cd0 con 0x7f23880cf780 2023-11-24T06:12:32.981 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.973+0000 7f236f7fe700 1 -- 172.21.15.97:0/189114183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237801f070 con 0x7f23880cf780 2023-11-24T06:12:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.973+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23881f8020 con 0x7f23880cf780 2023-11-24T06:12:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.973+0000 7f236f7fe700 1 -- 172.21.15.97:0/189114183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237800b8a0 con 0x7f23880cf780 2023-11-24T06:12:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.973+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23881f85e0 con 0x7f23880cf780 2023-11-24T06:12:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.974+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2388073620 con 0x7f23880cf780 2023-11-24T06:12:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.974+0000 7f236f7fe700 1 -- 172.21.15.97:0/189114183 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2378021030 con 0x7f23880cf780 2023-11-24T06:12:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.974+0000 7f236f7fe700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f237406b500 0x7f237406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.975+0000 7f236f7fe700 1 -- 172.21.15.97:0/189114183 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f237808b2e0 con 0x7f23880cf780 2023-11-24T06:12:32.982 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.975+0000 7f238659c700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f237406b500 0x7f237406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.975+0000 7f238659c700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f237406b500 0x7f237406d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2388062ab0 tx=0x7f237c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:32.983 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:32.979+0000 7f236f7fe700 1 -- 172.21.15.97:0/189114183 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2378055ec0 con 0x7f23880cf780 2023-11-24T06:12:33.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.164+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23880b37d0 con 0x7f237406b500 2023-11-24T06:12:33.169 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.166+0000 7f236f7fe700 1 -- 172.21.15.97:0/189114183 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50729 (secure 0 0 0) 0x7f237c00c040 con 0x7f237406b500 2023-11-24T06:12:33.170 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:33.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f237406b500 msgr2=0x7f237406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f237406b500 0x7f237406d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2388062ab0 tx=0x7f237c008040 comp rx=0 tx=0).stop 2023-11-24T06:12:33.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23880cf780 msgr2=0x7f23880ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23880cf780 0x7f23880ae620 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2378009130 tx=0x7f2378004750 comp rx=0 tx=0).stop 2023-11-24T06:12:33.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 shutdown_connections 2023-11-24T06:12:33.172 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f238815b200 0x7f2388097040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23880cf780 0x7f23880ae620 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f237406b500 0x7f237406d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 --2- 172.21.15.97:0/189114183 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f238815ae20 0x7f2388148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 >> 172.21.15.97:0/189114183 conn(0x7f23880b57b0 msgr2=0x7f2388150c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:33.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 shutdown_connections 2023-11-24T06:12:33.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.169+0000 7f238d156700 1 -- 172.21.15.97:0/189114183 wait complete. 2023-11-24T06:12:33.174 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:12:33.183 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2023-11-24T06:12:32.366608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225696,"kb_used_data":9536,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749024,"statfs":{"total":773094113280,"available":772863000576,"internally_reserved":0,"allocated":9764864,"data_stored":4872380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28350,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-60,"num_write_kb":-48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001030"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T06:12:27.102152+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:12:27.102152+0000","last_peered":"2023-11-24T06:12:27.102152+0000","last_clean":"2023-11-24T06:12:27.102152+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:12:27.102152+0000","last_undegraded":"2023-11-24T06:12:27.102152+0000","last_fullsized":"2023-11-24T06:12:27.102152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343204,"statfs":{"total":96636764160,"available":96607440896,"internally_reserved":0,"allocated":1654784,"data_stored":1044806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.755,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.82299999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.746}]},{"osd":5,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.88600000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1069056,"data_stored":455062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.71299999999999997,"15min":0.79300000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.75,"15min":0.86199999999999999},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68999999999999995,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.79700000000000004}]},{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":1.371,"15min":1.371},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69099999999999995,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.76400000000000001,"15min":0.85299999999999998},"min":{"1min":0.38800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.629,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69899999999999995,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":1.407,"15min":1.407},"last":0.65600000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.622}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.439,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80600000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28064,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343776,"statfs":{"total":96636764160,"available":96608026624,"internally_reserved":0,"allocated":1069056,"data_stored":454562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33900000000000002,"15min":0.34499999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"min":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.379},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.245}]},{"osd":4,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.374},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.624},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42699999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80600000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.44600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28640,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343200,"statfs":{"total":96636764160,"available":96607436800,"internally_reserved":0,"allocated":1658880,"data_stored":1044905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.624,"15min":0.75800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47899999999999998,"5min":12.378,"15min":12.378},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.45900000000000002,"15min":0.502},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.496,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.46700000000000003,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.48499999999999999,"15min":0.53000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.45200000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.46300000000000002,"15min":0.50600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52000000000000002,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.499},{"interface":"front","average":{"1min":0.375,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56000000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.96099999999999997,"15min":1.125},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73299999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68300000000000005,"15min":0.72699999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82199999999999995,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69299999999999995,"15min":0.73299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.96799999999999997,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":15.356,"15min":15.356},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71499999999999997,"15min":0.76100000000000001},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":4.726,"15min":4.726},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.71999999999999997,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.997,"15min":1.165},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75,"5min":15.388,"15min":15.388},"last":0.73499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":454950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32700000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.34399999999999997,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.372}]},{"osd":3,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54200000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.173,"15min":1.173},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.36399999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.83599999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.39700000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.65900000000000003,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49099999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.442}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":481785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.32700000000000001,"15min":0.33800000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.39200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.34999999999999998,"15min":0.36699999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.315}]},{"osd":1,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34499999999999997,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45700000000000002,"5min":1.032,"15min":1.032},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.252,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44400000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36499999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.48599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.622,"15min":0.65200000000000002},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64800000000000002,"5min":1.728,"15min":1.728},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.67100000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.495},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.626,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.504}]},{"osd":6,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69199999999999995,"5min":1.304,"15min":1.304},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.044,"5min":1.222,"15min":1.222},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69199999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.55200000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":454503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.437,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.45500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.745,"15min":0.745},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.65500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28092,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343748,"statfs":{"total":96636764160,"available":96607997952,"internally_reserved":0,"allocated":1097728,"data_stored":481807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.63200000000000001,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":3.335,"15min":3.335},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.622,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66900000000000004,"15min":0.72999999999999998},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.627},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71299999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.0419999999999998,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.76300000000000001},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":1.536,"15min":1.536},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81999999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93700000000000006,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.754},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90700000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:33.185 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:12:33.303 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.301+0000 7fd60fa42700 1 Processor -- start 2023-11-24T06:12:33.304 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.301+0000 7fd60fa42700 1 -- start start 2023-11-24T06:12:33.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60fa42700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fd60815e080 0x7fd6080b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60fa42700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6080af4d0 0x7fd6080afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60fa42700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6080ae430 con 0x7fd60806c000 2023-11-24T06:12:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60fa42700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080af0b0 con 0x7fd60815e080 2023-11-24T06:12:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60fa42700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b05a0 con 0x7fd6080af4d0 2023-11-24T06:12:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60dfdf700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd60806c000 0x7fd608073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44274/0 (socket says 172.21.15.97:44274) 2023-11-24T06:12:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60dfdf700 1 -- 172.21.15.97:0/2218654327 learned_addr learned my addr 172.21.15.97:0/2218654327 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60cfdd700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6080af4d0 0x7fd6080afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd60d7de700 1 --2- 172.21.15.97:0/2218654327 >> v2:172.21.15.183:3300/0 conn(0x7fd60815e080 0x7fd6080b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268019957 0 0) 0x7fd6080ae430 con 0x7fd60806c000 2023-11-24T06:12:33.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5f40035b0 con 0x7fd60806c000 2023-11-24T06:12:33.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3571455094 0 0) 0x7fd5f40035b0 con 0x7fd60806c000 2023-11-24T06:12:33.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6080af4d0 msgr2=0x7fd6080afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:33.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6080af4d0 0x7fd6080afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 >> v2:172.21.15.183:3300/0 conn(0x7fd60815e080 msgr2=0x7fd6080b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:33.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 --2- 172.21.15.97:0/2218654327 >> v2:172.21.15.183:3300/0 conn(0x7fd60815e080 0x7fd6080b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080ae710 con 0x7fd60806c000 2023-11-24T06:12:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.302+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7fd6000019f0 con 0x7fd60806c000 2023-11-24T06:12:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 >> v1:172.21.15.97:6789/0 conn(0x7fd60806c000 legacy=0x7fd608073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5f4003a30 0x7fd5f4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5f4004420 0x7fd5f40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd5f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080af0b0 con 0x7fd5f4004f60 2023-11-24T06:12:33.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b05a0 con 0x7fd5f4004420 2023-11-24T06:12:33.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f400bf50 con 0x7fd5f4003a30 2023-11-24T06:12:33.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60cfdd700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd6080af4d0 0x7fd6080afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:33.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60d7de700 1 --2- 172.21.15.97:0/2218654327 >> v2:172.21.15.183:3300/0 conn(0x7fd60815e080 0x7fd6080b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:33.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60dfdf700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd5f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60d7de700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5f4003a30 0x7fd5f4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60cfdd700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5f4004420 0x7fd5f40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60dfdf700 1 -- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5f4003a30 msgr2=0x7fd5f4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60dfdf700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd5f4003a30 0x7fd5f4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60dfdf700 1 -- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5f4004420 msgr2=0x7fd5f40048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60dfdf700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5f4004420 0x7fd5f40048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60dfdf700 1 -- 172.21.15.97:0/2218654327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f40035b0 con 0x7fd5f4004f60 2023-11-24T06:12:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd60dfdf700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd5f400ba10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd60000a820 tx=0x7fd5f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b18dbcdcdaab377 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60000a2f0 con 0x7fd5f4004f60 2023-11-24T06:12:33.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd600012620 con 0x7fd5f4004f60 2023-11-24T06:12:33.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.303+0000 7fd5fffff700 1 -- 172.21.15.97:0/2218654327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000129d0 con 0x7fd5f4004f60 2023-11-24T06:12:33.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.304+0000 7fd60fa42700 1 -- 172.21.15.97:0/2218654327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 msgr2=0x7fd5f400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.304+0000 7fd60fa42700 1 --2- 172.21.15.97:0/2218654327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd5f400ba10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd60000a820 tx=0x7fd5f4005880 comp rx=0 tx=0).stop 2023-11-24T06:12:33.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.304+0000 7fd60dfdf700 1 -- 172.21.15.97:0/2218654327 reap_dead start 2023-11-24T06:12:33.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.304+0000 7fd60fa42700 1 -- 172.21.15.97:0/2218654327 shutdown_connections 2023-11-24T06:12:33.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.304+0000 7fd60fa42700 1 -- 172.21.15.97:0/2218654327 >> 172.21.15.97:0/2218654327 conn(0x7fd608071b90 msgr2=0x7fd608071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:33.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.304+0000 7fd60fa42700 1 -- 172.21.15.97:0/2218654327 shutdown_connections 2023-11-24T06:12:33.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.304+0000 7fd60fa42700 1 -- 172.21.15.97:0/2218654327 wait complete. 2023-11-24T06:12:33.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.305+0000 7fd60fa42700 1 Processor -- start 2023-11-24T06:12:33.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.305+0000 7fd60fa42700 1 -- start start 2023-11-24T06:12:33.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60fa42700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd6081529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60fa42700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd60815ee70 0x7fd6081e9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60fa42700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd60815f2f0 0x7fd6081e9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60fa42700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080cf060 con 0x7fd5f4004f60 2023-11-24T06:12:33.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60fa42700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080ce450 con 0x7fd60815ee70 2023-11-24T06:12:33.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60fa42700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080ce730 con 0x7fd60815f2f0 2023-11-24T06:12:33.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd6081529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd6081529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51830/0 (socket says 172.21.15.97:51830) 2023-11-24T06:12:33.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 -- 172.21.15.97:0/174008383 learned_addr learned my addr 172.21.15.97:0/174008383 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:33.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60dfdf700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd60815f2f0 0x7fd6081e9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60cfdd700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd60815ee70 0x7fd6081e9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 -- 172.21.15.97:0/174008383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd60815f2f0 msgr2=0x7fd6081e9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd60815f2f0 0x7fd6081e9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 -- 172.21.15.97:0/174008383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd60815ee70 msgr2=0x7fd6081e9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd60815ee70 0x7fd6081e9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 -- 172.21.15.97:0/174008383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080cea10 con 0x7fd5f4004f60 2023-11-24T06:12:33.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.306+0000 7fd60d7de700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd6081529c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd604010a50 tx=0x7fd604010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:33.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.307+0000 7fd5fdffb700 1 -- 172.21.15.97:0/174008383 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604010ee0 con 0x7fd5f4004f60 2023-11-24T06:12:33.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.307+0000 7fd5fdffb700 1 -- 172.21.15.97:0/174008383 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60400e8d0 con 0x7fd5f4004f60 2023-11-24T06:12:33.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.307+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6080cecf0 con 0x7fd5f4004f60 2023-11-24T06:12:33.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.307+0000 7fd5fdffb700 1 -- 172.21.15.97:0/174008383 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60400eca0 con 0x7fd5f4004f60 2023-11-24T06:12:33.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.307+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd608153f60 con 0x7fd5f4004f60 2023-11-24T06:12:33.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.308+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6080753f0 con 0x7fd5f4004f60 2023-11-24T06:12:33.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.308+0000 7fd5fdffb700 1 -- 172.21.15.97:0/174008383 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd604016460 con 0x7fd5f4004f60 2023-11-24T06:12:33.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.308+0000 7fd5fdffb700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd5ec06b500 0x7fd5ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:33.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.308+0000 7fd5fdffb700 1 -- 172.21.15.97:0/174008383 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fd60408a2f0 con 0x7fd5f4004f60 2023-11-24T06:12:33.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.309+0000 7fd60cfdd700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd5ec06b500 0x7fd5ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:33.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.309+0000 7fd60cfdd700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd5ec06b500 0x7fd5ec06d9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd60806e4e0 tx=0x7fd5f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:33.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.313+0000 7fd5fdffb700 1 -- 172.21.15.97:0/174008383 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd604054d40 con 0x7fd5f4004f60 2023-11-24T06:12:33.508 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.506+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd608067f60 con 0x7fd5f4004f60 2023-11-24T06:12:33.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.507+0000 7fd5fdffb700 1 -- 172.21.15.97:0/174008383 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+13044 (secure 0 0 0) 0x7fd604058360 con 0x7fd5f4004f60 2023-11-24T06:12:33.509 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:33.510 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"d129d9f1-ad0a-431b-85e6-84e6cb3d2cc9","created":"2023-11-24T06:08:32.727729+0000","modified":"2023-11-24T06:12:30.100078+0000","last_up_change":"2023-11-24T06:08:44.304306+0000","last_in_change":"2023-11-24T06:08:35.171857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:08:44.352292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2e42b1a-b2f7-4bc5-9ae1-f8339cecfb77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6801","nonce":3900364081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6803","nonce":3900364081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6807","nonce":3900364081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3900364081},{"type":"v1","addr":"172.21.15.97:6805","nonce":3900364081}]},"public_addr":"172.21.15.97:6801/3900364081","cluster_addr":"172.21.15.97:6803/3900364081","heartbeat_back_addr":"172.21.15.97:6807/3900364081","heartbeat_front_addr":"172.21.15.97:6805/3900364081","state":["exists","up"]},{"osd":1,"uuid":"377f83d5-5100-48d5-8a3d-a78fcdd2175d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6809","nonce":3026562872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6811","nonce":3026562872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6815","nonce":3026562872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3026562872},{"type":"v1","addr":"172.21.15.97:6813","nonce":3026562872}]},"public_addr":"172.21.15.97:6809/3026562872","cluster_addr":"172.21.15.97:6811/3026562872","heartbeat_back_addr":"172.21.15.97:6815/3026562872","heartbeat_front_addr":"172.21.15.97:6813/3026562872","state":["exists","up"]},{"osd":2,"uuid":"06d31208-d031-4318-ab2a-50873f3d20c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6818","nonce":1638045649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6822","nonce":1638045649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6830","nonce":1638045649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1638045649},{"type":"v1","addr":"172.21.15.97:6826","nonce":1638045649}]},"public_addr":"172.21.15.97:6818/1638045649","cluster_addr":"172.21.15.97:6822/1638045649","heartbeat_back_addr":"172.21.15.97:6830/1638045649","heartbeat_front_addr":"172.21.15.97:6826/1638045649","state":["exists","up"]},{"osd":3,"uuid":"fa9af3f9-93c3-42e0-9d84-25d7a254ad66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6817","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6819","nonce":4004566867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6823","nonce":4004566867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6829","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6831","nonce":4004566867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":4004566867},{"type":"v1","addr":"172.21.15.97:6827","nonce":4004566867}]},"public_addr":"172.21.15.97:6819/4004566867","cluster_addr":"172.21.15.97:6823/4004566867","heartbeat_back_addr":"172.21.15.97:6831/4004566867","heartbeat_front_addr":"172.21.15.97:6827/4004566867","state":["exists","up"]},{"osd":4,"uuid":"2efd95ef-8292-4722-b41f-210931316c9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6801","nonce":1408518032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6803","nonce":1408518032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6807","nonce":1408518032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1408518032},{"type":"v1","addr":"172.21.15.183:6805","nonce":1408518032}]},"public_addr":"172.21.15.183:6801/1408518032","cluster_addr":"172.21.15.183:6803/1408518032","heartbeat_back_addr":"172.21.15.183:6807/1408518032","heartbeat_front_addr":"172.21.15.183:6805/1408518032","state":["exists","up"]},{"osd":5,"uuid":"1ff1124a-2cde-4e5a-9150-cea260ff05bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6819","nonce":4148254452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6824","nonce":4148254452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6831","nonce":4148254452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6827","nonce":4148254452},{"type":"v1","addr":"172.21.15.183:6829","nonce":4148254452}]},"public_addr":"172.21.15.183:6819/4148254452","cluster_addr":"172.21.15.183:6824/4148254452","heartbeat_back_addr":"172.21.15.183:6831/4148254452","heartbeat_front_addr":"172.21.15.183:6829/4148254452","state":["exists","up"]},{"osd":6,"uuid":"b97538d0-893b-4eff-9cca-db8fb6c7e962","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6818","nonce":3075593088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6821","nonce":3075593088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6828","nonce":3075593088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6823","nonce":3075593088},{"type":"v1","addr":"172.21.15.183:6825","nonce":3075593088}]},"public_addr":"172.21.15.183:6818/3075593088","cluster_addr":"172.21.15.183:6821/3075593088","heartbeat_back_addr":"172.21.15.183:6828/3075593088","heartbeat_front_addr":"172.21.15.183:6825/3075593088","state":["exists","up"]},{"osd":7,"uuid":"d9b1704b-8def-4fe7-b80d-910954755e82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6809","nonce":4128955131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6811","nonce":4128955131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6815","nonce":4128955131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":4128955131},{"type":"v1","addr":"172.21.15.183:6813","nonce":4128955131}]},"public_addr":"172.21.15.183:6809/4128955131","cluster_addr":"172.21.15.183:6811/4128955131","heartbeat_back_addr":"172.21.15.183:6815/4128955131","heartbeat_front_addr":"172.21.15.183:6813/4128955131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.381857+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.336715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.387036+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.408206+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.449691+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.426774+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.374810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:08:40.412554+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.183:6835/3868240464":"2023-11-25T06:12:27.090715+0000","172.21.15.183:6833/3868240464":"2023-11-25T06:12:27.090715+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:12:33.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd5ec06b500 msgr2=0x7fd5ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd5ec06b500 0x7fd5ec06d9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd60806e4e0 tx=0x7fd5f8008040 comp rx=0 tx=0).stop 2023-11-24T06:12:33.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 msgr2=0x7fd6081529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:33.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd6081529c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd604010a50 tx=0x7fd604010d60 comp rx=0 tx=0).stop 2023-11-24T06:12:33.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 shutdown_connections 2023-11-24T06:12:33.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fd5ec06b500 0x7fd5ec06d9c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fd60815f2f0 0x7fd6081e9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5f4004f60 0x7fd6081529c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 --2- 172.21.15.97:0/174008383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd60815ee70 0x7fd6081e9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 >> 172.21.15.97:0/174008383 conn(0x7fd608071b90 msgr2=0x7fd608073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 shutdown_connections 2023-11-24T06:12:33.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:33.509+0000 7fd60fa42700 1 -- 172.21.15.97:0/174008383 wait complete. 2023-11-24T06:12:34.524 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T06:12:34.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:12:34.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.652+0000 7f93a025c700 1 Processor -- start 2023-11-24T06:12:34.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.652+0000 7f93a025c700 1 -- start start 2023-11-24T06:12:34.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f93a025c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f939815e0c0 0x7f93980cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f93a025c700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93980d3a40 0x7f9398071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f93a025c700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93980d29e0 con 0x7f93980cf1c0 2023-11-24T06:12:34.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f93a025c700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93980b7ec0 con 0x7f93980d3a40 2023-11-24T06:12:34.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f93a025c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93980d2c80 con 0x7f939815e0c0 2023-11-24T06:12:34.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f939d7f7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93980d3a40 0x7f9398071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f939dff8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f939815e0c0 0x7f93980cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f939d7f7700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f93980d3a40 0x7f9398071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:46000/0 (socket says 172.21.15.97:46000) 2023-11-24T06:12:34.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.653+0000 7f939d7f7700 1 -- 172.21.15.97:0/370784989 learned_addr learned my addr 172.21.15.97:0/370784989 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:34.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939d7f7700 1 -- 172.21.15.97:0/370784989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f939815e0c0 msgr2=0x7f93980cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:34.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939d7f7700 1 --2- 172.21.15.97:0/370784989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f939815e0c0 0x7f93980cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939d7f7700 1 -- 172.21.15.97:0/370784989 >> v1:172.21.15.183:6789/0 conn(0x7f93980cf1c0 legacy=0x7f93980b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939d7f7700 1 -- 172.21.15.97:0/370784989 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980d2f60 con 0x7f93980d3a40 2023-11-24T06:12:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939dff8700 1 --2- 172.21.15.97:0/370784989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f939815e0c0 0x7f93980cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939d7f7700 1 --2- 172.21.15.97:0/370784989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93980d3a40 0x7f9398071610 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93940098f0 tx=0x7f9394009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd6108856596ed2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939cff6700 1 -- 172.21.15.97:0/370784989 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394011830 con 0x7f93980d3a40 2023-11-24T06:12:34.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939cff6700 1 -- 172.21.15.97:0/370784989 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9394011990 con 0x7f93980d3a40 2023-11-24T06:12:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.654+0000 7f939cff6700 1 -- 172.21.15.97:0/370784989 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394011d20 con 0x7f93980d3a40 2023-11-24T06:12:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.655+0000 7f93a025c700 1 -- 172.21.15.97:0/370784989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93980d3a40 msgr2=0x7f9398071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.655+0000 7f93a025c700 1 --2- 172.21.15.97:0/370784989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93980d3a40 0x7f9398071610 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f93940098f0 tx=0x7f9394009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:34.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.655+0000 7f93a025c700 1 -- 172.21.15.97:0/370784989 shutdown_connections 2023-11-24T06:12:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.655+0000 7f93a025c700 1 --2- 172.21.15.97:0/370784989 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f939815e0c0 0x7f93980cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.655+0000 7f93a025c700 1 --2- 172.21.15.97:0/370784989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93980d3a40 0x7f9398071610 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.655+0000 7f93a025c700 1 -- 172.21.15.97:0/370784989 >> 172.21.15.97:0/370784989 conn(0x7f939806b620 msgr2=0x7f93980d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:34.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.655+0000 7f93a025c700 1 -- 172.21.15.97:0/370784989 shutdown_connections 2023-11-24T06:12:34.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.655+0000 7f93a025c700 1 -- 172.21.15.97:0/370784989 wait complete. 2023-11-24T06:12:34.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.656+0000 7f93a025c700 1 Processor -- start 2023-11-24T06:12:34.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.656+0000 7f93a025c700 1 -- start start 2023-11-24T06:12:34.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f93a025c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93980cf1c0 0x7f93981f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f93a025c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93980d3a40 0x7f9398159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.667 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f93a025c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f939815e0c0 0x7f9398159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f93a025c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9398158f40 con 0x7f93980cf1c0 2023-11-24T06:12:34.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f93a025c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9398157f10 con 0x7f939815e0c0 2023-11-24T06:12:34.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f93a025c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9398158180 con 0x7f93980d3a40 2023-11-24T06:12:34.668 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93980cf1c0 0x7f93981f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93980cf1c0 0x7f93981f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51846/0 (socket says 172.21.15.97:51846) 2023-11-24T06:12:34.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 -- 172.21.15.97:0/596955261 learned_addr learned my addr 172.21.15.97:0/596955261 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:34.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939d7f7700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93980d3a40 0x7f9398159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.669 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939e7f9700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f939815e0c0 0x7f9398159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 -- 172.21.15.97:0/596955261 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93980d3a40 msgr2=0x7f9398159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f93980d3a40 0x7f9398159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 -- 172.21.15.97:0/596955261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f939815e0c0 msgr2=0x7f9398159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f939815e0c0 0x7f9398159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.670 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 -- 172.21.15.97:0/596955261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880094a0 con 0x7f93980cf1c0 2023-11-24T06:12:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.657+0000 7f939dff8700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93980cf1c0 0x7f93981f36b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f938800ebe0 tx=0x7f938800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.658+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388009760 con 0x7f93980cf1c0 2023-11-24T06:12:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.658+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938800c850 con 0x7f93980cf1c0 2023-11-24T06:12:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.658+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938800cbe0 con 0x7f93980cf1c0 2023-11-24T06:12:34.671 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.658+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9394009550 con 0x7f93980cf1c0 2023-11-24T06:12:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.658+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939815e620 con 0x7f93980cf1c0 2023-11-24T06:12:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.658+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f939815e4a0 con 0x7f93980cf1c0 2023-11-24T06:12:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.659+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f938800cd40 con 0x7f93980cf1c0 2023-11-24T06:12:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.659+0000 7f938e7fc700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f937c06b500 0x7f937c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.660+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f938808a1d0 con 0x7f93980cf1c0 2023-11-24T06:12:34.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.660+0000 7f939d7f7700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f937c06b500 0x7f937c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.660+0000 7f938e7fc700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7f937c071360 0x7f937c0737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.660+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 --> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f937c073e70 con 0x7f937c071360 2023-11-24T06:12:34.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.660+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938808a4b0 con 0x7f93980cf1c0 2023-11-24T06:12:34.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.660+0000 7f939e7f9700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7f937c071360 0x7f937c0737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.660+0000 7f939d7f7700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f937c06b500 0x7f937c06d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9394009c40 tx=0x7f9394008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.660+0000 7f939e7f9700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7f937c071360 0x7f937c0737a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.661+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 <== osd.0 v2:172.21.15.97:6800/3900364081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f937c073e70 con 0x7f937c071360 2023-11-24T06:12:34.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.675+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 --> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f93980d3520 con 0x7f937c071360 2023-11-24T06:12:34.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f938e7fc700 1 -- 172.21.15.97:0/596955261 <== osd.0 v2:172.21.15.97:6800/3900364081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9390001fc0 con 0x7f937c071360 2023-11-24T06:12:34.679 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:34.679 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:12:34.679 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:34.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7f937c071360 msgr2=0x7f937c0737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6800/3900364081,v1:172.21.15.97:6801/3900364081] conn(0x7f937c071360 0x7f937c0737a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f937c06b500 msgr2=0x7f937c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.680 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f937c06b500 0x7f937c06d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9394009c40 tx=0x7f9394008030 comp rx=0 tx=0).stop 2023-11-24T06:12:34.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93980cf1c0 msgr2=0x7f93981f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 --2- 172.21.15.97:0/596955261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93980cf1c0 0x7f93981f36b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f938800ebe0 tx=0x7f938800eef0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f939e7f9700 1 -- 172.21.15.97:0/596955261 reap_dead start 2023-11-24T06:12:34.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 shutdown_connections 2023-11-24T06:12:34.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 >> 172.21.15.97:0/596955261 conn(0x7f939806b620 msgr2=0x7f939815c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:34.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 shutdown_connections 2023-11-24T06:12:34.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.676+0000 7f93a025c700 1 -- 172.21.15.97:0/596955261 wait complete. 2023-11-24T06:12:34.690 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T06:12:34.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.810+0000 7f020784e700 1 Processor -- start 2023-11-24T06:12:34.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.810+0000 7f020784e700 1 -- start start 2023-11-24T06:12:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f020784e700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0200152cc0 0x7f020006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f020784e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020015c4b0 con 0x7f02000d0630 2023-11-24T06:12:34.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f020784e700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020015c790 con 0x7f02000b49a0 2023-11-24T06:12:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f020784e700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020015ca70 con 0x7f0200152cc0 2023-11-24T06:12:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f02055ea700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0200152cc0 0x7f020006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f02055ea700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0200152cc0 0x7f020006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51854/0 (socket says 172.21.15.97:51854) 2023-11-24T06:12:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f0205deb700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f02000d0630 0x7f020015fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44310/0 (socket says 172.21.15.97:44310) 2023-11-24T06:12:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f02055ea700 1 -- 172.21.15.97:0/3075656936 learned_addr learned my addr 172.21.15.97:0/3075656936 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f02055ea700 1 -- 172.21.15.97:0/3075656936 >> v1:172.21.15.183:6789/0 conn(0x7f02000b49a0 legacy=0x7f02000cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f02055ea700 1 -- 172.21.15.97:0/3075656936 >> v1:172.21.15.97:6789/0 conn(0x7f02000d0630 legacy=0x7f020015fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f01f7fff700 1 -- 172.21.15.97:0/3075656936 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569466434 0 0) 0x7f020015c4b0 con 0x7f02000d0630 2023-11-24T06:12:34.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f02055ea700 1 -- 172.21.15.97:0/3075656936 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020015cd50 con 0x7f0200152cc0 2023-11-24T06:12:34.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.811+0000 7f02055ea700 1 --2- 172.21.15.97:0/3075656936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0200152cc0 0x7f020006c000 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0098f0 tx=0x7f01fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abada6e57c9c6c21 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f01f7fff700 1 -- 172.21.15.97:0/3075656936 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc004230 con 0x7f0200152cc0 2023-11-24T06:12:34.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f01f7fff700 1 -- 172.21.15.97:0/3075656936 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc004390 con 0x7f0200152cc0 2023-11-24T06:12:34.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f01f7fff700 1 -- 172.21.15.97:0/3075656936 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0036a0 con 0x7f0200152cc0 2023-11-24T06:12:34.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f020784e700 1 -- 172.21.15.97:0/3075656936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0200152cc0 msgr2=0x7f020006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f020784e700 1 --2- 172.21.15.97:0/3075656936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0200152cc0 0x7f020006c000 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0098f0 tx=0x7f01fc009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:34.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f020784e700 1 -- 172.21.15.97:0/3075656936 shutdown_connections 2023-11-24T06:12:34.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f020784e700 1 --2- 172.21.15.97:0/3075656936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0200152cc0 0x7f020006c000 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f020784e700 1 -- 172.21.15.97:0/3075656936 >> 172.21.15.97:0/3075656936 conn(0x7f0200071b90 msgr2=0x7f0200071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:34.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f020784e700 1 -- 172.21.15.97:0/3075656936 shutdown_connections 2023-11-24T06:12:34.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.812+0000 7f020784e700 1 -- 172.21.15.97:0/3075656936 wait complete. 2023-11-24T06:12:34.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.813+0000 7f020784e700 1 Processor -- start 2023-11-24T06:12:34.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.813+0000 7f020784e700 1 -- start start 2023-11-24T06:12:34.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f020784e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02000b49a0 0x7f020014e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f020784e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02000d0630 0x7f0200148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f020784e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0200152cc0 0x7f020014fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f020784e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0200149050 con 0x7f02000b49a0 2023-11-24T06:12:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f020784e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0200149980 con 0x7f02000d0630 2023-11-24T06:12:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f020784e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0200149bd0 con 0x7f0200152cc0 2023-11-24T06:12:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f02055ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02000b49a0 0x7f020014e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f02055ea700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02000b49a0 0x7f020014e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51860/0 (socket says 172.21.15.97:51860) 2023-11-24T06:12:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f02055ea700 1 -- 172.21.15.97:0/4290636975 learned_addr learned my addr 172.21.15.97:0/4290636975 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f0204de9700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02000d0630 0x7f0200148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.814+0000 7f0205deb700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0200152cc0 0x7f020014fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f02055ea700 1 -- 172.21.15.97:0/4290636975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0200152cc0 msgr2=0x7f020014fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f02055ea700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0200152cc0 0x7f020014fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f02055ea700 1 -- 172.21.15.97:0/4290636975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02000d0630 msgr2=0x7f0200148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f02055ea700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02000d0630 0x7f0200148b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f02055ea700 1 -- 172.21.15.97:0/4290636975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc009550 con 0x7f02000b49a0 2023-11-24T06:12:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f02055ea700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02000b49a0 0x7f020014e5e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f01fc000c00 tx=0x7f01fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f01f5ffb700 1 -- 172.21.15.97:0/4290636975 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00b660 con 0x7f02000b49a0 2023-11-24T06:12:34.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f01f5ffb700 1 -- 172.21.15.97:0/4290636975 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc00b7e0 con 0x7f02000b49a0 2023-11-24T06:12:34.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0200149eb0 con 0x7f02000b49a0 2023-11-24T06:12:34.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f01f5ffb700 1 -- 172.21.15.97:0/4290636975 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00bb90 con 0x7f02000b49a0 2023-11-24T06:12:34.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.815+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020014a470 con 0x7f02000b49a0 2023-11-24T06:12:34.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.816+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02000753f0 con 0x7f02000b49a0 2023-11-24T06:12:34.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.817+0000 7f01f5ffb700 1 -- 172.21.15.97:0/4290636975 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f01fc00bcf0 con 0x7f02000b49a0 2023-11-24T06:12:34.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.817+0000 7f01f5ffb700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f01e406b500 0x7f01e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:34.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.817+0000 7f0204de9700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f01e406b500 0x7f01e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:34.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.817+0000 7f01f5ffb700 1 -- 172.21.15.97:0/4290636975 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f01fc08b7d0 con 0x7f02000b49a0 2023-11-24T06:12:34.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.818+0000 7f0204de9700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f01e406b500 0x7f01e406d9c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0200147380 tx=0x7f01f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:34.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:34.822+0000 7f01f5ffb700 1 -- 172.21.15.97:0/4290636975 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01fc0562b0 con 0x7f02000b49a0 2023-11-24T06:12:35.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.014+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0200067f60 con 0x7f01e406b500 2023-11-24T06:12:35.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.015+0000 7f01f5ffb700 1 -- 172.21.15.97:0/4290636975 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0200067f60 con 0x7f01e406b500 2023-11-24T06:12:35.019 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f01e406b500 msgr2=0x7f01e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f01e406b500 0x7f01e406d9c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0200147380 tx=0x7f01f0009040 comp rx=0 tx=0).stop 2023-11-24T06:12:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02000b49a0 msgr2=0x7f020014e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02000b49a0 0x7f020014e5e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f01fc000c00 tx=0x7f01fc004750 comp rx=0 tx=0).stop 2023-11-24T06:12:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 shutdown_connections 2023-11-24T06:12:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f01e406b500 0x7f01e406d9c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.020 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f0200152cc0 0x7f020014fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02000b49a0 0x7f020014e5e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 --2- 172.21.15.97:0/4290636975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02000d0630 0x7f0200148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 >> 172.21.15.97:0/4290636975 conn(0x7f0200071b90 msgr2=0x7f02001519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 shutdown_connections 2023-11-24T06:12:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.017+0000 7f020784e700 1 -- 172.21.15.97:0/4290636975 wait complete. 2023-11-24T06:12:35.021 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T06:12:35.031 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T06:12:35.031 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:12:35.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.152+0000 7fdebefd8700 1 Processor -- start 2023-11-24T06:12:35.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.152+0000 7fdebefd8700 1 -- start start 2023-11-24T06:12:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebefd8700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdeb80b01e0 0x7fdeb806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebefd8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80746a0 0x7fdeb80b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebefd8700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb80b05c0 con 0x7fdeb80b59c0 2023-11-24T06:12:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebefd8700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeb81470a0 con 0x7fdeb80b01e0 2023-11-24T06:12:35.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebefd8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80b5e70 con 0x7fdeb80746a0 2023-11-24T06:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebd575700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80746a0 0x7fdeb80b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebcd74700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdeb80b01e0 0x7fdeb806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebcd74700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fdeb80b01e0 0x7fdeb806e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:46028/0 (socket says 172.21.15.97:46028) 2023-11-24T06:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebd575700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80746a0 0x7fdeb80b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3301/0 says I am v2:172.21.15.97:38908/0 (socket says 172.21.15.97:38908) 2023-11-24T06:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebcd74700 1 -- 172.21.15.97:0/2988283236 learned_addr learned my addr 172.21.15.97:0/2988283236 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebd575700 1 -- 172.21.15.97:0/2988283236 >> v2:172.21.15.183:3300/0 conn(0x7fdeb80b01e0 msgr2=0x7fdeb806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebd575700 1 --2- 172.21.15.97:0/2988283236 >> v2:172.21.15.183:3300/0 conn(0x7fdeb80b01e0 0x7fdeb806e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.158 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebd575700 1 -- 172.21.15.97:0/2988283236 >> v1:172.21.15.183:6789/0 conn(0x7fdeb80b59c0 legacy=0x7fdeb8073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebd575700 1 -- 172.21.15.97:0/2988283236 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb80b4250 con 0x7fdeb80746a0 2023-11-24T06:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.153+0000 7fdebcd74700 1 --2- 172.21.15.97:0/2988283236 >> v2:172.21.15.183:3300/0 conn(0x7fdeb80b01e0 0x7fdeb806e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebd575700 1 --2- 172.21.15.97:0/2988283236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80746a0 0x7fdeb80b3b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009d90 tx=0x7fdeb000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36de07e97a76be6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdeb77fe700 1 -- 172.21.15.97:0/2988283236 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000e070 con 0x7fdeb80746a0 2023-11-24T06:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdeb77fe700 1 -- 172.21.15.97:0/2988283236 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb000bb20 con 0x7fdeb80746a0 2023-11-24T06:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdeb77fe700 1 -- 172.21.15.97:0/2988283236 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0012430 con 0x7fdeb80746a0 2023-11-24T06:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebefd8700 1 -- 172.21.15.97:0/2988283236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80746a0 msgr2=0x7fdeb80b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebefd8700 1 --2- 172.21.15.97:0/2988283236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80746a0 0x7fdeb80b3b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdeb0009d90 tx=0x7fdeb000b330 comp rx=0 tx=0).stop 2023-11-24T06:12:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebefd8700 1 -- 172.21.15.97:0/2988283236 shutdown_connections 2023-11-24T06:12:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebefd8700 1 --2- 172.21.15.97:0/2988283236 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80746a0 0x7fdeb80b3b50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebefd8700 1 --2- 172.21.15.97:0/2988283236 >> v2:172.21.15.183:3300/0 conn(0x7fdeb80b01e0 0x7fdeb806e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebefd8700 1 -- 172.21.15.97:0/2988283236 >> 172.21.15.97:0/2988283236 conn(0x7fdeb80d0b40 msgr2=0x7fdeb80d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebefd8700 1 -- 172.21.15.97:0/2988283236 shutdown_connections 2023-11-24T06:12:35.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.154+0000 7fdebefd8700 1 -- 172.21.15.97:0/2988283236 wait complete. 2023-11-24T06:12:35.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.155+0000 7fdebefd8700 1 Processor -- start 2023-11-24T06:12:35.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.155+0000 7fdebefd8700 1 -- start start 2023-11-24T06:12:35.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebefd8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb80746a0 0x7fdeb80c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebefd8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeb80b01e0 0x7fdeb80cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebefd8700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80b59c0 0x7fdeb80cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebefd8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80cc780 con 0x7fdeb80746a0 2023-11-24T06:12:35.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebefd8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80c6030 con 0x7fdeb80b01e0 2023-11-24T06:12:35.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebefd8700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80c6310 con 0x7fdeb80b59c0 2023-11-24T06:12:35.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebcd74700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb80746a0 0x7fdeb80c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebcd74700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb80746a0 0x7fdeb80c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51874/0 (socket says 172.21.15.97:51874) 2023-11-24T06:12:35.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.156+0000 7fdebcd74700 1 -- 172.21.15.97:0/125755054 learned_addr learned my addr 172.21.15.97:0/125755054 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:35.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebd575700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80b59c0 0x7fdeb80cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdeb7fff700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeb80b01e0 0x7fdeb80cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebcd74700 1 -- 172.21.15.97:0/125755054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80b59c0 msgr2=0x7fdeb80cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebcd74700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80b59c0 0x7fdeb80cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebcd74700 1 -- 172.21.15.97:0/125755054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeb80b01e0 msgr2=0x7fdeb80cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebcd74700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeb80b01e0 0x7fdeb80cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebcd74700 1 -- 172.21.15.97:0/125755054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac009550 con 0x7fdeb80746a0 2023-11-24T06:12:35.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebd575700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fdeb80b59c0 0x7fdeb80cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:35.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdeb7fff700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdeb80b01e0 0x7fdeb80cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:35.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebcd74700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb80746a0 0x7fdeb80c70d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00ebe0 tx=0x7fdeac009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00c8b0 con 0x7fdeb80746a0 2023-11-24T06:12:35.166 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac00ca10 con 0x7fdeb80746a0 2023-11-24T06:12:35.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb0009a70 con 0x7fdeb80746a0 2023-11-24T06:12:35.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00cd80 con 0x7fdeb80746a0 2023-11-24T06:12:35.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.157+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb80c68d0 con 0x7fdeb80746a0 2023-11-24T06:12:35.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.158+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdeb8146be0 con 0x7fdeb80746a0 2023-11-24T06:12:35.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.158+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fdeac013470 con 0x7fdeb80746a0 2023-11-24T06:12:35.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdeb57fa700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fde9c06b500 0x7fde9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fdeac08a170 con 0x7fdeb80746a0 2023-11-24T06:12:35.167 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdeb7fff700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fde9c06b500 0x7fde9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdeb57fa700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fde9c071360 0x7fde9c0737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 --> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde9c073e70 con 0x7fde9c071360 2023-11-24T06:12:35.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeac08a450 con 0x7fdeb80746a0 2023-11-24T06:12:35.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdebd575700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fde9c071360 0x7fde9c0737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdeb7fff700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fde9c06b500 0x7fde9c06d9c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdeb81f7560 tx=0x7fdea4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.159+0000 7fdebd575700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fde9c071360 0x7fde9c0737a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.168 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.160+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 <== osd.1 v2:172.21.15.97:6808/3026562872 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fde9c073e70 con 0x7fde9c071360 2023-11-24T06:12:35.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.175+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 --> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdeb806e1c0 con 0x7fde9c071360 2023-11-24T06:12:35.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.175+0000 7fdeb57fa700 1 -- 172.21.15.97:0/125755054 <== osd.1 v2:172.21.15.97:6808/3026562872 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdeb0003ca0 con 0x7fde9c071360 2023-11-24T06:12:35.178 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:35.178 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:12:35.178 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:35.179 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fde9c071360 msgr2=0x7fde9c0737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6808/3026562872,v1:172.21.15.97:6809/3026562872] conn(0x7fde9c071360 0x7fde9c0737a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fde9c06b500 msgr2=0x7fde9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fde9c06b500 0x7fde9c06d9c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdeb81f7560 tx=0x7fdea4009040 comp rx=0 tx=0).stop 2023-11-24T06:12:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb80746a0 msgr2=0x7fdeb80c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 --2- 172.21.15.97:0/125755054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdeb80746a0 0x7fdeb80c70d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00ebe0 tx=0x7fdeac009e70 comp rx=0 tx=0).stop 2023-11-24T06:12:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebd575700 1 -- 172.21.15.97:0/125755054 reap_dead start 2023-11-24T06:12:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 shutdown_connections 2023-11-24T06:12:35.180 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 >> 172.21.15.97:0/125755054 conn(0x7fdeb80d0b40 msgr2=0x7fdeb8061ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 shutdown_connections 2023-11-24T06:12:35.181 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.176+0000 7fdebefd8700 1 -- 172.21.15.97:0/125755054 wait complete. 2023-11-24T06:12:35.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T06:12:35.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.308+0000 7f454d72d700 1 Processor -- start 2023-11-24T06:12:35.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.308+0000 7f454d72d700 1 -- start start 2023-11-24T06:12:35.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f454d72d700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f454815bd60 0x7f45480d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f454d72d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45480cf2b0 0x7f45480c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f454d72d700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45480cc780 con 0x7f454815c140 2023-11-24T06:12:35.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f454d72d700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45480cc9b0 con 0x7f45480cf2b0 2023-11-24T06:12:35.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f454d72d700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45480ccc90 con 0x7f454815bd60 2023-11-24T06:12:35.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f45477fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45480cf2b0 0x7f45480c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f45477fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45480cf2b0 0x7f45480c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51888/0 (socket says 172.21.15.97:51888) 2023-11-24T06:12:35.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f45477fe700 1 -- 172.21.15.97:0/3422939871 learned_addr learned my addr 172.21.15.97:0/3422939871 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:35.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f4546ffd700 1 --2- 172.21.15.97:0/3422939871 >> v2:172.21.15.183:3300/0 conn(0x7f454815bd60 0x7f45480d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f45477fe700 1 -- 172.21.15.97:0/3422939871 >> v2:172.21.15.183:3300/0 conn(0x7f454815bd60 msgr2=0x7f45480d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f45477fe700 1 --2- 172.21.15.97:0/3422939871 >> v2:172.21.15.183:3300/0 conn(0x7f454815bd60 0x7f45480d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f45477fe700 1 -- 172.21.15.97:0/3422939871 >> v1:172.21.15.183:6789/0 conn(0x7f454815c140 legacy=0x7f45480cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f45477fe700 1 -- 172.21.15.97:0/3422939871 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45480ccf70 con 0x7f45480cf2b0 2023-11-24T06:12:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.309+0000 7f45477fe700 1 --2- 172.21.15.97:0/3422939871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45480cf2b0 0x7f45480c57c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f45480c8cf0 tx=0x7f454000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9412929fce68a5ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f4545ffb700 1 -- 172.21.15.97:0/3422939871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000e070 con 0x7f45480cf2b0 2023-11-24T06:12:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f4545ffb700 1 -- 172.21.15.97:0/3422939871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4540003a20 con 0x7f45480cf2b0 2023-11-24T06:12:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f4545ffb700 1 -- 172.21.15.97:0/3422939871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4540003dd0 con 0x7f45480cf2b0 2023-11-24T06:12:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f454d72d700 1 -- 172.21.15.97:0/3422939871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45480cf2b0 msgr2=0x7f45480c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f454d72d700 1 --2- 172.21.15.97:0/3422939871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45480cf2b0 0x7f45480c57c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f45480c8cf0 tx=0x7f454000b330 comp rx=0 tx=0).stop 2023-11-24T06:12:35.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f454d72d700 1 -- 172.21.15.97:0/3422939871 shutdown_connections 2023-11-24T06:12:35.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f454d72d700 1 --2- 172.21.15.97:0/3422939871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45480cf2b0 0x7f45480c57c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f454d72d700 1 --2- 172.21.15.97:0/3422939871 >> v2:172.21.15.183:3300/0 conn(0x7f454815bd60 0x7f45480d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f454d72d700 1 -- 172.21.15.97:0/3422939871 >> 172.21.15.97:0/3422939871 conn(0x7f4548073130 msgr2=0x7f454809aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f454d72d700 1 -- 172.21.15.97:0/3422939871 shutdown_connections 2023-11-24T06:12:35.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.310+0000 7f454d72d700 1 -- 172.21.15.97:0/3422939871 wait complete. 2023-11-24T06:12:35.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.312+0000 7f454d72d700 1 Processor -- start 2023-11-24T06:12:35.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.312+0000 7f454d72d700 1 -- start start 2023-11-24T06:12:35.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f454d72d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45480cf2b0 0x7f454809bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f454d72d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454815bd60 0x7f45480c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f454d72d700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f454815c140 0x7f454815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f454d72d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480c55b0 con 0x7f454815bd60 2023-11-24T06:12:35.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f454d72d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480b37a0 con 0x7f45480cf2b0 2023-11-24T06:12:35.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f454d72d700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454809c220 con 0x7f454815c140 2023-11-24T06:12:35.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454815bd60 0x7f45480c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454815bd60 0x7f45480c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51890/0 (socket says 172.21.15.97:51890) 2023-11-24T06:12:35.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 -- 172.21.15.97:0/3377915573 learned_addr learned my addr 172.21.15.97:0/3377915573 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:35.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45477fe700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f454815c140 0x7f454815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f4546ffd700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45480cf2b0 0x7f454809bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 -- 172.21.15.97:0/3377915573 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f454815c140 msgr2=0x7f454815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f454815c140 0x7f454815b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 -- 172.21.15.97:0/3377915573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45480cf2b0 msgr2=0x7f454809bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45480cf2b0 0x7f454809bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 -- 172.21.15.97:0/3377915573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540009b30 con 0x7f454815bd60 2023-11-24T06:12:35.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f45467fc700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454815bd60 0x7f45480c5070 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f453800b4d0 tx=0x7f453800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.321 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.313+0000 7f452f7fe700 1 -- 172.21.15.97:0/3377915573 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538011830 con 0x7f454815bd60 2023-11-24T06:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.314+0000 7f452f7fe700 1 -- 172.21.15.97:0/3377915573 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538011990 con 0x7f454815bd60 2023-11-24T06:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.314+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454809f2f0 con 0x7f454815bd60 2023-11-24T06:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.314+0000 7f452f7fe700 1 -- 172.21.15.97:0/3377915573 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538011d40 con 0x7f454815bd60 2023-11-24T06:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.314+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454809f8b0 con 0x7f454815bd60 2023-11-24T06:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.315+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454809fa70 con 0x7f454815bd60 2023-11-24T06:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.315+0000 7f452f7fe700 1 -- 172.21.15.97:0/3377915573 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f453801a400 con 0x7f454815bd60 2023-11-24T06:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.315+0000 7f452f7fe700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f453406b500 0x7f453406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.322 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.315+0000 7f452f7fe700 1 -- 172.21.15.97:0/3377915573 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f4538089bd0 con 0x7f454815bd60 2023-11-24T06:12:35.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.315+0000 7f4546ffd700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f453406b500 0x7f453406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.316+0000 7f4546ffd700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f453406b500 0x7f453406d9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f453c009930 tx=0x7f453c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.323 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.320+0000 7f452f7fe700 1 -- 172.21.15.97:0/3377915573 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4538054590 con 0x7f454815bd60 2023-11-24T06:12:35.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.509+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f45480ce160 con 0x7f453406b500 2023-11-24T06:12:35.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.509+0000 7f452f7fe700 1 -- 172.21.15.97:0/3377915573 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f453c009e00 con 0x7f453406b500 2023-11-24T06:12:35.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.511+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f453406b500 msgr2=0x7f453406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.516 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.511+0000 7f454d72d700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f453406b500 0x7f453406d9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f453c009930 tx=0x7f453c008040 comp rx=0 tx=0).stop 2023-11-24T06:12:35.517 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.511+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454815bd60 msgr2=0x7f45480c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.511+0000 7f454d72d700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454815bd60 0x7f45480c5070 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f453800b4d0 tx=0x7f453800b890 comp rx=0 tx=0).stop 2023-11-24T06:12:35.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.511+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 shutdown_connections 2023-11-24T06:12:35.518 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.512+0000 7f454d72d700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f453406b500 0x7f453406d9c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.512+0000 7f454d72d700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f454815c140 0x7f454815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.512+0000 7f454d72d700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45480cf2b0 0x7f454809bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.519 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.512+0000 7f454d72d700 1 --2- 172.21.15.97:0/3377915573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454815bd60 0x7f45480c5070 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.512+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 >> 172.21.15.97:0/3377915573 conn(0x7f4548073130 msgr2=0x7f45480c72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.512+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 shutdown_connections 2023-11-24T06:12:35.520 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.512+0000 7f454d72d700 1 -- 172.21.15.97:0/3377915573 wait complete. 2023-11-24T06:12:35.521 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T06:12:35.527 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T06:12:35.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:12:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.649+0000 7f1e187d0700 1 Processor -- start 2023-11-24T06:12:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.649+0000 7f1e187d0700 1 -- start start 2023-11-24T06:12:35.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e187d0700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f1e100746a0 0x7f1e1006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e187d0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e100b59c0 0x7f1e100b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e187d0700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e100b5e50 con 0x7f1e100b01e0 2023-11-24T06:12:35.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e187d0700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e101470a0 con 0x7f1e100b59c0 2023-11-24T06:12:35.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e187d0700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e100b05c0 con 0x7f1e100746a0 2023-11-24T06:12:35.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e16d6d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e100b59c0 0x7f1e100b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e1656c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f1e100b01e0 0x7f1e10073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44350/0 (socket says 172.21.15.97:44350) 2023-11-24T06:12:35.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e16d6d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e100b59c0 0x7f1e100b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51902/0 (socket says 172.21.15.97:51902) 2023-11-24T06:12:35.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e1656c700 1 -- 172.21.15.97:0/856117553 learned_addr learned my addr 172.21.15.97:0/856117553 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.650+0000 7f1e15d6b700 1 --2- 172.21.15.97:0/856117553 >> v2:172.21.15.183:3300/0 conn(0x7f1e100746a0 0x7f1e1006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e16d6d700 1 -- 172.21.15.97:0/856117553 >> v2:172.21.15.183:3300/0 conn(0x7f1e100746a0 msgr2=0x7f1e1006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e16d6d700 1 --2- 172.21.15.97:0/856117553 >> v2:172.21.15.183:3300/0 conn(0x7f1e100746a0 0x7f1e1006e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e1556a700 1 -- 172.21.15.97:0/856117553 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045908182 0 0) 0x7f1e00001110 con 0x7f1e100b01e0 2023-11-24T06:12:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e16d6d700 1 -- 172.21.15.97:0/856117553 >> v1:172.21.15.97:6789/0 conn(0x7f1e100b01e0 legacy=0x7f1e10073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e16d6d700 1 -- 172.21.15.97:0/856117553 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e100b4250 con 0x7f1e100b59c0 2023-11-24T06:12:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e16d6d700 1 --2- 172.21.15.97:0/856117553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100b59c0 0x7f1e100b3b50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1e100d1970 tx=0x7f1e0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43947640816e7c10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e1556a700 1 -- 172.21.15.97:0/856117553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0800e070 con 0x7f1e100b59c0 2023-11-24T06:12:35.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e1556a700 1 -- 172.21.15.97:0/856117553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0800bbd0 con 0x7f1e100b59c0 2023-11-24T06:12:35.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e1556a700 1 -- 172.21.15.97:0/856117553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08012430 con 0x7f1e100b59c0 2023-11-24T06:12:35.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e187d0700 1 -- 172.21.15.97:0/856117553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100b59c0 msgr2=0x7f1e100b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e187d0700 1 --2- 172.21.15.97:0/856117553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100b59c0 0x7f1e100b3b50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1e100d1970 tx=0x7f1e0800b330 comp rx=0 tx=0).stop 2023-11-24T06:12:35.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e187d0700 1 -- 172.21.15.97:0/856117553 shutdown_connections 2023-11-24T06:12:35.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e187d0700 1 --2- 172.21.15.97:0/856117553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100b59c0 0x7f1e100b3b50 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e187d0700 1 --2- 172.21.15.97:0/856117553 >> v2:172.21.15.183:3300/0 conn(0x7f1e100746a0 0x7f1e1006e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e187d0700 1 -- 172.21.15.97:0/856117553 >> 172.21.15.97:0/856117553 conn(0x7f1e100d0b40 msgr2=0x7f1e100d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e187d0700 1 -- 172.21.15.97:0/856117553 shutdown_connections 2023-11-24T06:12:35.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.651+0000 7f1e187d0700 1 -- 172.21.15.97:0/856117553 wait complete. 2023-11-24T06:12:35.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e187d0700 1 Processor -- start 2023-11-24T06:12:35.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e187d0700 1 -- start start 2023-11-24T06:12:35.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e187d0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100746a0 0x7f1e1015f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e187d0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e100b01e0 0x7f1e101e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e187d0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e100b59c0 0x7f1e10142440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e187d0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101e7e30 con 0x7f1e100746a0 2023-11-24T06:12:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e187d0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101e8060 con 0x7f1e100b01e0 2023-11-24T06:12:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e187d0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e101e8340 con 0x7f1e100b59c0 2023-11-24T06:12:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e1656c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100746a0 0x7f1e1015f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e1656c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100746a0 0x7f1e1015f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51904/0 (socket says 172.21.15.97:51904) 2023-11-24T06:12:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.653+0000 7f1e1656c700 1 -- 172.21.15.97:0/1488983193 learned_addr learned my addr 172.21.15.97:0/1488983193 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e15d6b700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e100b01e0 0x7f1e101e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e16d6d700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e100b59c0 0x7f1e10142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e1656c700 1 -- 172.21.15.97:0/1488983193 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e100b59c0 msgr2=0x7f1e10142440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e1656c700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e100b59c0 0x7f1e10142440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e1656c700 1 -- 172.21.15.97:0/1488983193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e100b01e0 msgr2=0x7f1e101e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e1656c700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e100b01e0 0x7f1e101e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e1656c700 1 -- 172.21.15.97:0/1488983193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e08009a70 con 0x7f1e100746a0 2023-11-24T06:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e16d6d700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f1e100b59c0 0x7f1e10142440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e1656c700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100746a0 0x7f1e1015f420 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c730 tx=0x7f1e0000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00004c80 con 0x7f1e100746a0 2023-11-24T06:12:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e00004e00 con 0x7f1e100746a0 2023-11-24T06:12:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e101e8620 con 0x7f1e100746a0 2023-11-24T06:12:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e00007310 con 0x7f1e100746a0 2023-11-24T06:12:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.654+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1015c0a0 con 0x7f1e100746a0 2023-11-24T06:12:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.655+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e100d0800 con 0x7f1e100746a0 2023-11-24T06:12:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.655+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1e00007470 con 0x7f1e100746a0 2023-11-24T06:12:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e06ffd700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1df406b500 0x7f1df406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f1e00089a60 con 0x7f1e100746a0 2023-11-24T06:12:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e15d6b700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1df406b500 0x7f1df406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e06ffd700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f1df4071360 0x7f1df40737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 --> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1df4073e70 con 0x7f1df4071360 2023-11-24T06:12:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e0008e2a0 con 0x7f1e100746a0 2023-11-24T06:12:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e16d6d700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f1df4071360 0x7f1df40737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e15d6b700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1df406b500 0x7f1df406d9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1e10161850 tx=0x7f1e0c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.656+0000 7f1e16d6d700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f1df4071360 0x7f1df40737a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.666 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.657+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 <== osd.2 v2:172.21.15.97:6816/1638045649 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1df4073e70 con 0x7f1df4071360 2023-11-24T06:12:35.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 --> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1e1006e1c0 con 0x7f1df4071360 2023-11-24T06:12:35.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e06ffd700 1 -- 172.21.15.97:0/1488983193 <== osd.2 v2:172.21.15.97:6816/1638045649 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1e080098f0 con 0x7f1df4071360 2023-11-24T06:12:35.675 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:35.675 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:12:35.675 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:35.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f1df4071360 msgr2=0x7f1df40737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6816/1638045649,v1:172.21.15.97:6818/1638045649] conn(0x7f1df4071360 0x7f1df40737a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1df406b500 msgr2=0x7f1df406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f1df406b500 0x7f1df406d9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1e10161850 tx=0x7f1e0c008040 comp rx=0 tx=0).stop 2023-11-24T06:12:35.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100746a0 msgr2=0x7f1e1015f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 --2- 172.21.15.97:0/1488983193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e100746a0 0x7f1e1015f420 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c730 tx=0x7f1e0000caf0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e16d6d700 1 -- 172.21.15.97:0/1488983193 reap_dead start 2023-11-24T06:12:35.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 shutdown_connections 2023-11-24T06:12:35.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 >> 172.21.15.97:0/1488983193 conn(0x7f1e100d0b40 msgr2=0x7f1e100b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 shutdown_connections 2023-11-24T06:12:35.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.672+0000 7f1e187d0700 1 -- 172.21.15.97:0/1488983193 wait complete. 2023-11-24T06:12:35.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T06:12:35.809 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.807+0000 7f4fc0951700 1 Processor -- start 2023-11-24T06:12:35.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.807+0000 7f4fc0951700 1 -- start start 2023-11-24T06:12:35.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fc0951700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f4fbc0adf00 0x7f4fbc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fc0951700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 0x7f4fbc0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fc0951700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc0af950 con 0x7f4fbc1446a0 2023-11-24T06:12:35.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fc0951700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc144a80 con 0x7f4fbc0adf00 2023-11-24T06:12:35.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fc0951700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc06c9c0 con 0x7f4fbc06c540 2023-11-24T06:12:35.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fba59c700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4fbc1446a0 0x7f4fbc0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44366/0 (socket says 172.21.15.97:44366) 2023-11-24T06:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fba59c700 1 -- 172.21.15.97:0/201778218 learned_addr learned my addr 172.21.15.97:0/201778218 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fbad9d700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 0x7f4fbc0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fb9d9b700 1 --2- 172.21.15.97:0/201778218 >> v2:172.21.15.183:3300/0 conn(0x7f4fbc0adf00 0x7f4fbc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558638727 0 0) 0x7f4fbc0af950 con 0x7f4fbc1446a0 2023-11-24T06:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.808+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa40035b0 con 0x7f4fbc1446a0 2023-11-24T06:12:35.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402550598 0 0) 0x7f4fa40035b0 con 0x7f4fbc1446a0 2023-11-24T06:12:35.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 msgr2=0x7f4fbc0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 0x7f4fbc0b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 >> v2:172.21.15.183:3300/0 conn(0x7f4fbc0adf00 msgr2=0x7f4fbc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 --2- 172.21.15.97:0/201778218 >> v2:172.21.15.183:3300/0 conn(0x7f4fbc0adf00 0x7f4fbc06c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0afc30 con 0x7f4fbc1446a0 2023-11-24T06:12:35.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f4fac001670 con 0x7f4fbc1446a0 2023-11-24T06:12:35.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 >> v1:172.21.15.97:6789/0 conn(0x7f4fbc1446a0 legacy=0x7f4fbc0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fa4004850 0x7f4fa4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 0x7f4fa4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc144a80 con 0x7f4fa4003a00 2023-11-24T06:12:35.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc06c9c0 con 0x7f4fa4004850 2023-11-24T06:12:35.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa400dc20 con 0x7f4fbc06c540 2023-11-24T06:12:35.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fba59c700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fbad9d700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 0x7f4fa4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb9d9b700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fa4004850 0x7f4fa4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fba59c700 1 -- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 msgr2=0x7f4fa4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fba59c700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 0x7f4fa4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fba59c700 1 -- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fa4004850 msgr2=0x7f4fa4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:35.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fba59c700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fa4004850 0x7f4fa4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fba59c700 1 -- 172.21.15.97:0/201778218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa40035b0 con 0x7f4fa4003a00 2023-11-24T06:12:35.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fbad9d700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fbc06c540 0x7f4fa4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:35.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb9d9b700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fa4004850 0x7f4fa4004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:35.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fba59c700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e20 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c370 tx=0x7f4fa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db1b0f2c2eb486b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac007c60 con 0x7f4fa4003a00 2023-11-24T06:12:35.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac005960 con 0x7f4fa4003a00 2023-11-24T06:12:35.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.809+0000 7f4fb959a700 1 -- 172.21.15.97:0/201778218 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac005d10 con 0x7f4fa4003a00 2023-11-24T06:12:35.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.810+0000 7f4fc0951700 1 -- 172.21.15.97:0/201778218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fa4003a00 msgr2=0x7f4fa4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.810+0000 7f4fc0951700 1 --2- 172.21.15.97:0/201778218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fa4003a00 0x7f4fa4003e20 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c370 tx=0x7f4fa4005880 comp rx=0 tx=0).stop 2023-11-24T06:12:35.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.810+0000 7f4fbad9d700 1 -- 172.21.15.97:0/201778218 reap_dead start 2023-11-24T06:12:35.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.810+0000 7f4fc0951700 1 -- 172.21.15.97:0/201778218 shutdown_connections 2023-11-24T06:12:35.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.810+0000 7f4fc0951700 1 -- 172.21.15.97:0/201778218 >> 172.21.15.97:0/201778218 conn(0x7f4fbc071b90 msgr2=0x7f4fbc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:35.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.810+0000 7f4fc0951700 1 -- 172.21.15.97:0/201778218 shutdown_connections 2023-11-24T06:12:35.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.810+0000 7f4fc0951700 1 -- 172.21.15.97:0/201778218 wait complete. 2023-11-24T06:12:35.824 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.811+0000 7f4fc0951700 1 Processor -- start 2023-11-24T06:12:35.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.811+0000 7f4fc0951700 1 -- start start 2023-11-24T06:12:35.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fc0951700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fa4003a00 0x7f4fbc137c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fc0951700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cf060 0x7f4fbc13a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fc0951700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fbc0cc230 0x7f4fbc0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fc0951700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc13a0b0 con 0x7f4fbc0cf060 2023-11-24T06:12:35.825 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fc0951700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc13aa10 con 0x7f4fbc0cc230 2023-11-24T06:12:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fc0951700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0cf7e0 con 0x7f4fa4003a00 2023-11-24T06:12:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fb9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cf060 0x7f4fbc13a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fb9d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cf060 0x7f4fbc13a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51928/0 (socket says 172.21.15.97:51928) 2023-11-24T06:12:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fb9d9b700 1 -- 172.21.15.97:0/3534335126 learned_addr learned my addr 172.21.15.97:0/3534335126 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:35.826 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fba59c700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fa4003a00 0x7f4fbc137c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.812+0000 7f4fbad9d700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fbc0cc230 0x7f4fbc0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fb9d9b700 1 -- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fa4003a00 msgr2=0x7f4fbc137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fb9d9b700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fa4003a00 0x7f4fbc137c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fb9d9b700 1 -- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fbc0cc230 msgr2=0x7f4fbc0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:35.827 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fb9d9b700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fbc0cc230 0x7f4fbc0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fb9d9b700 1 -- 172.21.15.97:0/3534335126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc139710 con 0x7f4fbc0cf060 2023-11-24T06:12:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fbad9d700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fbc0cc230 0x7f4fbc0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fba59c700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fa4003a00 0x7f4fbc137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fb9d9b700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cf060 0x7f4fbc13a4d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00108f0 tx=0x7f4fb0010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4faaffd700 1 -- 172.21.15.97:0/3534335126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0010e90 con 0x7f4fbc0cf060 2023-11-24T06:12:35.828 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4faaffd700 1 -- 172.21.15.97:0/3534335126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb000e8d0 con 0x7f4fbc0cf060 2023-11-24T06:12:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc1399f0 con 0x7f4fbc0cf060 2023-11-24T06:12:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4faaffd700 1 -- 172.21.15.97:0/3534335126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000ec60 con 0x7f4fbc0cf060 2023-11-24T06:12:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.813+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc0cf4e0 con 0x7f4fbc0cf060 2023-11-24T06:12:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.814+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc0753f0 con 0x7f4fbc0cf060 2023-11-24T06:12:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.814+0000 7f4faaffd700 1 -- 172.21.15.97:0/3534335126 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f4fb0016460 con 0x7f4fbc0cf060 2023-11-24T06:12:35.829 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.815+0000 7f4faaffd700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4f9c06b530 0x7f4f9c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.815+0000 7f4faaffd700 1 -- 172.21.15.97:0/3534335126 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f4fb008a120 con 0x7f4fbc0cf060 2023-11-24T06:12:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.815+0000 7f4fba59c700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4f9c06b530 0x7f4f9c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.815+0000 7f4fba59c700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4f9c06b530 0x7f4f9c06d9f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4fac015040 tx=0x7f4fac00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:35.830 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:35.819+0000 7f4faaffd700 1 -- 172.21.15.97:0/3534335126 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fb0054b70 con 0x7f4fbc0cf060 2023-11-24T06:12:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.010+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fbc0b04d0 con 0x7f4f9c06b530 2023-11-24T06:12:36.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.010+0000 7f4faaffd700 1 -- 172.21.15.97:0/3534335126 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4fac00cd30 con 0x7f4f9c06b530 2023-11-24T06:12:36.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.012+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4f9c06b530 msgr2=0x7f4f9c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.012+0000 7f4fc0951700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4f9c06b530 0x7f4f9c06d9f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4fac015040 tx=0x7f4fac00a000 comp rx=0 tx=0).stop 2023-11-24T06:12:36.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.012+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cf060 msgr2=0x7f4fbc13a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.012+0000 7f4fc0951700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cf060 0x7f4fbc13a4d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00108f0 tx=0x7f4fb0010cb0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.013+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 shutdown_connections 2023-11-24T06:12:36.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.013+0000 7f4fc0951700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fbc0cc230 0x7f4fbc0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.013+0000 7f4fc0951700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f4f9c06b530 0x7f4f9c06d9f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.013+0000 7f4fc0951700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4fa4003a00 0x7f4fbc137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.013+0000 7f4fc0951700 1 --2- 172.21.15.97:0/3534335126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fbc0cf060 0x7f4fbc13a4d0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.013+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 >> 172.21.15.97:0/3534335126 conn(0x7f4fbc071b90 msgr2=0x7f4fbc0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.013+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 shutdown_connections 2023-11-24T06:12:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.013+0000 7f4fc0951700 1 -- 172.21.15.97:0/3534335126 wait complete. 2023-11-24T06:12:36.017 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T06:12:36.027 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T06:12:36.028 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:12:36.151 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.149+0000 7f6cf1f1b700 1 Processor -- start 2023-11-24T06:12:36.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.149+0000 7f6cf1f1b700 1 -- start start 2023-11-24T06:12:36.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.149+0000 7f6cf1f1b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cec0b01e0 0x7f6cec06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6cf1f1b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0746a0 0x7f6cec0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6cf1f1b700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec0b05c0 con 0x7f6cec0b59c0 2023-11-24T06:12:36.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6cf1f1b700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec1470a0 con 0x7f6cec0b01e0 2023-11-24T06:12:36.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6cf1f1b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b5e70 con 0x7f6cec0746a0 2023-11-24T06:12:36.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6ceb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cec0b01e0 0x7f6cec06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6ceb7fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cec0b01e0 0x7f6cec06e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51934/0 (socket says 172.21.15.97:51934) 2023-11-24T06:12:36.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6ceb7fe700 1 -- 172.21.15.97:0/4254067278 learned_addr learned my addr 172.21.15.97:0/4254067278 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:36.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6cebfff700 1 --2- 172.21.15.97:0/4254067278 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0746a0 0x7f6cec0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6ceb7fe700 1 -- 172.21.15.97:0/4254067278 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0746a0 msgr2=0x7f6cec0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:36.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6ceb7fe700 1 --2- 172.21.15.97:0/4254067278 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0746a0 0x7f6cec0b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6ceb7fe700 1 -- 172.21.15.97:0/4254067278 >> v1:172.21.15.183:6789/0 conn(0x7f6cec0b59c0 legacy=0x7f6cec073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6ceb7fe700 1 -- 172.21.15.97:0/4254067278 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0b4250 con 0x7f6cec0b01e0 2023-11-24T06:12:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6ceb7fe700 1 --2- 172.21.15.97:0/4254067278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 0x7f6cec06e9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009920 tx=0x7f6cdc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57fcb4a7bfb3e1c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6cea7fc700 1 -- 172.21.15.97:0/4254067278 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00ba20 con 0x7f6cec0b01e0 2023-11-24T06:12:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6cea7fc700 1 -- 172.21.15.97:0/4254067278 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc00bb80 con 0x7f6cec0b01e0 2023-11-24T06:12:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.150+0000 7f6cea7fc700 1 -- 172.21.15.97:0/4254067278 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc009e70 con 0x7f6cec0b01e0 2023-11-24T06:12:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.151+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/4254067278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 msgr2=0x7f6cec06e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.151+0000 7f6cf1f1b700 1 --2- 172.21.15.97:0/4254067278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 0x7f6cec06e9c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009920 tx=0x7f6cdc009c30 comp rx=0 tx=0).stop 2023-11-24T06:12:36.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.151+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/4254067278 shutdown_connections 2023-11-24T06:12:36.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.151+0000 7f6cf1f1b700 1 --2- 172.21.15.97:0/4254067278 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0746a0 0x7f6cec0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.151+0000 7f6cf1f1b700 1 --2- 172.21.15.97:0/4254067278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 0x7f6cec06e9c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.151+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/4254067278 >> 172.21.15.97:0/4254067278 conn(0x7f6cec0d0b40 msgr2=0x7f6cec0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.151+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/4254067278 shutdown_connections 2023-11-24T06:12:36.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.151+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/4254067278 wait complete. 2023-11-24T06:12:36.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.152+0000 7f6cf1f1b700 1 Processor -- start 2023-11-24T06:12:36.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.153+0000 7f6cf1f1b700 1 -- start start 2023-11-24T06:12:36.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.153+0000 7f6cf1f1b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cec0746a0 0x7f6cec144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cf1f1b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 0x7f6cec1424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cf1f1b700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0b59c0 0x7f6cec0ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cf1f1b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec161500 con 0x7f6cec0b01e0 2023-11-24T06:12:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cf1f1b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0cea50 con 0x7f6cec0746a0 2023-11-24T06:12:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cf1f1b700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0ced30 con 0x7f6cec0b59c0 2023-11-24T06:12:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 0x7f6cec1424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 0x7f6cec1424f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51942/0 (socket says 172.21.15.97:51942) 2023-11-24T06:12:36.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 -- 172.21.15.97:0/1929424194 learned_addr learned my addr 172.21.15.97:0/1929424194 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cebfff700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0b59c0 0x7f6cec0ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceb7fe700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cec0746a0 0x7f6cec144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 -- 172.21.15.97:0/1929424194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0b59c0 msgr2=0x7f6cec0ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0b59c0 0x7f6cec0ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 -- 172.21.15.97:0/1929424194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cec0746a0 msgr2=0x7f6cec144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cec0746a0 0x7f6cec144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 -- 172.21.15.97:0/1929424194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc009550 con 0x7f6cec0b01e0 2023-11-24T06:12:36.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cebfff700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6cec0b59c0 0x7f6cec0ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceb7fe700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6cec0746a0 0x7f6cec144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6ceaffd700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 0x7f6cec1424f0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000c450 tx=0x7f6ce000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00052d0 con 0x7f6cec0b01e0 2023-11-24T06:12:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0005430 con 0x7f6cec0b01e0 2023-11-24T06:12:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec0cf010 con 0x7f6cec0b01e0 2023-11-24T06:12:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00038c0 con 0x7f6cec0b01e0 2023-11-24T06:12:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.154+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec142a30 con 0x7f6cec0b01e0 2023-11-24T06:12:36.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.155+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6cec06c270 con 0x7f6cec0b01e0 2023-11-24T06:12:36.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.156+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6ce0003a20 con 0x7f6cec0b01e0 2023-11-24T06:12:36.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.156+0000 7f6cd3fff700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6cd806b500 0x7f6cd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.156+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6ce00892f0 con 0x7f6cec0b01e0 2023-11-24T06:12:36.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.156+0000 7f6ceb7fe700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6cd806b500 0x7f6cd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.156+0000 7f6cd3fff700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f6cd8071360 0x7f6cd80737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.156+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 --> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cd8073e70 con 0x7f6cd8071360 2023-11-24T06:12:36.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.156+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ce0089550 con 0x7f6cec0b01e0 2023-11-24T06:12:36.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.157+0000 7f6cebfff700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f6cd8071360 0x7f6cd80737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.157+0000 7f6ceb7fe700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6cd806b500 0x7f6cd806d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009130 tx=0x7f6cdc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.157+0000 7f6cebfff700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f6cd8071360 0x7f6cd80737a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.157+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 <== osd.3 v2:172.21.15.97:6817/4004566867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6cd8073e70 con 0x7f6cd8071360 2023-11-24T06:12:36.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.172+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 --> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6cec1545e0 con 0x7f6cd8071360 2023-11-24T06:12:36.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.172+0000 7f6cd3fff700 1 -- 172.21.15.97:0/1929424194 <== osd.3 v2:172.21.15.97:6817/4004566867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6ce4003a20 con 0x7f6cd8071360 2023-11-24T06:12:36.175 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:12:36.175 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:36.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f6cd8071360 msgr2=0x7f6cd80737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6817/4004566867,v1:172.21.15.97:6819/4004566867] conn(0x7f6cd8071360 0x7f6cd80737a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6cd806b500 msgr2=0x7f6cd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6cd806b500 0x7f6cd806d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009130 tx=0x7f6cdc008030 comp rx=0 tx=0).stop 2023-11-24T06:12:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 msgr2=0x7f6cec1424f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 --2- 172.21.15.97:0/1929424194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cec0b01e0 0x7f6cec1424f0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000c450 tx=0x7f6ce000c810 comp rx=0 tx=0).stop 2023-11-24T06:12:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cebfff700 1 -- 172.21.15.97:0/1929424194 reap_dead start 2023-11-24T06:12:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 shutdown_connections 2023-11-24T06:12:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 >> 172.21.15.97:0/1929424194 conn(0x7f6cec0d0b40 msgr2=0x7f6cec145180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 shutdown_connections 2023-11-24T06:12:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.173+0000 7f6cf1f1b700 1 -- 172.21.15.97:0/1929424194 wait complete. 2023-11-24T06:12:36.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T06:12:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.305+0000 7f2b3277c700 1 Processor -- start 2023-11-24T06:12:36.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.306+0000 7f2b3277c700 1 -- start start 2023-11-24T06:12:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.306+0000 7f2b3277c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c0adf00 0x7f2b2c06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.306+0000 7f2b3277c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b2c06c540 0x7f2b2c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.306+0000 7f2b3277c700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c0af950 con 0x7f2b2c1446a0 2023-11-24T06:12:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.306+0000 7f2b3277c700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c144a80 con 0x7f2b2c06c540 2023-11-24T06:12:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.306+0000 7f2b3277c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c06c9b0 con 0x7f2b2c0adf00 2023-11-24T06:12:36.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b30d19700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2b2c06c540 0x7f2b2c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b2bfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2b2c1446a0 0x7f2b2c0b5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44402/0 (socket says 172.21.15.97:44402) 2023-11-24T06:12:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b2bfff700 1 -- 172.21.15.97:0/921477113 learned_addr learned my addr 172.21.15.97:0/921477113 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b2b7fe700 1 --2- 172.21.15.97:0/921477113 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c0adf00 0x7f2b2c06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b30d19700 1 -- 172.21.15.97:0/921477113 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c0adf00 msgr2=0x7f2b2c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b30d19700 1 --2- 172.21.15.97:0/921477113 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c0adf00 0x7f2b2c06c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b30d19700 1 -- 172.21.15.97:0/921477113 >> v1:172.21.15.97:6789/0 conn(0x7f2b2c1446a0 legacy=0x7f2b2c0b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b30d19700 1 -- 172.21.15.97:0/921477113 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0afc30 con 0x7f2b2c06c540 2023-11-24T06:12:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b2affd700 1 -- 172.21.15.97:0/921477113 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055751994 0 0) 0x7f2b2c0af950 con 0x7f2b2c1446a0 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b2b7fe700 1 --2- 172.21.15.97:0/921477113 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c0adf00 0x7f2b2c06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b30d19700 1 --2- 172.21.15.97:0/921477113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c06c540 0x7f2b2c0b3d40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009dc0 tx=0x7f2b2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a443db06360bb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b2affd700 1 -- 172.21.15.97:0/921477113 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400e070 con 0x7f2b2c06c540 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b2affd700 1 -- 172.21.15.97:0/921477113 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2400bb90 con 0x7f2b2c06c540 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.307+0000 7f2b2affd700 1 -- 172.21.15.97:0/921477113 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24012430 con 0x7f2b2c06c540 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.308+0000 7f2b3277c700 1 -- 172.21.15.97:0/921477113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c06c540 msgr2=0x7f2b2c0b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.308+0000 7f2b3277c700 1 --2- 172.21.15.97:0/921477113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c06c540 0x7f2b2c0b3d40 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009dc0 tx=0x7f2b2400b330 comp rx=0 tx=0).stop 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.308+0000 7f2b3277c700 1 -- 172.21.15.97:0/921477113 shutdown_connections 2023-11-24T06:12:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.308+0000 7f2b3277c700 1 --2- 172.21.15.97:0/921477113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c06c540 0x7f2b2c0b3d40 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.308+0000 7f2b3277c700 1 --2- 172.21.15.97:0/921477113 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c0adf00 0x7f2b2c06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.308+0000 7f2b3277c700 1 -- 172.21.15.97:0/921477113 >> 172.21.15.97:0/921477113 conn(0x7f2b2c071b90 msgr2=0x7f2b2c071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.308+0000 7f2b3277c700 1 -- 172.21.15.97:0/921477113 shutdown_connections 2023-11-24T06:12:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.308+0000 7f2b3277c700 1 -- 172.21.15.97:0/921477113 wait complete. 2023-11-24T06:12:36.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.309+0000 7f2b3277c700 1 Processor -- start 2023-11-24T06:12:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b3277c700 1 -- start start 2023-11-24T06:12:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b3277c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b2c06c540 0x7f2b2c157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b3277c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c0adf00 0x7f2b2c139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b3277c700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c1446a0 0x7f2b2c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b3277c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0ce530 con 0x7f2b2c0adf00 2023-11-24T06:12:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b3277c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c13a7c0 con 0x7f2b2c06c540 2023-11-24T06:12:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b3277c700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c13aaa0 con 0x7f2b2c1446a0 2023-11-24T06:12:36.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b2b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c0adf00 0x7f2b2c139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b2b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c0adf00 0x7f2b2c139d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51958/0 (socket says 172.21.15.97:51958) 2023-11-24T06:12:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.310+0000 7f2b2b7fe700 1 -- 172.21.15.97:0/660696592 learned_addr learned my addr 172.21.15.97:0/660696592 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b2bfff700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b2c06c540 0x7f2b2c157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b30d19700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c1446a0 0x7f2b2c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b2b7fe700 1 -- 172.21.15.97:0/660696592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c1446a0 msgr2=0x7f2b2c15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b2b7fe700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c1446a0 0x7f2b2c15d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b2b7fe700 1 -- 172.21.15.97:0/660696592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b2c06c540 msgr2=0x7f2b2c157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b2b7fe700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b2c06c540 0x7f2b2c157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b2b7fe700 1 -- 172.21.15.97:0/660696592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b24009a70 con 0x7f2b2c0adf00 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b2b7fe700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c0adf00 0x7f2b2c139d50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e810 tx=0x7f2b2000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b28ff9700 1 -- 172.21.15.97:0/660696592 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20013610 con 0x7f2b2c0adf00 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b28ff9700 1 -- 172.21.15.97:0/660696592 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b20013770 con 0x7f2b2c0adf00 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b28ff9700 1 -- 172.21.15.97:0/660696592 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20013b20 con 0x7f2b2c0adf00 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c13ad80 con 0x7f2b2c0adf00 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.311+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c157da0 con 0x7f2b2c0adf00 2023-11-24T06:12:36.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.312+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c0753f0 con 0x7f2b2c0adf00 2023-11-24T06:12:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.312+0000 7f2b28ff9700 1 -- 172.21.15.97:0/660696592 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2b20013c80 con 0x7f2b2c0adf00 2023-11-24T06:12:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.313+0000 7f2b28ff9700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2b1806b500 0x7f2b1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.313+0000 7f2b28ff9700 1 -- 172.21.15.97:0/660696592 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f2b20089b40 con 0x7f2b2c0adf00 2023-11-24T06:12:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.313+0000 7f2b2bfff700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2b1806b500 0x7f2b1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.313+0000 7f2b2bfff700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2b1806b500 0x7f2b1806d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c000f60 tx=0x7f2b1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.317+0000 7f2b28ff9700 1 -- 172.21.15.97:0/660696592 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b20054510 con 0x7f2b2c0adf00 2023-11-24T06:12:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.509+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b2c0b04d0 con 0x7f2b1806b500 2023-11-24T06:12:36.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.509+0000 7f2b28ff9700 1 -- 172.21.15.97:0/660696592 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2b1c0013a0 con 0x7f2b1806b500 2023-11-24T06:12:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2b1806b500 msgr2=0x7f2b1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2b1806b500 0x7f2b1806d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c000f60 tx=0x7f2b1c009040 comp rx=0 tx=0).stop 2023-11-24T06:12:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c0adf00 msgr2=0x7f2b2c139d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c0adf00 0x7f2b2c139d50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e810 tx=0x7f2b2000ebd0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 shutdown_connections 2023-11-24T06:12:36.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f2b1806b500 0x7f2b1806d9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f2b2c1446a0 0x7f2b2c15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b2c06c540 0x7f2b2c157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 --2- 172.21.15.97:0/660696592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b2c0adf00 0x7f2b2c139d50 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 >> 172.21.15.97:0/660696592 conn(0x7f2b2c071b90 msgr2=0x7f2b2c0b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 shutdown_connections 2023-11-24T06:12:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.511+0000 7f2b3277c700 1 -- 172.21.15.97:0/660696592 wait complete. 2023-11-24T06:12:36.515 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T06:12:36.525 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T06:12:36.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:12:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.648+0000 7f5fc837f700 1 Processor -- start 2023-11-24T06:12:36.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.648+0000 7f5fc837f700 1 -- start start 2023-11-24T06:12:36.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc837f700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fc0143b80 0x7f5fc006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc837f700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f5fc00ce090 0x7f5fc006c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc837f700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc006ee30 con 0x7f5fc0144190 2023-11-24T06:12:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc837f700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc0143f60 con 0x7f5fc0143b80 2023-11-24T06:12:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc837f700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00b3f30 con 0x7f5fc00ce090 2023-11-24T06:12:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc611b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fc0143b80 0x7f5fc006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc611b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fc0143b80 0x7f5fc006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51966/0 (socket says 172.21.15.97:51966) 2023-11-24T06:12:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc611b700 1 -- 172.21.15.97:0/6502236 learned_addr learned my addr 172.21.15.97:0/6502236 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc691c700 1 --1- 172.21.15.97:0/6502236 >> v1:172.21.15.97:6789/0 conn(0x7f5fc0144190 0x7f5fc015e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44420/0 (socket says 172.21.15.97:44420) 2023-11-24T06:12:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc591a700 1 --2- 172.21.15.97:0/6502236 >> v2:172.21.15.183:3300/0 conn(0x7f5fc00ce090 0x7f5fc006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc611b700 1 -- 172.21.15.97:0/6502236 >> v2:172.21.15.183:3300/0 conn(0x7f5fc00ce090 msgr2=0x7f5fc006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc611b700 1 --2- 172.21.15.97:0/6502236 >> v2:172.21.15.183:3300/0 conn(0x7f5fc00ce090 0x7f5fc006c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc611b700 1 -- 172.21.15.97:0/6502236 >> v1:172.21.15.97:6789/0 conn(0x7f5fc0144190 legacy=0x7f5fc015e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc611b700 1 -- 172.21.15.97:0/6502236 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00b4210 con 0x7f5fc0143b80 2023-11-24T06:12:36.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc5119700 1 -- 172.21.15.97:0/6502236 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993910892 0 0) 0x7f5fc006ee30 con 0x7f5fc0144190 2023-11-24T06:12:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.649+0000 7f5fc591a700 1 --2- 172.21.15.97:0/6502236 >> v2:172.21.15.183:3300/0 conn(0x7f5fc00ce090 0x7f5fc006c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:12:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc611b700 1 --2- 172.21.15.97:0/6502236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc0143b80 0x7f5fc006c000 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0071280 tx=0x7f5fb0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad3fe421c66a59e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc5119700 1 -- 172.21.15.97:0/6502236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000b870 con 0x7f5fc0143b80 2023-11-24T06:12:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc5119700 1 -- 172.21.15.97:0/6502236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb000b9d0 con 0x7f5fc0143b80 2023-11-24T06:12:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc5119700 1 -- 172.21.15.97:0/6502236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000bd60 con 0x7f5fc0143b80 2023-11-24T06:12:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc837f700 1 -- 172.21.15.97:0/6502236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc0143b80 msgr2=0x7f5fc006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc837f700 1 --2- 172.21.15.97:0/6502236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc0143b80 0x7f5fc006c000 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0071280 tx=0x7f5fb0009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc837f700 1 -- 172.21.15.97:0/6502236 shutdown_connections 2023-11-24T06:12:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc837f700 1 --2- 172.21.15.97:0/6502236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc0143b80 0x7f5fc006c000 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc837f700 1 --2- 172.21.15.97:0/6502236 >> v2:172.21.15.183:3300/0 conn(0x7f5fc00ce090 0x7f5fc006c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc837f700 1 -- 172.21.15.97:0/6502236 >> 172.21.15.97:0/6502236 conn(0x7f5fc0071b50 msgr2=0x7f5fc0071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc837f700 1 -- 172.21.15.97:0/6502236 shutdown_connections 2023-11-24T06:12:36.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.650+0000 7f5fc837f700 1 -- 172.21.15.97:0/6502236 wait complete. 2023-11-24T06:12:36.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc837f700 1 Processor -- start 2023-11-24T06:12:36.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc837f700 1 -- start start 2023-11-24T06:12:36.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc837f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc00ce090 0x7f5fc01e2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc837f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc0143b80 0x7f5fc01594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc837f700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fc0144190 0x7f5fc0159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc837f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0158690 con 0x7f5fc00ce090 2023-11-24T06:12:36.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc837f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc01e2750 con 0x7f5fc0143b80 2023-11-24T06:12:36.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc837f700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc0157a80 con 0x7f5fc0144190 2023-11-24T06:12:36.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc611b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc00ce090 0x7f5fc01e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc611b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc00ce090 0x7f5fc01e2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51970/0 (socket says 172.21.15.97:51970) 2023-11-24T06:12:36.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.652+0000 7f5fc611b700 1 -- 172.21.15.97:0/826239756 learned_addr learned my addr 172.21.15.97:0/826239756 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:36.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc591a700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc0143b80 0x7f5fc01594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc691c700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fc0144190 0x7f5fc0159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc611b700 1 -- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fc0144190 msgr2=0x7f5fc0159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc611b700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f5fc0144190 0x7f5fc0159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc611b700 1 -- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc0143b80 msgr2=0x7f5fc01594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc611b700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fc0143b80 0x7f5fc01594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc611b700 1 -- 172.21.15.97:0/826239756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb0009550 con 0x7f5fc00ce090 2023-11-24T06:12:36.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc611b700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc00ce090 0x7f5fc01e2210 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009fd0 tx=0x7f5fb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0011860 con 0x7f5fc00ce090 2023-11-24T06:12:36.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb001f070 con 0x7f5fc00ce090 2023-11-24T06:12:36.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc0157d60 con 0x7f5fc00ce090 2023-11-24T06:12:36.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0011bc0 con 0x7f5fc00ce090 2023-11-24T06:12:36.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.653+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc0158320 con 0x7f5fc00ce090 2023-11-24T06:12:36.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.654+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5fc00b4ab0 con 0x7f5fc00ce090 2023-11-24T06:12:36.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.654+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f5fb0021030 con 0x7f5fc00ce090 2023-11-24T06:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.655+0000 7f5fb6ffd700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fa406b500 0x7f5fa406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.655+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f5fb008bb20 con 0x7f5fc00ce090 2023-11-24T06:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.655+0000 7f5fc591a700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fa406b500 0x7f5fa406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.655+0000 7f5fb6ffd700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7f5fa4071360 0x7f5fa40737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.655+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 --> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5fa4073e70 con 0x7f5fa4071360 2023-11-24T06:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.655+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fb008be00 con 0x7f5fc00ce090 2023-11-24T06:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.655+0000 7f5fc691c700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7f5fa4071360 0x7f5fa40737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.664 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.655+0000 7f5fc591a700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fa406b500 0x7f5fa406d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5fc015afb0 tx=0x7f5fbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.656+0000 7f5fc691c700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7f5fa4071360 0x7f5fa40737a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.665 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.656+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 <== osd.4 v2:172.21.15.183:6800/1408518032 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5fa4073e70 con 0x7f5fa4071360 2023-11-24T06:12:36.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.671+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 --> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5fc00b47d0 con 0x7f5fa4071360 2023-11-24T06:12:36.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fb6ffd700 1 -- 172.21.15.97:0/826239756 <== osd.4 v2:172.21.15.183:6800/1408518032 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5fb80070b0 con 0x7f5fa4071360 2023-11-24T06:12:36.675 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:36.675 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:12:36.675 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:36.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7f5fa4071360 msgr2=0x7f5fa40737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc837f700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.183:6800/1408518032,v1:172.21.15.183:6801/1408518032] conn(0x7f5fa4071360 0x7f5fa40737a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fa406b500 msgr2=0x7f5fa406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc837f700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f5fa406b500 0x7f5fa406d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5fc015afb0 tx=0x7f5fbc008040 comp rx=0 tx=0).stop 2023-11-24T06:12:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc00ce090 msgr2=0x7f5fc01e2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc837f700 1 --2- 172.21.15.97:0/826239756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fc00ce090 0x7f5fc01e2210 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009fd0 tx=0x7f5fb0004750 comp rx=0 tx=0).stop 2023-11-24T06:12:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc691c700 1 -- 172.21.15.97:0/826239756 reap_dead start 2023-11-24T06:12:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 shutdown_connections 2023-11-24T06:12:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.672+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 >> 172.21.15.97:0/826239756 conn(0x7f5fc0071b50 msgr2=0x7f5fc00cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.673+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 shutdown_connections 2023-11-24T06:12:36.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.673+0000 7f5fc837f700 1 -- 172.21.15.97:0/826239756 wait complete. 2023-11-24T06:12:36.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T06:12:36.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.805+0000 7fa203457700 1 Processor -- start 2023-11-24T06:12:36.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.805+0000 7fa203457700 1 -- start start 2023-11-24T06:12:36.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa203457700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa203457700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1fc0b2fc0 0x7fa1fc0d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa203457700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc0d2d10 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa203457700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0cefd0 con 0x7fa1fc0b2fc0 2023-11-24T06:12:36.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa203457700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0cf2b0 con 0x7fa1fc0d1570 2023-11-24T06:12:36.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa2019f4700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1fc0b2fc0 0x7fa1fc0d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.810 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa2009f2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa2019f4700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fa1fc0b2fc0 0x7fa1fc0d3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:46144/0 (socket says 172.21.15.97:46144) 2023-11-24T06:12:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.806+0000 7fa2019f4700 1 -- 172.21.15.97:0/1299576404 learned_addr learned my addr 172.21.15.97:0/1299576404 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa2019f4700 1 -- 172.21.15.97:0/1299576404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 msgr2=0x7fa1fc0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa2019f4700 1 --2- 172.21.15.97:0/1299576404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa2019f4700 1 -- 172.21.15.97:0/1299576404 >> v1:172.21.15.183:6789/0 conn(0x7fa1fc15dfc0 legacy=0x7fa1fc0d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa2019f4700 1 -- 172.21.15.97:0/1299576404 --> v2:172.21.15.183:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc0d2fb0 con 0x7fa1fc0b2fc0 2023-11-24T06:12:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa2009f2700 1 --2- 172.21.15.97:0/1299576404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:36.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa2019f4700 1 --2- 172.21.15.97:0/1299576404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 0x7fa1fc0d3a10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009e50 tx=0x7fa1f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1b9eddfe47ad42f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa1f3fff700 1 -- 172.21.15.97:0/1299576404 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400e070 con 0x7fa1fc0b2fc0 2023-11-24T06:12:36.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa1f3fff700 1 -- 172.21.15.97:0/1299576404 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f400b910 con 0x7fa1fc0b2fc0 2023-11-24T06:12:36.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.807+0000 7fa1f3fff700 1 -- 172.21.15.97:0/1299576404 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400bcc0 con 0x7fa1fc0b2fc0 2023-11-24T06:12:36.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.808+0000 7fa203457700 1 -- 172.21.15.97:0/1299576404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 msgr2=0x7fa1fc0d3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.808+0000 7fa203457700 1 --2- 172.21.15.97:0/1299576404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 0x7fa1fc0d3a10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009e50 tx=0x7fa1f400b330 comp rx=0 tx=0).stop 2023-11-24T06:12:36.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.808+0000 7fa203457700 1 -- 172.21.15.97:0/1299576404 shutdown_connections 2023-11-24T06:12:36.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.808+0000 7fa203457700 1 --2- 172.21.15.97:0/1299576404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 0x7fa1fc0d3a10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.808+0000 7fa203457700 1 --2- 172.21.15.97:0/1299576404 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.808+0000 7fa203457700 1 -- 172.21.15.97:0/1299576404 >> 172.21.15.97:0/1299576404 conn(0x7fa1fc06b410 msgr2=0x7fa1fc06b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:36.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.808+0000 7fa203457700 1 -- 172.21.15.97:0/1299576404 shutdown_connections 2023-11-24T06:12:36.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.808+0000 7fa203457700 1 -- 172.21.15.97:0/1299576404 wait complete. 2023-11-24T06:12:36.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.809+0000 7fa203457700 1 Processor -- start 2023-11-24T06:12:36.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.809+0000 7fa203457700 1 -- start start 2023-11-24T06:12:36.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa203457700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 0x7fa1fc1f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa203457700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa203457700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc15dfc0 0x7fa1fc160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa203457700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc15e940 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa203457700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc15f720 con 0x7fa1fc0b2fc0 2023-11-24T06:12:36.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa203457700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1f4910 con 0x7fa1fc0d1570 2023-11-24T06:12:36.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc15dfc0 0x7fa1fc160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc15dfc0 0x7fa1fc160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51990/0 (socket says 172.21.15.97:51990) 2023-11-24T06:12:36.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 -- 172.21.15.97:0/1371412011 learned_addr learned my addr 172.21.15.97:0/1371412011 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:36.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2009f2700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2011f3700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 0x7fa1fc1f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 -- 172.21.15.97:0/1371412011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 msgr2=0x7fa1fc152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 -- 172.21.15.97:0/1371412011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 msgr2=0x7fa1fc1f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:36.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 0x7fa1fc1f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:36.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 -- 172.21.15.97:0/1371412011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f80094a0 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa2019f4700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc15dfc0 0x7fa1fc160250 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4012000 tx=0x7fa1f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.818 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa1f1ffb700 1 -- 172.21.15.97:0/1371412011 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4004800 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa1f1ffb700 1 -- 172.21.15.97:0/1371412011 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f4004960 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.810+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f4009b30 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.811+0000 7fa1f1ffb700 1 -- 172.21.15.97:0/1371412011 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f401c680 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.811+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc15edf0 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.812+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc067f60 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.812+0000 7fa1f1ffb700 1 -- 172.21.15.97:0/1371412011 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fa1f401c7e0 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.813+0000 7fa1f1ffb700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa1e006b500 0x7fa1e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:36.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.813+0000 7fa2011f3700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa1e006b500 0x7fa1e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:36.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.813+0000 7fa1f1ffb700 1 -- 172.21.15.97:0/1371412011 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fa1f408b820 con 0x7fa1fc15dfc0 2023-11-24T06:12:36.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.813+0000 7fa2011f3700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa1e006b500 0x7fa1e006d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc15d3f0 tx=0x7fa1ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:36.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:36.817+0000 7fa1f1ffb700 1 -- 172.21.15.97:0/1371412011 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1f4056170 con 0x7fa1fc15dfc0 2023-11-24T06:12:37.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.008+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1fc15e3a0 con 0x7fa1e006b500 2023-11-24T06:12:37.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.009+0000 7fa1f1ffb700 1 -- 172.21.15.97:0/1371412011 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa1ec001cc0 con 0x7fa1e006b500 2023-11-24T06:12:37.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa1e006b500 msgr2=0x7fa1e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa1e006b500 0x7fa1e006d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc15d3f0 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2023-11-24T06:12:37.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc15dfc0 msgr2=0x7fa1fc160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc15dfc0 0x7fa1fc160250 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4012000 tx=0x7fa1f4004ad0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 shutdown_connections 2023-11-24T06:12:37.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1fc15dfc0 0x7fa1fc160250 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1fc0b2fc0 0x7fa1fc1f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fa1e006b500 0x7fa1e006d9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 --2- 172.21.15.97:0/1371412011 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fa1fc0d1570 0x7fa1fc152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 >> 172.21.15.97:0/1371412011 conn(0x7fa1fc06b410 msgr2=0x7fa1fc06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 shutdown_connections 2023-11-24T06:12:37.016 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.011+0000 7fa203457700 1 -- 172.21.15.97:0/1371412011 wait complete. 2023-11-24T06:12:37.016 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T06:12:37.026 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T06:12:37.026 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:12:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.148+0000 7f6a903c2700 1 Processor -- start 2023-11-24T06:12:37.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.148+0000 7f6a903c2700 1 -- start start 2023-11-24T06:12:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a903c2700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f6a8806f480 0x7f6a8806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a903c2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806e830 0x7f6a8806ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a903c2700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8806f1c0 con 0x7f6a88144600 2023-11-24T06:12:37.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a903c2700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8806aff0 con 0x7f6a8806f480 2023-11-24T06:12:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a903c2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8806b200 con 0x7f6a8806e830 2023-11-24T06:12:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a8e15e700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6a88144600 0x7f6a880b4ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44446/0 (socket says 172.21.15.97:44446) 2023-11-24T06:12:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a8e15e700 1 -- 172.21.15.97:0/3704923168 learned_addr learned my addr 172.21.15.97:0/3704923168 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a8d95d700 1 --2- 172.21.15.97:0/3704923168 >> v2:172.21.15.183:3300/0 conn(0x7f6a8806f480 0x7f6a8806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a8e95f700 1 --2- 172.21.15.97:0/3704923168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806e830 0x7f6a8806ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a8d15c700 1 -- 172.21.15.97:0/3704923168 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049930812 0 0) 0x7f6a84001110 con 0x7f6a88144600 2023-11-24T06:12:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.149+0000 7f6a8d15c700 1 -- 172.21.15.97:0/3704923168 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a740035b0 con 0x7f6a88144600 2023-11-24T06:12:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8e95f700 1 -- 172.21.15.97:0/3704923168 >> v2:172.21.15.183:3300/0 conn(0x7f6a8806f480 msgr2=0x7f6a8806c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8e95f700 1 --2- 172.21.15.97:0/3704923168 >> v2:172.21.15.183:3300/0 conn(0x7f6a8806f480 0x7f6a8806c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8e95f700 1 -- 172.21.15.97:0/3704923168 >> v1:172.21.15.97:6789/0 conn(0x7f6a88144600 legacy=0x7f6a880b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8e95f700 1 -- 172.21.15.97:0/3704923168 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8806b4e0 con 0x7f6a8806e830 2023-11-24T06:12:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8d95d700 1 --2- 172.21.15.97:0/3704923168 >> v2:172.21.15.183:3300/0 conn(0x7f6a8806f480 0x7f6a8806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:12:37.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8e95f700 1 --2- 172.21.15.97:0/3704923168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806e830 0x7f6a8806ec80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009d90 tx=0x7f6a8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9d0db10767c863e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8d15c700 1 -- 172.21.15.97:0/3704923168 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000e070 con 0x7f6a8806e830 2023-11-24T06:12:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8d15c700 1 -- 172.21.15.97:0/3704923168 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80003a20 con 0x7f6a8806e830 2023-11-24T06:12:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.150+0000 7f6a8d15c700 1 -- 172.21.15.97:0/3704923168 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84001800 con 0x7f6a8806e830 2023-11-24T06:12:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.151+0000 7f6a903c2700 1 -- 172.21.15.97:0/3704923168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806e830 msgr2=0x7f6a8806ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.151+0000 7f6a903c2700 1 --2- 172.21.15.97:0/3704923168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806e830 0x7f6a8806ec80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009d90 tx=0x7f6a8000b330 comp rx=0 tx=0).stop 2023-11-24T06:12:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.151+0000 7f6a903c2700 1 -- 172.21.15.97:0/3704923168 shutdown_connections 2023-11-24T06:12:37.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.151+0000 7f6a903c2700 1 --2- 172.21.15.97:0/3704923168 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806e830 0x7f6a8806ec80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.151+0000 7f6a903c2700 1 --2- 172.21.15.97:0/3704923168 >> v2:172.21.15.183:3300/0 conn(0x7f6a8806f480 0x7f6a8806c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.151+0000 7f6a903c2700 1 -- 172.21.15.97:0/3704923168 >> 172.21.15.97:0/3704923168 conn(0x7f6a880d0b40 msgr2=0x7f6a880d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.151+0000 7f6a903c2700 1 -- 172.21.15.97:0/3704923168 shutdown_connections 2023-11-24T06:12:37.157 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.151+0000 7f6a903c2700 1 -- 172.21.15.97:0/3704923168 wait complete. 2023-11-24T06:12:37.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.152+0000 7f6a903c2700 1 Processor -- start 2023-11-24T06:12:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.152+0000 7f6a903c2700 1 -- start start 2023-11-24T06:12:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a903c2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8806e830 0x7f6a88159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a903c2700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806f480 0x7f6a8813a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a903c2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a88144600 0x7f6a88073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a903c2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8813aad0 con 0x7f6a8806e830 2023-11-24T06:12:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a903c2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8813adb0 con 0x7f6a88144600 2023-11-24T06:12:37.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a903c2700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8813b090 con 0x7f6a8806f480 2023-11-24T06:12:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8806e830 0x7f6a88159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8806e830 0x7f6a88159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52000/0 (socket says 172.21.15.97:52000) 2023-11-24T06:12:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 -- 172.21.15.97:0/1568529331 learned_addr learned my addr 172.21.15.97:0/1568529331 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 -- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806f480 msgr2=0x7f6a8813a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f6a8806f480 0x7f6a8813a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 -- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a88144600 msgr2=0x7f6a88073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a88144600 0x7f6a88073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 -- 172.21.15.97:0/1568529331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a780094a0 con 0x7f6a8806e830 2023-11-24T06:12:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a8e15e700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8806e830 0x7f6a88159d10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6a84001930 tx=0x7f6a8400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84004c80 con 0x7f6a8806e830 2023-11-24T06:12:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a84004de0 con 0x7f6a8806e830 2023-11-24T06:12:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a80009a70 con 0x7f6a8806e830 2023-11-24T06:12:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84007310 con 0x7f6a8806e830 2023-11-24T06:12:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.153+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8815a250 con 0x7f6a8806e830 2023-11-24T06:12:37.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.154+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a8806bd80 con 0x7f6a8806e830 2023-11-24T06:12:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6a84007470 con 0x7f6a8806e830 2023-11-24T06:12:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a7effd700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a6c06b500 0x7f6a6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6a840897e0 con 0x7f6a8806e830 2023-11-24T06:12:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a8d95d700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a6c06b500 0x7f6a6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a7effd700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f6a6c071360 0x7f6a6c0737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 --> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a6c073e70 con 0x7f6a6c071360 2023-11-24T06:12:37.164 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a84089ac0 con 0x7f6a8806e830 2023-11-24T06:12:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a8e95f700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f6a6c071360 0x7f6a6c0737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.155+0000 7f6a8d95d700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a6c06b500 0x7f6a6c06d9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006f10 tx=0x7f6a78006e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.156+0000 7f6a8e95f700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f6a6c071360 0x7f6a6c0737a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.165 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.157+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 <== osd.5 v2:172.21.15.183:6817/4148254452 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6a6c073e70 con 0x7f6a6c071360 2023-11-24T06:12:37.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.171+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 --> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6a8806baa0 con 0x7f6a6c071360 2023-11-24T06:12:37.174 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a7effd700 1 -- 172.21.15.97:0/1568529331 <== osd.5 v2:172.21.15.183:6817/4148254452 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6a80003e70 con 0x7f6a6c071360 2023-11-24T06:12:37.175 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:37.175 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:12:37.175 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:37.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f6a6c071360 msgr2=0x7f6a6c0737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.183:6817/4148254452,v1:172.21.15.183:6819/4148254452] conn(0x7f6a6c071360 0x7f6a6c0737a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a6c06b500 msgr2=0x7f6a6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f6a6c06b500 0x7f6a6c06d9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6a78006f10 tx=0x7f6a78006e60 comp rx=0 tx=0).stop 2023-11-24T06:12:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8806e830 msgr2=0x7f6a88159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 --2- 172.21.15.97:0/1568529331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a8806e830 0x7f6a88159d10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6a84001930 tx=0x7f6a8400ca60 comp rx=0 tx=0).stop 2023-11-24T06:12:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a8e95f700 1 -- 172.21.15.97:0/1568529331 reap_dead start 2023-11-24T06:12:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 shutdown_connections 2023-11-24T06:12:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 >> 172.21.15.97:0/1568529331 conn(0x7f6a880d0b40 msgr2=0x7f6a880ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.172+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 shutdown_connections 2023-11-24T06:12:37.177 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.173+0000 7f6a903c2700 1 -- 172.21.15.97:0/1568529331 wait complete. 2023-11-24T06:12:37.187 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T06:12:37.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.304+0000 7f7948fef700 1 Processor -- start 2023-11-24T06:12:37.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.304+0000 7f7948fef700 1 -- start start 2023-11-24T06:12:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7948fef700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79440ce140 0x7f794415f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7948fef700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79441414c0 con 0x7f794415af10 2023-11-24T06:12:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7948fef700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79440ce9f0 con 0x7f794415f630 2023-11-24T06:12:37.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7948fef700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794415c300 con 0x7f79440ce140 2023-11-24T06:12:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7941d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79440ce140 0x7f794415f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7941d9b700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f79440ce140 0x7f794415f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52012/0 (socket says 172.21.15.97:52012) 2023-11-24T06:12:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7941d9b700 1 -- 172.21.15.97:0/2528924246 learned_addr learned my addr 172.21.15.97:0/2528924246 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7941d9b700 1 -- 172.21.15.97:0/2528924246 >> v1:172.21.15.183:6789/0 conn(0x7f794415f630 legacy=0x7f794406b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7941d9b700 1 -- 172.21.15.97:0/2528924246 >> v1:172.21.15.97:6789/0 conn(0x7f794415af10 legacy=0x7f794415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7941d9b700 1 -- 172.21.15.97:0/2528924246 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440cecd0 con 0x7f79440ce140 2023-11-24T06:12:37.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f794159a700 1 -- 172.21.15.97:0/2528924246 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776370937 0 0) 0x7f79441414c0 con 0x7f794415af10 2023-11-24T06:12:37.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f7941d9b700 1 --2- 172.21.15.97:0/2528924246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f794415f0f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f79340098f0 tx=0x7f7934009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7023b9bf3262297b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f794159a700 1 -- 172.21.15.97:0/2528924246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934004230 con 0x7f79440ce140 2023-11-24T06:12:37.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f794159a700 1 -- 172.21.15.97:0/2528924246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934004390 con 0x7f79440ce140 2023-11-24T06:12:37.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.306+0000 7f794159a700 1 -- 172.21.15.97:0/2528924246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79340036a0 con 0x7f79440ce140 2023-11-24T06:12:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.307+0000 7f7948fef700 1 -- 172.21.15.97:0/2528924246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 msgr2=0x7f794415f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.307+0000 7f7948fef700 1 --2- 172.21.15.97:0/2528924246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f794415f0f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f79340098f0 tx=0x7f7934009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:37.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.307+0000 7f7948fef700 1 -- 172.21.15.97:0/2528924246 shutdown_connections 2023-11-24T06:12:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.307+0000 7f7948fef700 1 --2- 172.21.15.97:0/2528924246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f794415f0f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.307+0000 7f7948fef700 1 -- 172.21.15.97:0/2528924246 >> 172.21.15.97:0/2528924246 conn(0x7f79440cd570 msgr2=0x7f79440cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.307+0000 7f7948fef700 1 -- 172.21.15.97:0/2528924246 shutdown_connections 2023-11-24T06:12:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.307+0000 7f7948fef700 1 -- 172.21.15.97:0/2528924246 wait complete. 2023-11-24T06:12:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.309+0000 7f7948fef700 1 Processor -- start 2023-11-24T06:12:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.309+0000 7f7948fef700 1 -- start start 2023-11-24T06:12:37.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7948fef700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f79441401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7948fef700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f794415af10 0x7f7944159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7948fef700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7944158500 0x7f7944143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7948fef700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944158d10 con 0x7f79440ce140 2023-11-24T06:12:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7948fef700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7944158ff0 con 0x7f794415af10 2023-11-24T06:12:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7948fef700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79441592d0 con 0x7f7944158500 2023-11-24T06:12:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f79441401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f79441401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52016/0 (socket says 172.21.15.97:52016) 2023-11-24T06:12:37.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 -- 172.21.15.97:0/937811216 learned_addr learned my addr 172.21.15.97:0/937811216 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:37.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7941d9b700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f794415af10 0x7f7944159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7942d9d700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7944158500 0x7f7944143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 -- 172.21.15.97:0/937811216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7944158500 msgr2=0x7f7944143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7944158500 0x7f7944143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 -- 172.21.15.97:0/937811216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f794415af10 msgr2=0x7f7944159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f794415af10 0x7f7944159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 -- 172.21.15.97:0/937811216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7934009550 con 0x7f79440ce140 2023-11-24T06:12:37.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f794259c700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f79441401f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f7938001eb0 tx=0x7f793800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7932ffd700 1 -- 172.21.15.97:0/937811216 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380054e0 con 0x7f79440ce140 2023-11-24T06:12:37.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7932ffd700 1 -- 172.21.15.97:0/937811216 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7938003680 con 0x7f79440ce140 2023-11-24T06:12:37.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79441595b0 con 0x7f79440ce140 2023-11-24T06:12:37.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.310+0000 7f7932ffd700 1 -- 172.21.15.97:0/937811216 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380039f0 con 0x7f79440ce140 2023-11-24T06:12:37.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.311+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7944158990 con 0x7f79440ce140 2023-11-24T06:12:37.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.311+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794415a2d0 con 0x7f79440ce140 2023-11-24T06:12:37.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.312+0000 7f7932ffd700 1 -- 172.21.15.97:0/937811216 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7938003b50 con 0x7f79440ce140 2023-11-24T06:12:37.319 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.312+0000 7f7932ffd700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f792406b500 0x7f792406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.312+0000 7f7941d9b700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f792406b500 0x7f792406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.312+0000 7f7932ffd700 1 -- 172.21.15.97:0/937811216 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f7938089260 con 0x7f79440ce140 2023-11-24T06:12:37.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.313+0000 7f7941d9b700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f792406b500 0x7f792406d9c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f79440ce540 tx=0x7f7934008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.320 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.317+0000 7f7932ffd700 1 -- 172.21.15.97:0/937811216 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7938053d60 con 0x7f79440ce140 2023-11-24T06:12:37.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.507+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f79440cefb0 con 0x7f792406b500 2023-11-24T06:12:37.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.507+0000 7f7932ffd700 1 -- 172.21.15.97:0/937811216 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f79340041b0 con 0x7f792406b500 2023-11-24T06:12:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f792406b500 msgr2=0x7f792406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f792406b500 0x7f792406d9c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f79440ce540 tx=0x7f7934008030 comp rx=0 tx=0).stop 2023-11-24T06:12:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 msgr2=0x7f79441401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f79441401f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f7938001eb0 tx=0x7f793800c8b0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 shutdown_connections 2023-11-24T06:12:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f792406b500 0x7f792406d9c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f7944158500 0x7f7944143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79440ce140 0x7f79441401f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 --2- 172.21.15.97:0/937811216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f794415af10 0x7f7944159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 >> 172.21.15.97:0/937811216 conn(0x7f79440cd570 msgr2=0x7f794406eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 shutdown_connections 2023-11-24T06:12:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.509+0000 7f7948fef700 1 -- 172.21.15.97:0/937811216 wait complete. 2023-11-24T06:12:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T06:12:37.527 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T06:12:37.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:12:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.648+0000 7f15493d5700 1 Processor -- start 2023-11-24T06:12:37.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.648+0000 7f15493d5700 1 -- start start 2023-11-24T06:12:37.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.648+0000 7f15493d5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 0x7f15440ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.648+0000 7f15493d5700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15440cf050 0x7f15440d2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.648+0000 7f15493d5700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15440d21a0 con 0x7f154415e150 2023-11-24T06:12:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.648+0000 7f15493d5700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154415e530 con 0x7f15440cf050 2023-11-24T06:12:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.648+0000 7f15493d5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15440cf4c0 con 0x7f15440b9a40 2023-11-24T06:12:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15437fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15440cf050 0x7f15440d2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15437fe700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15440cf050 0x7f15440d2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52028/0 (socket says 172.21.15.97:52028) 2023-11-24T06:12:37.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15437fe700 1 -- 172.21.15.97:0/129357040 learned_addr learned my addr 172.21.15.97:0/129357040 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15427fc700 1 --2- 172.21.15.97:0/129357040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 0x7f15440ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15437fe700 1 -- 172.21.15.97:0/129357040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 msgr2=0x7f15440ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15437fe700 1 --2- 172.21.15.97:0/129357040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 0x7f15440ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15437fe700 1 -- 172.21.15.97:0/129357040 >> v1:172.21.15.183:6789/0 conn(0x7f154415e150 legacy=0x7f1544161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15437fe700 1 -- 172.21.15.97:0/129357040 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544162390 con 0x7f15440cf050 2023-11-24T06:12:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f15437fe700 1 --2- 172.21.15.97:0/129357040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15440cf050 0x7f15440d2910 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f153c009e80 tx=0x7f153c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0095273a7e6b83d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f1541ffb700 1 -- 172.21.15.97:0/129357040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c00e070 con 0x7f15440cf050 2023-11-24T06:12:37.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f1541ffb700 1 -- 172.21.15.97:0/129357040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c003a20 con 0x7f15440cf050 2023-11-24T06:12:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.649+0000 7f1541ffb700 1 -- 172.21.15.97:0/129357040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c003dd0 con 0x7f15440cf050 2023-11-24T06:12:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.650+0000 7f15493d5700 1 -- 172.21.15.97:0/129357040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15440cf050 msgr2=0x7f15440d2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.650+0000 7f15493d5700 1 --2- 172.21.15.97:0/129357040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15440cf050 0x7f15440d2910 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f153c009e80 tx=0x7f153c00b330 comp rx=0 tx=0).stop 2023-11-24T06:12:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.650+0000 7f15493d5700 1 -- 172.21.15.97:0/129357040 shutdown_connections 2023-11-24T06:12:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.650+0000 7f15493d5700 1 --2- 172.21.15.97:0/129357040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15440cf050 0x7f15440d2910 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.650+0000 7f15493d5700 1 --2- 172.21.15.97:0/129357040 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 0x7f15440ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.650+0000 7f15493d5700 1 -- 172.21.15.97:0/129357040 >> 172.21.15.97:0/129357040 conn(0x7f154406b620 msgr2=0x7f1544071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.650+0000 7f15493d5700 1 -- 172.21.15.97:0/129357040 shutdown_connections 2023-11-24T06:12:37.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.650+0000 7f15493d5700 1 -- 172.21.15.97:0/129357040 wait complete. 2023-11-24T06:12:37.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.651+0000 7f15493d5700 1 Processor -- start 2023-11-24T06:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.651+0000 7f15493d5700 1 -- start start 2023-11-24T06:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15493d5700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 0x7f1544156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15493d5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15440cf050 0x7f1544161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15493d5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154415e150 0x7f1544160100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15493d5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15441568b0 con 0x7f154415e150 2023-11-24T06:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15493d5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154415f5f0 con 0x7f15440cf050 2023-11-24T06:12:37.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15493d5700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154415f8d0 con 0x7f15440b9a40 2023-11-24T06:12:37.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154415e150 0x7f1544160100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154415e150 0x7f1544160100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52034/0 (socket says 172.21.15.97:52034) 2023-11-24T06:12:37.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 -- 172.21.15.97:0/2286053494 learned_addr learned my addr 172.21.15.97:0/2286053494 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:37.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15427fc700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15440cf050 0x7f1544161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f1542ffd700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 0x7f1544156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 -- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 msgr2=0x7f1544156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 0x7f1544156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.660 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 -- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15440cf050 msgr2=0x7f1544161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15440cf050 0x7f1544161070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 -- 172.21.15.97:0/2286053494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c009b30 con 0x7f154415e150 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f1542ffd700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f15440b9a40 0x7f1544156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15427fc700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15440cf050 0x7f1544161070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.652+0000 7f15437fe700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154415e150 0x7f1544160100 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f15440b6a30 tx=0x7f153c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.653+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c004240 con 0x7f154415e150 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.653+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c0043e0 con 0x7f154415e150 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.653+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154415fbb0 con 0x7f154415e150 2023-11-24T06:12:37.661 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.653+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c004790 con 0x7f154415e150 2023-11-24T06:12:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.653+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15440caf10 con 0x7f154415e150 2023-11-24T06:12:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.653+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15440d23f0 con 0x7f154415e150 2023-11-24T06:12:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.654+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f153c019070 con 0x7f154415e150 2023-11-24T06:12:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.654+0000 7f152b7fe700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f153006b500 0x7f153006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.655+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f153c08b360 con 0x7f154415e150 2023-11-24T06:12:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.655+0000 7f1542ffd700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f153006b500 0x7f153006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.662 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.655+0000 7f152b7fe700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f1530071360 0x7f15300737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.655+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 --> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1530073e70 con 0x7f1530071360 2023-11-24T06:12:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.655+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f153c08b730 con 0x7f154415e150 2023-11-24T06:12:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.655+0000 7f1542ffd700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f153006b500 0x7f153006d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1534009930 tx=0x7f1534008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.655+0000 7f15427fc700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f1530071360 0x7f15300737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.655+0000 7f15427fc700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f1530071360 0x7f15300737a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.663 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.656+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 <== osd.6 v2:172.21.15.183:6816/3075593088 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1530073e70 con 0x7f1530071360 2023-11-24T06:12:37.673 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.671+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 --> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1544162950 con 0x7f1530071360 2023-11-24T06:12:37.674 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f152b7fe700 1 -- 172.21.15.97:0/2286053494 <== osd.6 v2:172.21.15.183:6816/3075593088 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f153800abc0 con 0x7f1530071360 2023-11-24T06:12:37.674 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:37.675 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:12:37.675 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f1530071360 msgr2=0x7f15300737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.183:6816/3075593088,v1:172.21.15.183:6818/3075593088] conn(0x7f1530071360 0x7f15300737a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f153006b500 msgr2=0x7f153006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f153006b500 0x7f153006d9c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1534009930 tx=0x7f1534008040 comp rx=0 tx=0).stop 2023-11-24T06:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154415e150 msgr2=0x7f1544160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 --2- 172.21.15.97:0/2286053494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f154415e150 0x7f1544160100 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f15440b6a30 tx=0x7f153c004ad0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15437fe700 1 -- 172.21.15.97:0/2286053494 reap_dead start 2023-11-24T06:12:37.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 shutdown_connections 2023-11-24T06:12:37.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 >> 172.21.15.97:0/2286053494 conn(0x7f154406b620 msgr2=0x7f15440b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 shutdown_connections 2023-11-24T06:12:37.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.672+0000 7f15493d5700 1 -- 172.21.15.97:0/2286053494 wait complete. 2023-11-24T06:12:37.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T06:12:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.805+0000 7f0239fd9700 1 Processor -- start 2023-11-24T06:12:37.808 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.806+0000 7f0239fd9700 1 -- start start 2023-11-24T06:12:37.811 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.806+0000 7f0239fd9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f023406c710 0x7f02340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.806+0000 7f0239fd9700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02340ae430 con 0x7f023415e080 2023-11-24T06:12:37.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.806+0000 7f0239fd9700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02340ae710 con 0x7f02340af4d0 2023-11-24T06:12:37.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.806+0000 7f0239fd9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340af0b0 con 0x7f023406c710 2023-11-24T06:12:37.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.806+0000 7f02337fe700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f023415e080 0x7f023406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44492/0 (socket says 172.21.15.97:44492) 2023-11-24T06:12:37.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.806+0000 7f02337fe700 1 -- 172.21.15.97:0/2819515903 learned_addr learned my addr 172.21.15.97:0/2819515903 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:37.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f0232ffd700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f023406c710 0x7f02340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.812 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493472319 0 0) 0x7f02340ae430 con 0x7f023415e080 2023-11-24T06:12:37.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02200035b0 con 0x7f023415e080 2023-11-24T06:12:37.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 <== mon.1 v1:172.21.15.183:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201828493 0 0) 0x7f02340ae710 con 0x7f02340af4d0 2023-11-24T06:12:37.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 --> v1:172.21.15.183:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02340ae430 con 0x7f02340af4d0 2023-11-24T06:12:37.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2265018312 0 0) 0x7f02200035b0 con 0x7f023415e080 2023-11-24T06:12:37.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f023406c710 msgr2=0x7f02340b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f023406c710 0x7f02340b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 >> v1:172.21.15.183:6789/0 conn(0x7f02340af4d0 legacy=0x7f023406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.813 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02340ae9f0 con 0x7f023415e080 2023-11-24T06:12:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 <== mon.0 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 260110518 0 0) 0x7f021c001670 con 0x7f023415e080 2023-11-24T06:12:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 >> v1:172.21.15.97:6789/0 conn(0x7f023415e080 legacy=0x7f023406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f023406c710 0x7f0220003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f0220004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0220004d90 0x7f0220005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340af0b0 con 0x7f0220004120 2023-11-24T06:12:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022000baf0 con 0x7f0220004d90 2023-11-24T06:12:37.814 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022000bcb0 con 0x7f023406c710 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f0232ffd700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f0220004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f02337fe700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f023406c710 0x7f0220003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f0233fff700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0220004d90 0x7f0220005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f0232ffd700 1 -- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f023406c710 msgr2=0x7f0220003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f0232ffd700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f023406c710 0x7f0220003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f0232ffd700 1 -- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0220004d90 msgr2=0x7f0220005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f0232ffd700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0220004d90 0x7f0220005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.807+0000 7f0232ffd700 1 -- 172.21.15.97:0/2819515903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02200035b0 con 0x7f0220004120 2023-11-24T06:12:37.815 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f0232ffd700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f0220004850 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0224009b60 tx=0x7f0220005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f75e163aa9cddc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022400e6c0 con 0x7f0220004120 2023-11-24T06:12:37.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022400e820 con 0x7f0220004120 2023-11-24T06:12:37.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f02327fc700 1 -- 172.21.15.97:0/2819515903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022400eb90 con 0x7f0220004120 2023-11-24T06:12:37.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f0239fd9700 1 -- 172.21.15.97:0/2819515903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 msgr2=0x7f0220004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f0239fd9700 1 --2- 172.21.15.97:0/2819515903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f0220004850 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0224009b60 tx=0x7f0220005880 comp rx=0 tx=0).stop 2023-11-24T06:12:37.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f0233fff700 1 -- 172.21.15.97:0/2819515903 reap_dead start 2023-11-24T06:12:37.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f0239fd9700 1 -- 172.21.15.97:0/2819515903 shutdown_connections 2023-11-24T06:12:37.816 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f0239fd9700 1 -- 172.21.15.97:0/2819515903 >> 172.21.15.97:0/2819515903 conn(0x7f0234071b90 msgr2=0x7f0234071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:37.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f0239fd9700 1 -- 172.21.15.97:0/2819515903 shutdown_connections 2023-11-24T06:12:37.817 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.808+0000 7f0239fd9700 1 -- 172.21.15.97:0/2819515903 wait complete. 2023-11-24T06:12:37.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.810+0000 7f0239fd9700 1 Processor -- start 2023-11-24T06:12:37.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.810+0000 7f0239fd9700 1 -- start start 2023-11-24T06:12:37.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.810+0000 7f0239fd9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f02341420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.819 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.810+0000 7f0239fd9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02341452c0 0x7f0234160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.810+0000 7f0239fd9700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02340b1820 0x7f02340b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.810+0000 7f0239fd9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0234142620 con 0x7f0220004120 2023-11-24T06:12:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.810+0000 7f0239fd9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0234160910 con 0x7f02341452c0 2023-11-24T06:12:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.810+0000 7f0239fd9700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340b21e0 con 0x7f02340b1820 2023-11-24T06:12:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f02341420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f02341420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52052/0 (socket says 172.21.15.97:52052) 2023-11-24T06:12:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 -- 172.21.15.97:0/1641282738 learned_addr learned my addr 172.21.15.97:0/1641282738 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:37.820 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f0233fff700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02340b1820 0x7f02340b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 -- 172.21.15.97:0/1641282738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02340b1820 msgr2=0x7f02340b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02340b1820 0x7f02340b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f0232ffd700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02341452c0 0x7f0234160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 -- 172.21.15.97:0/1641282738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02341452c0 msgr2=0x7f0234160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02341452c0 0x7f0234160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 -- 172.21.15.97:0/1641282738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023415e8b0 con 0x7f0220004120 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f02337fe700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f02341420e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f021c007340 tx=0x7f021c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f022bfff700 1 -- 172.21.15.97:0/1641282738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c00f660 con 0x7f0220004120 2023-11-24T06:12:37.821 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f022bfff700 1 -- 172.21.15.97:0/1641282738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c00f7c0 con 0x7f0220004120 2023-11-24T06:12:37.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f022bfff700 1 -- 172.21.15.97:0/1641282738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c00fb70 con 0x7f0220004120 2023-11-24T06:12:37.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02341602c0 con 0x7f0220004120 2023-11-24T06:12:37.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.811+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0234151be0 con 0x7f0220004120 2023-11-24T06:12:37.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.812+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02340753f0 con 0x7f0220004120 2023-11-24T06:12:37.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.813+0000 7f022bfff700 1 -- 172.21.15.97:0/1641282738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f021c00fcd0 con 0x7f0220004120 2023-11-24T06:12:37.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.813+0000 7f022bfff700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f021406b500 0x7f021406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:37.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.813+0000 7f022bfff700 1 -- 172.21.15.97:0/1641282738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f021c089e50 con 0x7f0220004120 2023-11-24T06:12:37.822 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.813+0000 7f0232ffd700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f021406b500 0x7f021406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:37.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.813+0000 7f0232ffd700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f021406b500 0x7f021406d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f023415a250 tx=0x7f0224008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:37.823 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:37.818+0000 7f022bfff700 1 -- 172.21.15.97:0/1641282738 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f021c0548a0 con 0x7f0220004120 2023-11-24T06:12:38.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.008+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f0234160a50 con 0x7f021406b500 2023-11-24T06:12:38.011 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.008+0000 7f022bfff700 1 -- 172.21.15.97:0/1641282738 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f022400e990 con 0x7f021406b500 2023-11-24T06:12:38.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f021406b500 msgr2=0x7f021406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.013 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f021406b500 0x7f021406d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f023415a250 tx=0x7f0224008030 comp rx=0 tx=0).stop 2023-11-24T06:12:38.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 msgr2=0x7f02341420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f02341420e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f021c007340 tx=0x7f021c007650 comp rx=0 tx=0).stop 2023-11-24T06:12:38.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 shutdown_connections 2023-11-24T06:12:38.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f021406b500 0x7f021406d9c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f02340b1820 0x7f02340b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0220004120 0x7f02341420e0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.014 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 --2- 172.21.15.97:0/1641282738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02341452c0 0x7f0234160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.010+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 >> 172.21.15.97:0/1641282738 conn(0x7f0234071b90 msgr2=0x7f023406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.011+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 shutdown_connections 2023-11-24T06:12:38.015 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.011+0000 7f0239fd9700 1 -- 172.21.15.97:0/1641282738 wait complete. 2023-11-24T06:12:38.015 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T06:12:38.025 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T06:12:38.025 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:12:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.147+0000 7f489a73e700 1 Processor -- start 2023-11-24T06:12:38.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.147+0000 7f489a73e700 1 -- start start 2023-11-24T06:12:38.152 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f489a73e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 0x7f489406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f489a73e700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f48940cd810 0x7f489406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f489a73e700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48940cf530 con 0x7f4894073f90 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f489a73e700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489406b8f0 con 0x7f48940cd810 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f489a73e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489406bbd0 con 0x7f4894145620 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f4898cdb700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4894073f90 0x7f48940cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44516/0 (socket says 172.21.15.97:44516) 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f4898cdb700 1 -- 172.21.15.97:0/1869645579 learned_addr learned my addr 172.21.15.97:0/1869645579 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f4893fff700 1 --2- 172.21.15.97:0/1869645579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 0x7f489406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.153 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f48937fe700 1 --2- 172.21.15.97:0/1869645579 >> v2:172.21.15.183:3300/0 conn(0x7f48940cd810 0x7f489406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f4892ffd700 1 -- 172.21.15.97:0/1869645579 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995566617 0 0) 0x7f48940cf530 con 0x7f4894073f90 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.148+0000 7f4892ffd700 1 -- 172.21.15.97:0/1869645579 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c0035b0 con 0x7f4894073f90 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4892ffd700 1 -- 172.21.15.97:0/1869645579 <== mon.0 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 964944567 0 0) 0x7f487c0035b0 con 0x7f4894073f90 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4893fff700 1 -- 172.21.15.97:0/1869645579 >> v2:172.21.15.183:3300/0 conn(0x7f48940cd810 msgr2=0x7f489406aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4893fff700 1 --2- 172.21.15.97:0/1869645579 >> v2:172.21.15.183:3300/0 conn(0x7f48940cd810 0x7f489406aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4893fff700 1 -- 172.21.15.97:0/1869645579 >> v1:172.21.15.97:6789/0 conn(0x7f4894073f90 legacy=0x7f48940cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4893fff700 1 -- 172.21.15.97:0/1869645579 --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48940cf1c0 con 0x7f4894145620 2023-11-24T06:12:38.154 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4893fff700 1 --2- 172.21.15.97:0/1869645579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 0x7f489406dc00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4888009840 tx=0x7f4888009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=716fdc5508172de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4892ffd700 1 -- 172.21.15.97:0/1869645579 <== mon.2 v2:172.21.15.183:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0019f0 con 0x7f4894145620 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4892ffd700 1 -- 172.21.15.97:0/1869645579 <== mon.2 v2:172.21.15.183:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4888004030 con 0x7f4894145620 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f4892ffd700 1 -- 172.21.15.97:0/1869645579 <== mon.2 v2:172.21.15.183:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48880043a0 con 0x7f4894145620 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f489a73e700 1 -- 172.21.15.97:0/1869645579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 msgr2=0x7f489406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f489a73e700 1 --2- 172.21.15.97:0/1869645579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 0x7f489406dc00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4888009840 tx=0x7f4888009b50 comp rx=0 tx=0).stop 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f489a73e700 1 -- 172.21.15.97:0/1869645579 shutdown_connections 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f489a73e700 1 --2- 172.21.15.97:0/1869645579 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 0x7f489406dc00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.155 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f489a73e700 1 --2- 172.21.15.97:0/1869645579 >> v2:172.21.15.183:3300/0 conn(0x7f48940cd810 0x7f489406aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.149+0000 7f489a73e700 1 -- 172.21.15.97:0/1869645579 >> 172.21.15.97:0/1869645579 conn(0x7f48940d0b40 msgr2=0x7f48940d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.150+0000 7f489a73e700 1 -- 172.21.15.97:0/1869645579 shutdown_connections 2023-11-24T06:12:38.156 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.150+0000 7f489a73e700 1 -- 172.21.15.97:0/1869645579 wait complete. 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.151+0000 7f489a73e700 1 Processor -- start 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.151+0000 7f489a73e700 1 -- start start 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f489a73e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4894073f90 0x7f4894156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f489a73e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48940cd810 0x7f4894143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f489a73e700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 0x7f48940ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f489a73e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b3e90 con 0x7f4894073f90 2023-11-24T06:12:38.159 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f489a73e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b4410 con 0x7f48940cd810 2023-11-24T06:12:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f489a73e700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4894143be0 con 0x7f4894145620 2023-11-24T06:12:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4894073f90 0x7f4894156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4894073f90 0x7f4894156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52066/0 (socket says 172.21.15.97:52066) 2023-11-24T06:12:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 -- 172.21.15.97:0/1668024888 learned_addr learned my addr 172.21.15.97:0/1668024888 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 -- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 msgr2=0x7f48940ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f4894145620 0x7f48940ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 -- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48940cd810 msgr2=0x7f4894143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.160 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48940cd810 0x7f4894143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 -- 172.21.15.97:0/1668024888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48880094a0 con 0x7f4894073f90 2023-11-24T06:12:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4893fff700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4894073f90 0x7f4894156990 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4888009b90 tx=0x7f4888004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888003df0 con 0x7f4894073f90 2023-11-24T06:12:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488801f070 con 0x7f4894073f90 2023-11-24T06:12:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4894143ec0 con 0x7f4894073f90 2023-11-24T06:12:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888011870 con 0x7f4894073f90 2023-11-24T06:12:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.152+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4894156ed0 con 0x7f4894073f90 2023-11-24T06:12:38.161 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.153+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f48941440d0 con 0x7f4894073f90 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.154+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f4888021030 con 0x7f4894073f90 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.154+0000 7f4890ff9700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f488006b500 0x7f488006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.154+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f488808b140 con 0x7f4894073f90 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.154+0000 7f48937fe700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f488006b500 0x7f488006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.154+0000 7f4890ff9700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7f4880071360 0x7f48800737a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.154+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 --> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4880073e70 con 0x7f4880071360 2023-11-24T06:12:38.162 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.154+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f488808d5b0 con 0x7f4894073f90 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.155+0000 7f4898cdb700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7f4880071360 0x7f48800737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.155+0000 7f48937fe700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f488006b500 0x7f488006d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4884006b20 tx=0x7f4884006ab0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.155+0000 7f4898cdb700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7f4880071360 0x7f48800737a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.163 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.156+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 <== osd.7 v2:172.21.15.183:6808/4128955131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4880073e70 con 0x7f4880071360 2023-11-24T06:12:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 --> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4894145a00 con 0x7f4880071360 2023-11-24T06:12:38.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f4890ff9700 1 -- 172.21.15.97:0/1668024888 <== osd.7 v2:172.21.15.183:6808/4128955131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f488c0016d0 con 0x7f4880071360 2023-11-24T06:12:38.174 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-11-24T06:12:38.174 INFO:teuthology.orchestra.run.smithi097.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:12:38.174 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-11-24T06:12:38.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7f4880071360 msgr2=0x7f48800737a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.183:6808/4128955131,v1:172.21.15.183:6809/4128955131] conn(0x7f4880071360 0x7f48800737a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f488006b500 msgr2=0x7f488006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f488006b500 0x7f488006d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4884006b20 tx=0x7f4884006ab0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.175 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4894073f90 msgr2=0x7f4894156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 --2- 172.21.15.97:0/1668024888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4894073f90 0x7f4894156990 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4888009b90 tx=0x7f4888004750 comp rx=0 tx=0).stop 2023-11-24T06:12:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f4898cdb700 1 -- 172.21.15.97:0/1668024888 reap_dead start 2023-11-24T06:12:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 shutdown_connections 2023-11-24T06:12:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 >> 172.21.15.97:0/1668024888 conn(0x7f48940d0b40 msgr2=0x7f48940b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.171+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 shutdown_connections 2023-11-24T06:12:38.176 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.172+0000 7f489a73e700 1 -- 172.21.15.97:0/1668024888 wait complete. 2023-11-24T06:12:38.186 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T06:12:38.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.304+0000 7f072f715700 1 Processor -- start 2023-11-24T06:12:38.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.304+0000 7f072f715700 1 -- start start 2023-11-24T06:12:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072f715700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f072815e080 0x7f07280b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072f715700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f072806c710 0x7f07280afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072f715700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07280ae430 con 0x7f07280af4d0 2023-11-24T06:12:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072f715700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07280af0b0 con 0x7f072815e080 2023-11-24T06:12:38.309 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072f715700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07280b05a0 con 0x7f072806c710 2023-11-24T06:12:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072d4b1700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f072815e080 0x7f07280b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072d4b1700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f072815e080 0x7f07280b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52076/0 (socket says 172.21.15.97:52076) 2023-11-24T06:12:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072d4b1700 1 -- 172.21.15.97:0/800066448 learned_addr learned my addr 172.21.15.97:0/800066448 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072dcb2700 1 --2- 172.21.15.97:0/800066448 >> v2:172.21.15.183:3300/0 conn(0x7f072806c710 0x7f07280afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072d4b1700 1 -- 172.21.15.97:0/800066448 >> v2:172.21.15.183:3300/0 conn(0x7f072806c710 msgr2=0x7f07280afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072d4b1700 1 --2- 172.21.15.97:0/800066448 >> v2:172.21.15.183:3300/0 conn(0x7f072806c710 0x7f07280afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.310 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072d4b1700 1 -- 172.21.15.97:0/800066448 >> v1:172.21.15.183:6789/0 conn(0x7f07280af4d0 legacy=0x7f072806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072d4b1700 1 -- 172.21.15.97:0/800066448 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280ae710 con 0x7f072815e080 2023-11-24T06:12:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f072d4b1700 1 --2- 172.21.15.97:0/800066448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072815e080 0x7f07280b5050 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f072806e060 tx=0x7f0718009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea1761fda1fdc81 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f071ffff700 1 -- 172.21.15.97:0/800066448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718004230 con 0x7f072815e080 2023-11-24T06:12:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f071ffff700 1 -- 172.21.15.97:0/800066448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718004390 con 0x7f072815e080 2023-11-24T06:12:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.305+0000 7f071ffff700 1 -- 172.21.15.97:0/800066448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180036a0 con 0x7f072815e080 2023-11-24T06:12:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.306+0000 7f072f715700 1 -- 172.21.15.97:0/800066448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072815e080 msgr2=0x7f07280b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.306+0000 7f072f715700 1 --2- 172.21.15.97:0/800066448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072815e080 0x7f07280b5050 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f072806e060 tx=0x7f0718009c00 comp rx=0 tx=0).stop 2023-11-24T06:12:38.311 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.306+0000 7f072f715700 1 -- 172.21.15.97:0/800066448 shutdown_connections 2023-11-24T06:12:38.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.306+0000 7f072f715700 1 --2- 172.21.15.97:0/800066448 >> v2:172.21.15.183:3300/0 conn(0x7f072806c710 0x7f07280afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.306+0000 7f072f715700 1 --2- 172.21.15.97:0/800066448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072815e080 0x7f07280b5050 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.306+0000 7f072f715700 1 -- 172.21.15.97:0/800066448 >> 172.21.15.97:0/800066448 conn(0x7f0728071b90 msgr2=0x7f0728071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.306+0000 7f072f715700 1 -- 172.21.15.97:0/800066448 shutdown_connections 2023-11-24T06:12:38.312 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.306+0000 7f072f715700 1 -- 172.21.15.97:0/800066448 wait complete. 2023-11-24T06:12:38.314 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.307+0000 7f072f715700 1 Processor -- start 2023-11-24T06:12:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.307+0000 7f072f715700 1 -- start start 2023-11-24T06:12:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072f715700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072806c710 0x7f0728160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072f715700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07280af4d0 0x7f07281f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072f715700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f072815e080 0x7f07281f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072f715700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0728158b50 con 0x7f072806c710 2023-11-24T06:12:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072f715700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07281613e0 con 0x7f072815e080 2023-11-24T06:12:38.315 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072f715700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0728148720 con 0x7f07280af4d0 2023-11-24T06:12:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072806c710 0x7f0728160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072806c710 0x7f0728160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52082/0 (socket says 172.21.15.97:52082) 2023-11-24T06:12:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 -- 172.21.15.97:0/4121997298 learned_addr learned my addr 172.21.15.97:0/4121997298 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 -- 172.21.15.97:0/4121997298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07280af4d0 msgr2=0x7f07281f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07280af4d0 0x7f07281f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 -- 172.21.15.97:0/4121997298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f072815e080 msgr2=0x7f07281f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072dcb2700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f072815e080 0x7f07281f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.316 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f072815e080 0x7f07281f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 -- 172.21.15.97:0/4121997298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718009550 con 0x7f072806c710 2023-11-24T06:12:38.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.308+0000 7f072d4b1700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072806c710 0x7f0728160b00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0718009870 tx=0x7f0718004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.309+0000 7f071dffb700 1 -- 172.21.15.97:0/4121997298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718011630 con 0x7f072806c710 2023-11-24T06:12:38.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.309+0000 7f071dffb700 1 -- 172.21.15.97:0/4121997298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718011790 con 0x7f072806c710 2023-11-24T06:12:38.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.309+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0728161100 con 0x7f072806c710 2023-11-24T06:12:38.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.309+0000 7f071dffb700 1 -- 172.21.15.97:0/4121997298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718011b20 con 0x7f072806c710 2023-11-24T06:12:38.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.309+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0728159060 con 0x7f072806c710 2023-11-24T06:12:38.317 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.310+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07281596b0 con 0x7f072806c710 2023-11-24T06:12:38.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.310+0000 7f071dffb700 1 -- 172.21.15.97:0/4121997298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f0718011c80 con 0x7f072806c710 2023-11-24T06:12:38.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.310+0000 7f071dffb700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f070c06b500 0x7f070c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.311+0000 7f071dffb700 1 -- 172.21.15.97:0/4121997298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f071808b670 con 0x7f072806c710 2023-11-24T06:12:38.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.311+0000 7f072ccb0700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f070c06b500 0x7f070c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.311+0000 7f072ccb0700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f070c06b500 0x7f070c06d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0728153b60 tx=0x7f0724007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.318 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.315+0000 7f071dffb700 1 -- 172.21.15.97:0/4121997298 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0718056150 con 0x7f072806c710 2023-11-24T06:12:38.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.504+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f07280aecd0 con 0x7f070c06b500 2023-11-24T06:12:38.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.504+0000 7f071dffb700 1 -- 172.21.15.97:0/4121997298 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0724005020 con 0x7f070c06b500 2023-11-24T06:12:38.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f070c06b500 msgr2=0x7f070c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f070c06b500 0x7f070c06d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0728153b60 tx=0x7f0724007a40 comp rx=0 tx=0).stop 2023-11-24T06:12:38.509 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072806c710 msgr2=0x7f0728160b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072806c710 0x7f0728160b00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f0718009870 tx=0x7f0718004750 comp rx=0 tx=0).stop 2023-11-24T06:12:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 shutdown_connections 2023-11-24T06:12:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f072815e080 0x7f07281f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f072806c710 0x7f0728160b00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f070c06b500 0x7f070c06d9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 --2- 172.21.15.97:0/4121997298 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f07280af4d0 0x7f07281f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 >> 172.21.15.97:0/4121997298 conn(0x7f0728071b90 msgr2=0x7f072806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.510 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 shutdown_connections 2023-11-24T06:12:38.511 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.506+0000 7f072f715700 1 -- 172.21.15.97:0/4121997298 wait complete. 2023-11-24T06:12:38.511 INFO:teuthology.orchestra.run.smithi097.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T06:12:38.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:38.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.638+0000 7f3b04bf3700 1 Processor -- start 2023-11-24T06:12:38.641 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.638+0000 7f3b04bf3700 1 -- start start 2023-11-24T06:12:38.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3b04bf3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b000b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3b04bf3700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b0006c710 0x7f3b000b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3b04bf3700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b000af950 con 0x7f3b000adf00 2023-11-24T06:12:38.648 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3b04bf3700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b00144a80 con 0x7f3b0006c710 2023-11-24T06:12:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3b04bf3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b000afc30 con 0x7f3b001446a0 2023-11-24T06:12:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3afed9d700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3b0006c710 0x7f3b000b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3afdd9b700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f3b000adf00 0x7f3b0006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44544/0 (socket says 172.21.15.97:44544) 2023-11-24T06:12:38.649 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3afdd9b700 1 -- 172.21.15.97:0/4259010818 learned_addr learned my addr 172.21.15.97:0/4259010818 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:38.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.639+0000 7f3afe59c700 1 --2- 172.21.15.97:0/4259010818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b000b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afed9d700 1 -- 172.21.15.97:0/4259010818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 msgr2=0x7f3b000b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:38.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afed9d700 1 --2- 172.21.15.97:0/4259010818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b000b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.650 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afed9d700 1 -- 172.21.15.97:0/4259010818 >> v1:172.21.15.97:6789/0 conn(0x7f3b000adf00 legacy=0x7f3b0006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afed9d700 1 -- 172.21.15.97:0/4259010818 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000aff10 con 0x7f3b0006c710 2023-11-24T06:12:38.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afe59c700 1 --2- 172.21.15.97:0/4259010818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b000b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:12:38.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afed9d700 1 --2- 172.21.15.97:0/4259010818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0006c710 0x7f3b000b3d40 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009d90 tx=0x7f3af800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8645e8b2988d78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.651 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afd59a700 1 -- 172.21.15.97:0/4259010818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af800bdd0 con 0x7f3b0006c710 2023-11-24T06:12:38.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afd59a700 1 -- 172.21.15.97:0/4259010818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8011460 con 0x7f3b0006c710 2023-11-24T06:12:38.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3afd59a700 1 -- 172.21.15.97:0/4259010818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8011810 con 0x7f3b0006c710 2023-11-24T06:12:38.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3b04bf3700 1 -- 172.21.15.97:0/4259010818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0006c710 msgr2=0x7f3b000b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/4259010818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0006c710 0x7f3b000b3d40 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009d90 tx=0x7f3af800b330 comp rx=0 tx=0).stop 2023-11-24T06:12:38.652 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3b04bf3700 1 -- 172.21.15.97:0/4259010818 shutdown_connections 2023-11-24T06:12:38.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/4259010818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b0006c710 0x7f3b000b3d40 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/4259010818 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b000b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3b04bf3700 1 -- 172.21.15.97:0/4259010818 >> 172.21.15.97:0/4259010818 conn(0x7f3b00071b90 msgr2=0x7f3b00071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.653 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3b04bf3700 1 -- 172.21.15.97:0/4259010818 shutdown_connections 2023-11-24T06:12:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.640+0000 7f3b04bf3700 1 -- 172.21.15.97:0/4259010818 wait complete. 2023-11-24T06:12:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.642+0000 7f3b04bf3700 1 Processor -- start 2023-11-24T06:12:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.642+0000 7f3b04bf3700 1 -- start start 2023-11-24T06:12:38.654 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.642+0000 7f3b04bf3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0006c710 0x7f3b0013d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3b04bf3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b000adf00 0x7f3b00159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3b04bf3700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b00142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.655 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3b04bf3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0013d540 con 0x7f3b000adf00 2023-11-24T06:12:38.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3b04bf3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0015a010 con 0x7f3b0006c710 2023-11-24T06:12:38.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3b04bf3700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0013dfd0 con 0x7f3b001446a0 2023-11-24T06:12:38.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b000adf00 0x7f3b00159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b000adf00 0x7f3b00159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52098/0 (socket says 172.21.15.97:52098) 2023-11-24T06:12:38.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 -- 172.21.15.97:0/2266375822 learned_addr learned my addr 172.21.15.97:0/2266375822 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:38.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afe59c700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0006c710 0x7f3b0013d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.656 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afed9d700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b00142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 -- 172.21.15.97:0/2266375822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 msgr2=0x7f3b00142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b00142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 -- 172.21.15.97:0/2266375822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0006c710 msgr2=0x7f3b0013d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0006c710 0x7f3b0013d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 -- 172.21.15.97:0/2266375822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af8009a70 con 0x7f3b000adf00 2023-11-24T06:12:38.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3afdd9b700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b000adf00 0x7f3b00159ad0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f3af000c500 tx=0x7f3af000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.657 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3aeeffd700 1 -- 172.21.15.97:0/2266375822 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0005d70 con 0x7f3b000adf00 2023-11-24T06:12:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3aeeffd700 1 -- 172.21.15.97:0/2266375822 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af00070d0 con 0x7f3b000adf00 2023-11-24T06:12:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0015aa60 con 0x7f3b000adf00 2023-11-24T06:12:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3aeeffd700 1 -- 172.21.15.97:0/2266375822 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0007440 con 0x7f3b000adf00 2023-11-24T06:12:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.643+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0013da10 con 0x7f3b000adf00 2023-11-24T06:12:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.644+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b000753f0 con 0x7f3b000adf00 2023-11-24T06:12:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.645+0000 7f3aeeffd700 1 -- 172.21.15.97:0/2266375822 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f3af00075a0 con 0x7f3b000adf00 2023-11-24T06:12:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.645+0000 7f3aeeffd700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ae006b500 0x7f3ae006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:38.658 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.645+0000 7f3aeeffd700 1 -- 172.21.15.97:0/2266375822 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3af00891a0 con 0x7f3b000adf00 2023-11-24T06:12:38.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.645+0000 7f3afe59c700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ae006b500 0x7f3ae006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:38.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.645+0000 7f3afe59c700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ae006b500 0x7f3ae006d9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c900 tx=0x7f3af4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:38.659 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.650+0000 7f3aeeffd700 1 -- 172.21.15.97:0/2266375822 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3af0053d30 con 0x7f3b000adf00 2023-11-24T06:12:38.841 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.838+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b000b6020 con 0x7f3ae006b500 2023-11-24T06:12:38.843 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.840+0000 7f3aeeffd700 1 -- 172.21.15.97:0/2266375822 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50648 (secure 0 0 0) 0x7f3af400cdd0 con 0x7f3ae006b500 2023-11-24T06:12:38.844 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:38.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ae006b500 msgr2=0x7f3ae006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ae006b500 0x7f3ae006d9c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c900 tx=0x7f3af4008040 comp rx=0 tx=0).stop 2023-11-24T06:12:38.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b000adf00 msgr2=0x7f3b00159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:38.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b000adf00 0x7f3b00159ad0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f3af000c500 tx=0x7f3af000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 shutdown_connections 2023-11-24T06:12:38.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f3ae006b500 0x7f3ae006d9c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f3b001446a0 0x7f3b00142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b0006c710 0x7f3b0013d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 --2- 172.21.15.97:0/2266375822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b000adf00 0x7f3b00159ad0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:38.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 >> 172.21.15.97:0/2266375822 conn(0x7f3b00071b90 msgr2=0x7f3b0015e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:38.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 shutdown_connections 2023-11-24T06:12:38.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:38.842+0000 7f3b04bf3700 1 -- 172.21.15.97:0/2266375822 wait complete. 2023-11-24T06:12:38.847 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:12:38.857 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2023-11-24T06:12:38.367525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225860,"kb_used_data":9672,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754748860,"statfs":{"total":773094113280,"available":772862832640,"internally_reserved":0,"allocated":9904128,"data_stored":4908672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-135,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.266826"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":222,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T06:12:29.104885+0000","last_change":"2023-11-24T06:08:46.522656+0000","last_active":"2023-11-24T06:12:29.104885+0000","last_peered":"2023-11-24T06:12:29.104885+0000","last_clean":"2023-11-24T06:12:29.104885+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:12:29.104885+0000","last_undegraded":"2023-11-24T06:12:29.104885+0000","last_fullsized":"2023-11-24T06:12:29.104885+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:08:45.305881+0000","last_clean_scrub_stamp":"2023-11-24T06:08:45.305881+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:08.515626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28664,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343176,"statfs":{"total":96636764160,"available":96607412224,"internally_reserved":0,"allocated":1683456,"data_stored":1056360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.755,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.314}]},{"osd":2,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45600000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.499},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.437},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.71299999999999997,"15min":0.79300000000000004},"min":{"1min":0.38400000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70799999999999996,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.627},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.75,"15min":0.86199999999999999},"min":{"1min":0.371,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68999999999999995,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.504}]},{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.39900000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":1.371,"15min":1.371},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.58599999999999997},"min":{"1min":0.39700000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69099999999999995,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.76400000000000001,"15min":0.85299999999999998},"min":{"1min":0.38800000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.752,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.629,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.69899999999999995,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.39200000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.80300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.438},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58099999999999996,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":1.407,"15min":1.407},"last":0.56599999999999995}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.56799999999999995,"15min":0.55200000000000005},"min":{"1min":0.42399999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.439,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81999999999999995,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.46999999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80600000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.377,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.88700000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.33900000000000002,"15min":0.34499999999999997},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.35899999999999999,"15min":0.36299999999999999},"min":{"1min":0.26000000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.28499999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.33900000000000002,"15min":0.33800000000000002},"min":{"1min":0.25,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.375,"15min":0.379},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.27100000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56000000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.312},{"interface":"front","average":{"1min":0.374,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.56899999999999995,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.32800000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.46000000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.621,"15min":0.624},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.623,"15min":0.63500000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83299999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54600000000000004}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63900000000000001,"15min":0.65100000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89600000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.79600000000000004,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.64100000000000001,"15min":0.65100000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.80600000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.51100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28668,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343172,"statfs":{"total":96636764160,"available":96607408128,"internally_reserved":0,"allocated":1683456,"data_stored":1056360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.624,"15min":0.75800000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.47899999999999998,"5min":12.378,"15min":12.378},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.45900000000000002,"15min":0.502},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.496,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.25}]},{"osd":2,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.46700000000000003,"15min":0.50900000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.316},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.48499999999999999,"15min":0.53000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.54900000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.30599999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.46300000000000002,"15min":0.50600000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52000000000000002,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56000000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.34899999999999998}]},{"osd":4,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.96099999999999997,"15min":1.125},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73299999999999998,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.68300000000000005,"15min":0.72699999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82199999999999995,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.69299999999999995,"15min":0.73299999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79500000000000004,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.505},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.96799999999999997,"15min":1.1279999999999999},"min":{"1min":0.39600000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.79000000000000004,"5min":15.356,"15min":15.356},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.42499999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.86099999999999999,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.71499999999999997,"15min":0.76100000000000001},"min":{"1min":0.433,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":4.726,"15min":4.726},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Nov 24 06:11:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.71999999999999997,"15min":0.76400000000000001},"min":{"1min":0.47399999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.997,"15min":1.165},"min":{"1min":0.41799999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.75,"5min":15.388,"15min":15.388},"last":0.52200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.32600000000000001,"15min":0.32700000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.246},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.34399999999999997,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.379}]},{"osd":1,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34300000000000003,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.51300000000000001,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.39100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.29599999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.26200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54200000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.314},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41499999999999998,"15min":0.42099999999999999},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.71499999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59199999999999997,"15min":0.59399999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.173,"15min":1.173},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.40799999999999997,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.45700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.83599999999999997,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.65900000000000003,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78600000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.62}]},{"osd":7,"last update":"Fri Nov 24 06:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":4.0990000000000002,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.64300000000000002,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.65200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.29799999999999999,"5min":0.32700000000000001,"15min":0.33800000000000002},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.39200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.29899999999999999,"5min":0.34999999999999998,"15min":0.36699999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.42799999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32100000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.34499999999999997,"15min":0.35699999999999998},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45700000000000002,"5min":1.032,"15min":1.032},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.36799999999999999,"15min":0.38400000000000001},"min":{"1min":0.252,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44400000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39100000000000001,"15min":0.40100000000000002},"min":{"1min":0.29099999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.49299999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.48599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.33400000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66500000000000004,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.622,"15min":0.65200000000000002},"min":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.64800000000000002,"5min":1.728,"15min":1.728},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.67100000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.626,"15min":0.629},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.623,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69199999999999995,"5min":1.304,"15min":1.304},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:11:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61599999999999999,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.044,"5min":1.222,"15min":1.222},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.61199999999999999},"min":{"1min":0.41299999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.69199999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52800000000000002,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.74299999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.437,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57199999999999995,"15min":0.57399999999999995},"min":{"1min":0.40000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.55500000000000005,"15min":0.54100000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.499},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.45500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.52800000000000002,"15min":0.53000000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.54200000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70099999999999996,"5min":0.745,"15min":0.745},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.38100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69399999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58599999999999997,"15min":0.58699999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.75900000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.58499999999999996,"15min":0.59199999999999997},"min":{"1min":0.39900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71099999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.436,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76400000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.745}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.63200000000000001,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":3.335,"15min":3.335},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.622,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66900000000000004,"15min":0.72999999999999998},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.502}]},{"osd":2,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71299999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.0419999999999998,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.76300000000000001},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":1.536,"15min":1.536},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.53600000000000003}]},{"osd":6,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81999999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.438},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93700000000000006,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90700000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.65400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:38.861 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T06:08:45.305881+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=8, tm_sec=45, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=12, tm_sec=33, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T06:12:38.861 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T06:12:54.591 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:12:54.594 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:12:58.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:12:58.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.992+0000 7fc873ce0700 1 Processor -- start 2023-11-24T06:12:58.995 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.993+0000 7fc873ce0700 1 -- start start 2023-11-24T06:12:58.997 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.993+0000 7fc873ce0700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc86c1473a0 0x7fc86c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.993+0000 7fc873ce0700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7fc86c147780 0x7fc86c06c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.993+0000 7fc873ce0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c06dc20 0x7fc86c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.993+0000 7fc873ce0700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c06c9a0 con 0x7fc86c1473a0 2023-11-24T06:12:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.993+0000 7fc873ce0700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c15eb00 con 0x7fc86c147780 2023-11-24T06:12:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.993+0000 7fc873ce0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c15ede0 con 0x7fc86c06dc20 2023-11-24T06:12:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc86c1473a0 0x7fc86c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.998 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc86c1473a0 0x7fc86c15e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52104/0 (socket says 172.21.15.97:52104) 2023-11-24T06:12:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 -- 172.21.15.97:0/4222411292 learned_addr learned my addr 172.21.15.97:0/4222411292 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc87227d700 1 --2- 172.21.15.97:0/4222411292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c06dc20 0x7fc86c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc87127b700 1 --2- 172.21.15.97:0/4222411292 >> v2:172.21.15.183:3300/0 conn(0x7fc86c147780 0x7fc86c06c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 -- 172.21.15.97:0/4222411292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c06dc20 msgr2=0x7fc86c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 --2- 172.21.15.97:0/4222411292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c06dc20 0x7fc86c073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 -- 172.21.15.97:0/4222411292 >> v2:172.21.15.183:3300/0 conn(0x7fc86c147780 msgr2=0x7fc86c06c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 --2- 172.21.15.97:0/4222411292 >> v2:172.21.15.183:3300/0 conn(0x7fc86c147780 0x7fc86c06c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:58.999 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 -- 172.21.15.97:0/4222411292 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c15f0c0 con 0x7fc86c1473a0 2023-11-24T06:12:59.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc871a7c700 1 --2- 172.21.15.97:0/4222411292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c1473a0 0x7fc86c15e4b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc86c147d30 tx=0x7fc85c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f5b5a080f8fe9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc870a7a700 1 -- 172.21.15.97:0/4222411292 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011830 con 0x7fc86c1473a0 2023-11-24T06:12:59.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc870a7a700 1 -- 172.21.15.97:0/4222411292 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c011990 con 0x7fc86c1473a0 2023-11-24T06:12:59.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.994+0000 7fc870a7a700 1 -- 172.21.15.97:0/4222411292 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011d20 con 0x7fc86c1473a0 2023-11-24T06:12:59.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 -- 172.21.15.97:0/4222411292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c1473a0 msgr2=0x7fc86c15e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 --2- 172.21.15.97:0/4222411292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c1473a0 0x7fc86c15e4b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fc86c147d30 tx=0x7fc85c009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.000 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 -- 172.21.15.97:0/4222411292 shutdown_connections 2023-11-24T06:12:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 --2- 172.21.15.97:0/4222411292 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c06dc20 0x7fc86c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 --2- 172.21.15.97:0/4222411292 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c1473a0 0x7fc86c15e4b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 --2- 172.21.15.97:0/4222411292 >> v2:172.21.15.183:3300/0 conn(0x7fc86c147780 0x7fc86c06c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 -- 172.21.15.97:0/4222411292 >> 172.21.15.97:0/4222411292 conn(0x7fc86c0b3130 msgr2=0x7fc86c0d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 -- 172.21.15.97:0/4222411292 shutdown_connections 2023-11-24T06:12:59.001 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.995+0000 7fc873ce0700 1 -- 172.21.15.97:0/4222411292 wait complete. 2023-11-24T06:12:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.996+0000 7fc873ce0700 1 Processor -- start 2023-11-24T06:12:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.996+0000 7fc873ce0700 1 -- start start 2023-11-24T06:12:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc873ce0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc86c06dc20 0x7fc86c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc873ce0700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c1473a0 0x7fc86c1f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc873ce0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c147780 0x7fc86c1f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc873ce0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c133680 con 0x7fc86c147780 2023-11-24T06:12:59.004 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc873ce0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0b1660 con 0x7fc86c06dc20 2023-11-24T06:12:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc873ce0700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0b1940 con 0x7fc86c1473a0 2023-11-24T06:12:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c147780 0x7fc86c1f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c147780 0x7fc86c1f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52112/0 (socket says 172.21.15.97:52112) 2023-11-24T06:12:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc871a7c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc86c06dc20 0x7fc86c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 -- 172.21.15.97:0/3670447347 learned_addr learned my addr 172.21.15.97:0/3670447347 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87127b700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c1473a0 0x7fc86c1f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.005 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 -- 172.21.15.97:0/3670447347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c1473a0 msgr2=0x7fc86c1f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c1473a0 0x7fc86c1f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 -- 172.21.15.97:0/3670447347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc86c06dc20 msgr2=0x7fc86c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc86c06dc20 0x7fc86c0b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 -- 172.21.15.97:0/3670447347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c009550 con 0x7fc86c147780 2023-11-24T06:12:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.997+0000 7fc87227d700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c147780 0x7fc86c1f72f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc86400efc0 tx=0x7fc86400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.998+0000 7fc8627fc700 1 -- 172.21.15.97:0/3670447347 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400cad0 con 0x7fc86c147780 2023-11-24T06:12:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.998+0000 7fc8627fc700 1 -- 172.21.15.97:0/3670447347 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400f040 con 0x7fc86c147780 2023-11-24T06:12:59.006 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.998+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c133890 con 0x7fc86c147780 2023-11-24T06:12:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.998+0000 7fc8627fc700 1 -- 172.21.15.97:0/3670447347 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400ce70 con 0x7fc86c147780 2023-11-24T06:12:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.998+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c139990 con 0x7fc86c147780 2023-11-24T06:12:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.999+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c133db0 con 0x7fc86c147780 2023-11-24T06:12:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:58.999+0000 7fc8627fc700 1 -- 172.21.15.97:0/3670447347 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fc864014480 con 0x7fc86c147780 2023-11-24T06:12:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.000+0000 7fc8627fc700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc85006b500 0x7fc85006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.000+0000 7fc871a7c700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc85006b500 0x7fc85006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.000+0000 7fc8627fc700 1 -- 172.21.15.97:0/3670447347 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fc86408a930 con 0x7fc86c147780 2023-11-24T06:12:59.007 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.000+0000 7fc871a7c700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc85006b500 0x7fc85006d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009c10 tx=0x7fc85c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.008 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.004+0000 7fc8627fc700 1 -- 172.21.15.97:0/3670447347 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc86c133db0 con 0x7fc86c147780 2023-11-24T06:12:59.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.193+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 --> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc86c06b090 con 0x7fc85006b500 2023-11-24T06:12:59.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.195+0000 7fc8627fc700 1 -- 172.21.15.97:0/3670447347 <== mgr.4101 v2:172.21.15.97:6832/830621211 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50683 (secure 0 0 0) 0x7fc85c011810 con 0x7fc85006b500 2023-11-24T06:12:59.198 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc85006b500 msgr2=0x7fc85006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.200 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc85006b500 0x7fc85006d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009c10 tx=0x7fc85c008030 comp rx=0 tx=0).stop 2023-11-24T06:12:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c147780 msgr2=0x7fc86c1f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c147780 0x7fc86c1f72f0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fc86400efc0 tx=0x7fc86400c0c0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 shutdown_connections 2023-11-24T06:12:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc86c147780 0x7fc86c1f72f0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc86c06dc20 0x7fc86c0b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7fc85006b500 0x7fc85006d9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 --2- 172.21.15.97:0/3670447347 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7fc86c1473a0 0x7fc86c1f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.201 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.197+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 >> 172.21.15.97:0/3670447347 conn(0x7fc86c0b3130 msgr2=0x7fc86c1618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.198+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 shutdown_connections 2023-11-24T06:12:59.202 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.198+0000 7fc873ce0700 1 -- 172.21.15.97:0/3670447347 wait complete. 2023-11-24T06:12:59.202 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2023-11-24T06:12:59.211 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2023-11-24T06:12:58.371394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225848,"kb_used_data":9672,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754748872,"statfs":{"total":773094113280,"available":772862844928,"internally_reserved":0,"allocated":9904128,"data_stored":4908672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002355"},"pg_stats":[{"pgid":"1.0","version":"88'119","reported_seq":234,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T06:12:42.435123+0000","last_change":"2023-11-24T06:12:42.435123+0000","last_active":"2023-11-24T06:12:42.435123+0000","last_peered":"2023-11-24T06:12:42.435123+0000","last_clean":"2023-11-24T06:12:42.435123+0000","last_became_active":"2023-11-24T06:08:46.521506+0000","last_became_peered":"2023-11-24T06:08:46.521506+0000","last_unstale":"2023-11-24T06:12:42.435123+0000","last_undegraded":"2023-11-24T06:12:42.435123+0000","last_fullsized":"2023-11-24T06:12:42.435123+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"88'119","last_scrub_stamp":"2023-11-24T06:12:42.434983+0000","last_deep_scrub":"88'119","last_deep_scrub_stamp":"2023-11-24T06:12:42.434983+0000","last_clean_scrub_stamp":"2023-11-24T06:12:42.434983+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:59:32.952490+0000","scrub_duration":0.0057953689999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343180,"statfs":{"total":96636764160,"available":96607416320,"internally_reserved":0,"allocated":1683456,"data_stored":1056360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53900000000000003,"15min":0.53000000000000003},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53500000000000003,"15min":0.52300000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54700000000000004,"15min":0.54200000000000004},"min":{"1min":0.40600000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.755,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5800000000000001,"5min":2.5800000000000001,"15min":2.5800000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72699999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.63}]},{"osd":4,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51700000000000002,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.68200000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":0.876},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.59399999999999997},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.62,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83099999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Nov 24 06:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.39000000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85399999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.66100000000000003,"15min":0.77500000000000002},"min":{"1min":0.37,"5min":0.313,"15min":0.313},"max":{"1min":0.77800000000000002,"5min":8.4190000000000005,"15min":8.4190000000000005},"last":0.75},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.67700000000000005,"15min":0.83799999999999997},"min":{"1min":0.34699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":9.5960000000000001,"15min":9.5960000000000001},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.55800000000000005},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":1.371,"15min":1.371},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56399999999999995,"15min":0.58299999999999996},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69799999999999995,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.70099999999999996,"15min":0.83199999999999996},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.78200000000000003,"5min":8.4849999999999994,"15min":8.4849999999999994},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.624,"15min":0.61399999999999999},"min":{"1min":0.38,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80100000000000005,"5min":5.2830000000000004,"15min":5.2830000000000004},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.42799999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57399999999999995,"15min":0.59399999999999997},"min":{"1min":0.40999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.753,"5min":1.407,"15min":1.407},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.374,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.70299999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.497},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.315,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70299999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67200000000000004,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.74399999999999999,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61699999999999999,"15min":0.61199999999999999},"min":{"1min":0.39400000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77300000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61399999999999999,"15min":0.60999999999999999},"min":{"1min":0.46500000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.71699999999999997,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33500000000000002,"15min":0.34300000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.505,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.61299999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.33900000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.376,"15min":0.38},"min":{"1min":0.26000000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39600000000000002,"15min":0.40300000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59799999999999998,"15min":0.61399999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.86099999999999999,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.621,"15min":0.63700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94299999999999995,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Nov 24 06:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.60999999999999999,"15min":0.63100000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Nov 24 06:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.63200000000000001,"15min":0.64800000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Nov 24 06:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.629,"15min":0.63300000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.875,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95999999999999996,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.751}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28660,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343180,"statfs":{"total":96636764160,"available":96607416320,"internally_reserved":0,"allocated":1683456,"data_stored":1056360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.52500000000000002,"15min":0.72499999999999998},"min":{"1min":0.22800000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.65400000000000003,"5min":12.378,"15min":12.378},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.42699999999999999,"15min":0.49099999999999999},"min":{"1min":0.20999999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52000000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.436,"15min":0.498},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.496,"5min":4.5199999999999996,"15min":4.5199999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.45000000000000001,"15min":0.51800000000000002},"min":{"1min":0.28999999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.67300000000000004,"5min":4.6900000000000004,"15min":4.6900000000000004},"last":0.441}]},{"osd":3,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.43099999999999999,"15min":0.495},"min":{"1min":0.25,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.54200000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.38},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.25,"15min":0.25},"max":{"1min":0.69199999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.46300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.83699999999999997,"15min":1.0840000000000001},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79400000000000004,"5min":15.319000000000001,"15min":15.319000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.65000000000000002,"15min":0.71599999999999997},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78100000000000003,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.66200000000000003,"15min":0.72299999999999998},"min":{"1min":0.434,"5min":0.373,"15min":0.373},"max":{"1min":0.73299999999999998,"5min":4.7590000000000003,"15min":4.7590000000000003},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.84799999999999998,"15min":1.0880000000000001},"min":{"1min":0.40200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.77300000000000002,"5min":15.356,"15min":15.356},"last":0.61599999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.65900000000000003,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77200000000000002,"5min":4.4550000000000001,"15min":4.4550000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.67800000000000005,"15min":0.749},"min":{"1min":0.45500000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.746,"5min":4.726,"15min":4.726},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.68300000000000005,"15min":0.752},"min":{"1min":0.48299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.78200000000000003,"5min":4.7869999999999999,"15min":4.7869999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.86899999999999999,"15min":1.123},"min":{"1min":0.44800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76600000000000001,"5min":15.388,"15min":15.388},"last":0.74199999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33100000000000002,"15min":0.32900000000000001},"min":{"1min":0.17100000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.57299999999999995,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34699999999999998,"15min":0.34100000000000003},"min":{"1min":0.22700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.55900000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.39100000000000001},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.375,"15min":0.38500000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53600000000000003,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.42099999999999999},"min":{"1min":0.28399999999999997,"5min":0.252,"15min":0.252},"max":{"1min":0.627,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.435,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.86699999999999999,"5min":1.173,"15min":1.173},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95399999999999996,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.63800000000000001,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":4.5540000000000003,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":0.751},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.92000000000000004,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.63100000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91700000000000004,"5min":3.319,"15min":3.319},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.61299999999999999,"15min":0.60599999999999998},"min":{"1min":0.443,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0129999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:12:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67300000000000004,"15min":0.63900000000000001},"min":{"1min":0.38800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.417,"5min":4.0990000000000002,"15min":4.0990000000000002},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.442,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97899999999999998,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.86699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.32800000000000001,"15min":0.33800000000000002},"min":{"1min":0.191,"5min":0.187,"15min":0.187},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.318},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.34300000000000003,"15min":0.36499999999999999},"min":{"1min":0.248,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.48999999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.25600000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.36499999999999999,"15min":0.38300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.39300000000000002,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.41699999999999998,"15min":0.42799999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.77700000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.432}]},{"osd":4,"last update":"Fri Nov 24 06:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.41399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.97199999999999998,"5min":1.6399999999999999,"15min":1.6399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.61399999999999999,"15min":0.64900000000000002},"min":{"1min":0.45800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90200000000000002,"5min":1.728,"15min":1.728},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.38,"15min":0.38},"max":{"1min":0.872,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.47599999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85899999999999999,"5min":1.089,"15min":1.089},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.46600000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40200000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83399999999999996,"5min":1.304,"15min":1.304},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Nov 24 06:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.61799999999999999,"15min":0.63200000000000001},"min":{"1min":0.42299999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":1.222,"15min":1.222},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0349999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.51900000000000002},"min":{"1min":0.39900000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74199999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52100000000000002,"15min":0.503},"min":{"1min":0.435,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68500000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.33900000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.53000000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78600000000000003,"5min":1.044,"15min":1.044},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.65300000000000002,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.752,"5min":1.071,"15min":1.071},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.81100000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.629},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.39000000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.72799999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.71099999999999997,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53400000000000003,"15min":0.54300000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":0.745,"15min":0.745},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58499999999999996,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76100000000000001,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:12:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57899999999999996,"15min":0.58999999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.77400000000000002,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.42999999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.745,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1089536,"data_stored":465992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.63200000000000001,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64800000000000002,"5min":3.335,"15min":3.335},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.622,"15min":0.66200000000000003},"min":{"1min":0.36699999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":3.4580000000000002,"15min":3.4580000000000002},"last":0.442}]},{"osd":1,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.66900000000000004,"15min":0.72999999999999998},"min":{"1min":0.42699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.746,"5min":4.4489999999999998,"15min":4.4489999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73699999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57499999999999996,"15min":0.58499999999999996},"min":{"1min":0.42599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71199999999999997,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71299999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.73799999999999999},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":3.0419999999999998,"5min":3.4940000000000002,"15min":3.4940000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.76300000000000001},"min":{"1min":0.44800000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":4.5069999999999997,"15min":4.5069999999999997},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.59099999999999997},"min":{"1min":0.30399999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":1.536,"15min":1.536},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.39400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84699999999999998,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.628,"15min":0.64600000000000002},"min":{"1min":0.40999999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.81999999999999995,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64500000000000002,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.93700000000000006,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Nov 24 06:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"min":{"1min":0.42499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.35999999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90700000000000003,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:12:59.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T06:12:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.330+0000 7f6417251700 1 Processor -- start 2023-11-24T06:12:59.333 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.330+0000 7f6417251700 1 -- start start 2023-11-24T06:12:59.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f6417251700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f641015e250 0x7f6410073f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f6417251700 1 --2- >> v2:172.21.15.183:3300/0 conn(0x7f64100afea0 0x7f64100744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.335 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f6417251700 1 -- --> v1:172.21.15.183:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64100b48f0 con 0x7f64100b0560 2023-11-24T06:12:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f6417251700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f641015e630 con 0x7f641015e250 2023-11-24T06:12:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f6417251700 1 -- --> v2:172.21.15.183:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64100b4bd0 con 0x7f64100afea0 2023-11-24T06:12:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f6414fed700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f641015e250 0x7f6410073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f6414fed700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f641015e250 0x7f6410073f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52116/0 (socket says 172.21.15.97:52116) 2023-11-24T06:12:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f6414fed700 1 -- 172.21.15.97:0/2878144318 learned_addr learned my addr 172.21.15.97:0/2878144318 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.331+0000 7f64157ee700 1 --2- 172.21.15.97:0/2878144318 >> v2:172.21.15.183:3300/0 conn(0x7f64100afea0 0x7f64100744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f6414fed700 1 -- 172.21.15.97:0/2878144318 >> v2:172.21.15.183:3300/0 conn(0x7f64100afea0 msgr2=0x7f64100744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:59.336 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f6414fed700 1 --2- 172.21.15.97:0/2878144318 >> v2:172.21.15.183:3300/0 conn(0x7f64100afea0 0x7f64100744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f6414fed700 1 -- 172.21.15.97:0/2878144318 >> v1:172.21.15.183:6789/0 conn(0x7f64100b0560 legacy=0x7f6410161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f6414fed700 1 -- 172.21.15.97:0/2878144318 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64100b50d0 con 0x7f641015e250 2023-11-24T06:12:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f6414fed700 1 --2- 172.21.15.97:0/2878144318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f641015e250 0x7f6410073f90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6410068b70 tx=0x7f6404009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959234a70d37123d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f640f7fe700 1 -- 172.21.15.97:0/2878144318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404011830 con 0x7f641015e250 2023-11-24T06:12:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f640f7fe700 1 -- 172.21.15.97:0/2878144318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6404011990 con 0x7f641015e250 2023-11-24T06:12:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f640f7fe700 1 -- 172.21.15.97:0/2878144318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404011d20 con 0x7f641015e250 2023-11-24T06:12:59.337 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f6417251700 1 -- 172.21.15.97:0/2878144318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f641015e250 msgr2=0x7f6410073f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.332+0000 7f6417251700 1 --2- 172.21.15.97:0/2878144318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f641015e250 0x7f6410073f90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6410068b70 tx=0x7f6404009bd0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.333+0000 7f6417251700 1 -- 172.21.15.97:0/2878144318 shutdown_connections 2023-11-24T06:12:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.333+0000 7f6417251700 1 --2- 172.21.15.97:0/2878144318 >> v2:172.21.15.183:3300/0 conn(0x7f64100afea0 0x7f64100744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.333+0000 7f6417251700 1 --2- 172.21.15.97:0/2878144318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f641015e250 0x7f6410073f90 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.333+0000 7f6417251700 1 -- 172.21.15.97:0/2878144318 >> 172.21.15.97:0/2878144318 conn(0x7f6410072960 msgr2=0x7f641006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.333+0000 7f6417251700 1 -- 172.21.15.97:0/2878144318 shutdown_connections 2023-11-24T06:12:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.333+0000 7f6417251700 1 -- 172.21.15.97:0/2878144318 wait complete. 2023-11-24T06:12:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.334+0000 7f6417251700 1 Processor -- start 2023-11-24T06:12:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.334+0000 7f6417251700 1 -- start start 2023-11-24T06:12:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6417251700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64100afea0 0x7f6410147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6417251700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64100b0560 0x7f64100cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.341 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6417251700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f641015e250 0x7f64101f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6417251700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64100ccbf0 con 0x7f64100b0560 2023-11-24T06:12:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6417251700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6410148bc0 con 0x7f64100afea0 2023-11-24T06:12:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6417251700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6410148520 con 0x7f641015e250 2023-11-24T06:12:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64100b0560 0x7f64100cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64100b0560 0x7f64100cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52124/0 (socket says 172.21.15.97:52124) 2023-11-24T06:12:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 -- 172.21.15.97:0/2989542786 learned_addr learned my addr 172.21.15.97:0/2989542786 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:59.342 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6414fed700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64100afea0 0x7f6410147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f64157ee700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f641015e250 0x7f64101f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 -- 172.21.15.97:0/2989542786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f641015e250 msgr2=0x7f64101f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f641015e250 0x7f64101f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 -- 172.21.15.97:0/2989542786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64100afea0 msgr2=0x7f6410147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64100afea0 0x7f6410147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 -- 172.21.15.97:0/2989542786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6404009550 con 0x7f64100b0560 2023-11-24T06:12:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640ffff700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64100b0560 0x7f64100cf140 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c4b0 tx=0x7f63fc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.343 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640d7fa700 1 -- 172.21.15.97:0/2989542786 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0052d0 con 0x7f64100b0560 2023-11-24T06:12:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640d7fa700 1 -- 172.21.15.97:0/2989542786 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc005430 con 0x7f64100b0560 2023-11-24T06:12:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64100cf9c0 con 0x7f64100b0560 2023-11-24T06:12:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f640d7fa700 1 -- 172.21.15.97:0/2989542786 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc0038c0 con 0x7f64100b0560 2023-11-24T06:12:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.335+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64100cff80 con 0x7f64100b0560 2023-11-24T06:12:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.336+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6410068fb0 con 0x7f64100b0560 2023-11-24T06:12:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.337+0000 7f640d7fa700 1 -- 172.21.15.97:0/2989542786 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f63fc003a20 con 0x7f64100b0560 2023-11-24T06:12:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.337+0000 7f640d7fa700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f63f406b500 0x7f63f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.344 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.337+0000 7f6414fed700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f63f406b500 0x7f63f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.337+0000 7f640d7fa700 1 -- 172.21.15.97:0/2989542786 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f63fc0890f0 con 0x7f64100b0560 2023-11-24T06:12:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.338+0000 7f6414fed700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f63f406b500 0x7f63f406d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6404014040 tx=0x7f6404008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.345 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.342+0000 7f640d7fa700 1 -- 172.21.15.97:0/2989542786 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63fc0bbe90 con 0x7f64100b0560 2023-11-24T06:12:59.533 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.530+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6410074ac0 con 0x7f64100b0560 2023-11-24T06:12:59.535 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.533+0000 7f640d7fa700 1 -- 172.21.15.97:0/2989542786 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v20)=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7f63fc053bd0 con 0x7f64100b0560 2023-11-24T06:12:59.536 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.534+0000 7f640d7fa700 1 -- 172.21.15.97:0/2989542786 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f63fc086f20 con 0x7f64100b0560 2023-11-24T06:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f63f406b500 msgr2=0x7f63f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f63f406b500 0x7f63f406d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6404014040 tx=0x7f6404008030 comp rx=0 tx=0).stop 2023-11-24T06:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64100b0560 msgr2=0x7f64100cf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.539 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64100b0560 0x7f64100cf140 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f63fc00c4b0 tx=0x7f63fc00c7c0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 shutdown_connections 2023-11-24T06:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7f63f406b500 0x7f63f406d9c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7f641015e250 0x7f64101f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64100afea0 0x7f6410147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 --2- 172.21.15.97:0/2989542786 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64100b0560 0x7f64100cf140 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 >> 172.21.15.97:0/2989542786 conn(0x7f6410072960 msgr2=0x7f6410160780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 shutdown_connections 2023-11-24T06:12:59.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.536+0000 7f6417251700 1 -- 172.21.15.97:0/2989542786 wait complete. 2023-11-24T06:12:59.551 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:12:59.552 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:12:59.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.669+0000 7ff7e2239700 1 Processor -- start 2023-11-24T06:12:59.672 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7e2239700 1 -- start start 2023-11-24T06:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7e2239700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc15e080 0x7ff7dc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7e2239700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7dc0af4d0 0x7ff7dc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7e2239700 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7dc0ae430 con 0x7ff7dc06c000 2023-11-24T06:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7e2239700 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc0af0b0 con 0x7ff7dc0af4d0 2023-11-24T06:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7e2239700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0b05a0 con 0x7ff7dc15e080 2023-11-24T06:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7daffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7dc0af4d0 0x7ff7dc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.675 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7dbfff700 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff7dc06c000 0x7ff7dc073f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:44584/0 (socket says 172.21.15.97:44584) 2023-11-24T06:12:59.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.670+0000 7ff7daffd700 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7dc0af4d0 0x7ff7dc0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52130/0 (socket says 172.21.15.97:52130) 2023-11-24T06:12:59.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7dbfff700 1 -- 172.21.15.97:0/1838347674 learned_addr learned my addr 172.21.15.97:0/1838347674 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-24T06:12:59.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7db7fe700 1 --2- 172.21.15.97:0/1838347674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc15e080 0x7ff7dc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7daffd700 1 -- 172.21.15.97:0/1838347674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc15e080 msgr2=0x7ff7dc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:12:59.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7daffd700 1 --2- 172.21.15.97:0/1838347674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc15e080 0x7ff7dc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7daffd700 1 -- 172.21.15.97:0/1838347674 >> v1:172.21.15.97:6789/0 conn(0x7ff7dc06c000 legacy=0x7ff7dc073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7daffd700 1 -- 172.21.15.97:0/1838347674 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc0ae710 con 0x7ff7dc0af4d0 2023-11-24T06:12:59.676 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7da7fc700 1 -- 172.21.15.97:0/1838347674 <== mon.0 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167639476 0 0) 0x7ff7dc0ae430 con 0x7ff7dc06c000 2023-11-24T06:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7daffd700 1 --2- 172.21.15.97:0/1838347674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc0af4d0 0x7ff7dc0afb90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009870 tx=0x7ff7cc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292a68171986892c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7da7fc700 1 -- 172.21.15.97:0/1838347674 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc011830 con 0x7ff7dc0af4d0 2023-11-24T06:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7da7fc700 1 -- 172.21.15.97:0/1838347674 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff7cc011990 con 0x7ff7dc0af4d0 2023-11-24T06:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.671+0000 7ff7da7fc700 1 -- 172.21.15.97:0/1838347674 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc011d40 con 0x7ff7dc0af4d0 2023-11-24T06:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.672+0000 7ff7e2239700 1 -- 172.21.15.97:0/1838347674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc0af4d0 msgr2=0x7ff7dc0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.672+0000 7ff7e2239700 1 --2- 172.21.15.97:0/1838347674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc0af4d0 0x7ff7dc0afb90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009870 tx=0x7ff7cc009b80 comp rx=0 tx=0).stop 2023-11-24T06:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.672+0000 7ff7e2239700 1 -- 172.21.15.97:0/1838347674 shutdown_connections 2023-11-24T06:12:59.677 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.672+0000 7ff7e2239700 1 --2- 172.21.15.97:0/1838347674 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc15e080 0x7ff7dc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.672+0000 7ff7e2239700 1 --2- 172.21.15.97:0/1838347674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc0af4d0 0x7ff7dc0afb90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.672+0000 7ff7e2239700 1 -- 172.21.15.97:0/1838347674 >> 172.21.15.97:0/1838347674 conn(0x7ff7dc071b90 msgr2=0x7ff7dc071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.672+0000 7ff7e2239700 1 -- 172.21.15.97:0/1838347674 shutdown_connections 2023-11-24T06:12:59.678 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.672+0000 7ff7e2239700 1 -- 172.21.15.97:0/1838347674 wait complete. 2023-11-24T06:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.673+0000 7ff7e2239700 1 Processor -- start 2023-11-24T06:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.673+0000 7ff7e2239700 1 -- start start 2023-11-24T06:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.674+0000 7ff7e2239700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc06c000 0x7ff7dc160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.674+0000 7ff7e2239700 1 --2- >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc0af4d0 0x7ff7dc160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.674+0000 7ff7e2239700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7dc15e080 0x7ff7dc0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.674+0000 7ff7e2239700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0cca90 con 0x7ff7dc06c000 2023-11-24T06:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.674+0000 7ff7e2239700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc0c6be0 con 0x7ff7dc15e080 2023-11-24T06:12:59.681 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.674+0000 7ff7e2239700 1 -- --> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc14d5a0 con 0x7ff7dc0af4d0 2023-11-24T06:12:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.674+0000 7ff7db7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc06c000 0x7ff7dc160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.674+0000 7ff7db7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc06c000 0x7ff7dc160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52134/0 (socket says 172.21.15.97:52134) 2023-11-24T06:12:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7db7fe700 1 -- 172.21.15.97:0/2020321265 learned_addr learned my addr 172.21.15.97:0/2020321265 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-24T06:12:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7daffd700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc0af4d0 0x7ff7dc160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7dbfff700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7dc15e080 0x7ff7dc0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7db7fe700 1 -- 172.21.15.97:0/2020321265 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc0af4d0 msgr2=0x7ff7dc160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7db7fe700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc0af4d0 0x7ff7dc160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.682 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7db7fe700 1 -- 172.21.15.97:0/2020321265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7dc15e080 msgr2=0x7ff7dc0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7db7fe700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7dc15e080 0x7ff7dc0c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7db7fe700 1 -- 172.21.15.97:0/2020321265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc0094a0 con 0x7ff7dc06c000 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7db7fe700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc06c000 0x7ff7dc160630 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000e870 tx=0x7ff7d000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7c3fff700 1 -- 172.21.15.97:0/2020321265 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0009590 con 0x7ff7dc06c000 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7c3fff700 1 -- 172.21.15.97:0/2020321265 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff7d0013680 con 0x7ff7dc06c000 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc14d880 con 0x7ff7dc06c000 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7c3fff700 1 -- 172.21.15.97:0/2020321265 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0013a10 con 0x7ff7dc06c000 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.675+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc14de40 con 0x7ff7dc06c000 2023-11-24T06:12:59.683 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.676+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc0753f0 con 0x7ff7dc06c000 2023-11-24T06:12:59.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.677+0000 7ff7c3fff700 1 -- 172.21.15.97:0/2020321265 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff7d0013b70 con 0x7ff7dc06c000 2023-11-24T06:12:59.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.677+0000 7ff7c3fff700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b530 0x7ff7c806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:12:59.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.677+0000 7ff7c3fff700 1 -- 172.21.15.97:0/2020321265 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4971+0+0 (secure 0 0 0) 0x7ff7d0089db0 con 0x7ff7dc06c000 2023-11-24T06:12:59.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.677+0000 7ff7daffd700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b530 0x7ff7c806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:12:59.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.678+0000 7ff7daffd700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b530 0x7ff7c806d9f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009870 tx=0x7ff7cc01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:12:59.684 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.682+0000 7ff7c3fff700 1 -- 172.21.15.97:0/2020321265 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff7d00546e0 con 0x7ff7dc06c000 2023-11-24T06:12:59.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.913+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7dc067f60 con 0x7ff7dc06c000 2023-11-24T06:12:59.916 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.914+0000 7ff7c3fff700 1 -- 172.21.15.97:0/2020321265 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+3705 (secure 0 0 0) 0x7ff7d0021020 con 0x7ff7dc06c000 2023-11-24T06:12:59.917 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-24T06:12:59.917 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":58,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5816,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/4115557494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6833","nonce":4115557494},{"type":"v1","addr":"172.21.15.183:6835","nonce":4115557494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6903,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3848239046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3848239046},{"type":"v1","addr":"172.21.15.97:6836","nonce":3848239046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6906,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3181686967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":3181686967},{"type":"v1","addr":"172.21.15.97:6837","nonce":3181686967}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6834/4023096288","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4023096288},{"type":"v1","addr":"172.21.15.183:6834","nonce":4023096288}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[]} 2023-11-24T06:12:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b530 msgr2=0x7ff7c806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b530 0x7ff7c806d9f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009870 tx=0x7ff7cc01c040 comp rx=0 tx=0).stop 2023-11-24T06:12:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc06c000 msgr2=0x7ff7dc160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:12:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc06c000 0x7ff7dc160630 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000e870 tx=0x7ff7d000eb80 comp rx=0 tx=0).stop 2023-11-24T06:12:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 shutdown_connections 2023-11-24T06:12:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7dc15e080 0x7ff7dc0c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.919 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7dc06c000 0x7ff7dc160630 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.97:6832/830621211,v1:172.21.15.97:6833/830621211] conn(0x7ff7c806b530 0x7ff7c806d9f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 --2- 172.21.15.97:0/2020321265 >> [v2:172.21.15.183:3301/0,v1:172.21.15.183:6790/0] conn(0x7ff7dc0af4d0 0x7ff7dc160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:12:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 >> 172.21.15.97:0/2020321265 conn(0x7ff7dc071b90 msgr2=0x7ff7dc0b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:12:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 shutdown_connections 2023-11-24T06:12:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-24T06:12:59.916+0000 7ff7e2239700 1 -- 172.21.15.97:0/2020321265 wait complete. 2023-11-24T06:12:59.920 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 58 2023-11-24T06:12:59.931 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T06:12:59.931 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T06:12:59.931 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:12:59.931 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:59.940 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:12:59.940 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:12:59.940 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:12:59.989 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:12:59.989 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:12:59.989 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.000 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:13:00.000 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:13:00.000 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.049 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:13:00.049 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T06:13:00.050 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T06:13:00.050 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.092 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T06:13:00.092 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T06:13:00.092 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.125 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T06:13:00.126 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T06:13:00.126 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.197 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T06:13:00.197 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T06:13:00.197 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.271 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T06:13:00.272 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T06:13:00.272 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.305 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T06:13:00.306 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T06:13:00.306 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.379 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T06:13:00.380 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T06:13:00.380 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.450 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T06:13:00.450 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T06:13:00.451 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.536 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T06:13:00.536 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T06:13:00.536 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T06:13:00.537 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.561 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T06:13:00.561 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T06:13:00.561 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.583 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T06:13:00.583 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T06:13:00.584 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T06:13:00.584 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.612 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T06:13:00.613 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T06:13:00.613 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.634 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T06:13:00.634 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T06:13:00.634 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:13:00.686 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T06:13:00.686 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T06:13:00.686 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T06:13:00.719 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:13:00.720 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T06:13:00.966 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:13:00.966 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T06:13:01.103 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:13:01.103 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T06:13:01.194 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:13:01.194 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T06:13:01.285 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:13:01.286 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T06:13:01.580 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:13:01.580 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T06:13:01.672 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:13:01.673 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T06:13:01.867 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:13:01.868 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T06:13:01.955 INFO:tasks.ceph:Archiving mon data... 2023-11-24T06:13:01.956 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/data/mon.a.tgz 2023-11-24T06:13:01.956 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2023-11-24T06:13:01.974 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.44BK9s6Q63 2023-11-24T06:13:01.974 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.44BK9s6Q63 2023-11-24T06:13:02.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.44BK9s6Q63 2023-11-24T06:13:02.258 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.44BK9s6Q63 is 723KB 2023-11-24T06:13:02.358 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.44BK9s6Q63 2023-11-24T06:13:02.374 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/data/mon.b.tgz 2023-11-24T06:13:02.374 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2023-11-24T06:13:02.390 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.CUKBnFblai 2023-11-24T06:13:02.391 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.CUKBnFblai 2023-11-24T06:13:02.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.CUKBnFblai 2023-11-24T06:13:02.730 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.CUKBnFblai is 985KB 2023-11-24T06:13:02.831 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.CUKBnFblai 2023-11-24T06:13:02.847 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/data/mon.c.tgz 2023-11-24T06:13:02.847 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2023-11-24T06:13:02.902 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.1DxL7Icp2g 2023-11-24T06:13:02.902 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.1DxL7Icp2g 2023-11-24T06:13:03.141 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.1DxL7Icp2g 2023-11-24T06:13:03.225 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.1DxL7Icp2g is 985KB 2023-11-24T06:13:03.328 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.1DxL7Icp2g 2023-11-24T06:13:03.344 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T06:13:03.344 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:13:03.347 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:13:03.410 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T06:13:03.410 DEBUG:teuthology.orchestra.run.smithi097:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:13:03.419 DEBUG:teuthology.orchestra.run.smithi183:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:13:03.475 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:13:03.483 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:13:03.489 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T06:13:03.490 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/remote/smithi097/crash 2023-11-24T06:13:03.491 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:13:03.520 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/remote/smithi183/crash 2023-11-24T06:13:03.521 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:13:03.554 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T06:13:03.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:13:03.565 DEBUG:teuthology.orchestra.run.smithi183:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:13:03.640 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:13:03.668 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:13:03.693 INFO:tasks.ceph:Compressing logs... 2023-11-24T06:13:03.694 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:13:03.710 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:13:03.735 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144615.log 2023-11-24T06:13:03.771 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T06:13:03.772 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133559.log 2023-11-24T06:13:03.772 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:13:03.772 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.133559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133696.log 2023-11-24T06:13:03.772 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.133559.log.gz 2023-11-24T06:13:03.772 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T06:13:03.772 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.133696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133839.log 2023-11-24T06:13:03.772 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.6.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133696.log.gz 2023-11-24T06:13:03.773 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.133839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133741.log 2023-11-24T06:13:03.773 INFO:teuthology.orchestra.run.smithi183.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133839.log.gz 2023-11-24T06:13:03.773 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.143364.log 2023-11-24T06:13:03.773 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.133741.log: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:13:03.773 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133650.log 2023-11-24T06:13:03.773 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.143364.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.133741.log.gz 2023-11-24T06:13:03.773 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.143364.log.gz 2023-11-24T06:13:03.773 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.141585.log 2023-11-24T06:13:03.774 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T06:13:03.774 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.133650.log: /var/log/ceph/ceph-client.1.141585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.143368.log 0.0% 2023-11-24T06:13:03.774 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph-client.1.141585.log.gz 2023-11-24T06:13:03.774 INFO:teuthology.orchestra.run.smithi183.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133650.log.gz 2023-11-24T06:13:03.774 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:13:03.774 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.143368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.143368.log.gz 2023-11-24T06:13:03.774 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.audit.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:13:03.774 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.142172.log 2023-11-24T06:13:03.775 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.140815.log 2023-11-24T06:13:03.775 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.142172.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.1.142771.log -- replaced with /var/log/ceph/ceph-client.1.142172.log.gz 2023-11-24T06:13:03.775 INFO:teuthology.orchestra.run.smithi183.stderr: 2023-11-24T06:13:03.775 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.3.140815.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T06:13:03.775 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.140815.log.gz 2023-11-24T06:13:03.775 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.142771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.142771.log.gz 2023-11-24T06:13:03.775 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.143992.log 2023-11-24T06:13:03.775 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.141589.log 2023-11-24T06:13:03.776 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.143992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.143992.log.gz 2023-11-24T06:13:03.776 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.141589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.141589.log.gz 2023-11-24T06:13:03.776 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T06:13:03.776 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.143988.log 2023-11-24T06:13:03.776 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.1.143988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.143988.log.gz 2023-11-24T06:13:03.776 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.142767.log 2023-11-24T06:13:03.776 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.142767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.142767.log.gz 2023-11-24T06:13:03.776 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133604.log 2023-11-24T06:13:03.777 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.133604.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.133604.log.gz 2023-11-24T06:13:03.777 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133512.log 2023-11-24T06:13:03.780 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.133512.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133512.log.gz 2023-11-24T06:13:03.780 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T06:13:03.781 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.c.log: 93.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T06:13:03.781 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.140819.log 2023-11-24T06:13:03.781 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.3.140819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.140819.log.gz 2023-11-24T06:13:03.781 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.140179.log 2023-11-24T06:13:03.781 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.140179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.140179.log.gz 2023-11-24T06:13:03.781 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.142176.log 2023-11-24T06:13:03.781 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.142176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.142176.log.gz 2023-11-24T06:13:03.781 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T06:13:03.782 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.140187.log 2023-11-24T06:13:03.789 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.1.140187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.140187.log.gz 2023-11-24T06:13:03.789 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-24T06:13:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156117.log 2023-11-24T06:13:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140691.log 2023-11-24T06:13:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144615.log: /var/log/ceph/ceph-client.admin.156117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146677.log 2023-11-24T06:13:03.799 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144615.log.gz 2023-11-24T06:13:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140691.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156117.log.gz 2023-11-24T06:13:03.799 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141061.log 2023-11-24T06:13:03.799 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.140691.log.gz 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153632.log 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141061.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146677.log.gz 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147476.log 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0%/var/log/ceph/ceph-client.admin.153632.log: -- replaced with /var/log/ceph/ceph-client.admin.141061.log.gz 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141966.log 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.153632.log.gz 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152812.log 2023-11-24T06:13:03.800 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141966.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147476.log.gz 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164470.log 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152812.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141966.log.gz 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159848.log 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.152812.log.gz 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.155366.log 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159848.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164470.log.gz 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143970.log 2023-11-24T06:13:03.801 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159848.log.gz 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.155366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147407.log 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.155366.log.gz 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160143.log 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147407.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143970.log.gz 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163105.log 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160143.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147407.log.gz 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139883.log 2023-11-24T06:13:03.802 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163105.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160143.log.gz 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140688.log 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139883.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163105.log.gz 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.121978.log 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140688.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139883.log.gz 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162971.log 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140688.log.gz 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.tmp-client.admin.121978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146401.log 2023-11-24T06:13:03.803 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.121978.log.gz 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144062.log 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146401.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162971.log.gz 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153521.log 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146401.log.gz 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157276.log 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153521.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144062.log.gz 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140256.log 2023-11-24T06:13:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157276.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153521.log.gz 2023-11-24T06:13:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149996.log 2023-11-24T06:13:03.805 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157276.log.gz 2023-11-24T06:13:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160670.log 2023-11-24T06:13:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149996.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140256.log.gz 2023-11-24T06:13:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156730.log 2023-11-24T06:13:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160670.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149996.log.gz 2023-11-24T06:13:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153291.log 2023-11-24T06:13:03.805 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160670.log.gz 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.162377.log 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153291.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156730.log.gz 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156071.log 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153291.log.gz 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.162377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140394.log 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.162377.log.gz 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T06:13:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156071.log: /var/log/ceph/ceph-client.admin.140394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161276.log 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156071.log.gz 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.2.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140394.log.gz 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147530.log 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156025.log 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161276.log.gz 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156343.log 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156025.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147530.log.gz 2023-11-24T06:13:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154685.log 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156343.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156025.log.gz 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162483.log 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156343.log.gz 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161610.log 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154685.log.gz 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155810.log 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162070.log 2023-11-24T06:13:03.808 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162483.log.gz 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155810.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161610.log.gz 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147622.log 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.155810.log.gz 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156960.log 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152076.log 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi097.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.162070.log.gz 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147622.log.gz 2023-11-24T06:13:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141060.log 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151832.log 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156960.log.gz 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141060.log: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.152076.log.gz--verbose 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:-- /var/log/ceph/ceph-client.admin.147716.log 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141060.log.gz 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150180.log 2023-11-24T06:13:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144891.log 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151832.log.gz 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159622.log 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.147716.log.gz 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150180.log.gz 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141428.log 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144891.log.gz 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161556.log 2023-11-24T06:13:03.811 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150669.log 2023-11-24T06:13:03.812 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159622.log.gz 2023-11-24T06:13:03.812 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141428.log.gz 2023-11-24T06:13:03.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157882.log 2023-11-24T06:13:03.812 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.161556.log.gz 2023-11-24T06:13:03.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150669.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:13:03.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163559.log 2023-11-24T06:13:03.812 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.150669.log.gz 2023-11-24T06:13:03.812 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157882.log.gz 2023-11-24T06:13:03.813 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146263.log 2023-11-24T06:13:03.813 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159035.log 2023-11-24T06:13:03.813 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146263.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163559.log.gz 2023-11-24T06:13:03.813 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141055.log 2023-11-24T06:13:03.813 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146263.log.gz 2023-11-24T06:13:03.813 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165114.log 2023-11-24T06:13:03.813 INFO:teuthology.orchestra.run.smithi097.stderr: 85.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:13:03.813 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.159035.log.gz 2023-11-24T06:13:03.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154304.log 2023-11-24T06:13:03.814 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141055.log.gz 2023-11-24T06:13:03.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151397.log 2023-11-24T06:13:03.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153791.log 2023-11-24T06:13:03.814 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.165114.log.gz 2023-11-24T06:13:03.814 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151397.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154304.log.gz 2023-11-24T06:13:03.814 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154491.log 2023-11-24T06:13:03.814 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151397.log.gz 2023-11-24T06:13:03.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149904.log 2023-11-24T06:13:03.815 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153791.log.gz 2023-11-24T06:13:03.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159576.log 2023-11-24T06:13:03.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146813.log 2023-11-24T06:13:03.815 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154491.log.gz 2023-11-24T06:13:03.815 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159576.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.149904.log.gz 2023-11-24T06:13:03.815 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145202.log 2023-11-24T06:13:03.815 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159576.log.gz 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160032.log 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146813.log: /var/log/ceph/ceph-client.admin.145202.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146813.log.gz 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153585.log 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145202.log.gz 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161794.log 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160032.log: /var/log/ceph/ceph-client.admin.153585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156389.log 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.160032.log.gz 2023-11-24T06:13:03.816 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161794.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.153585.log.gz 2023-11-24T06:13:03.817 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164286.log 2023-11-24T06:13:03.817 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161794.log.gz 2023-11-24T06:13:03.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163354.log 2023-11-24T06:13:03.817 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156389.log.gz 2023-11-24T06:13:03.817 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153153.log 2023-11-24T06:13:03.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164286.log: /var/log/ceph/ceph-client.admin.163354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144477.log 2023-11-24T06:13:03.817 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164286.log.gz 2023-11-24T06:13:03.817 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153153.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163354.log.gz 2023-11-24T06:13:03.818 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153245.log 2023-11-24T06:13:03.818 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144477.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153153.log.gz 2023-11-24T06:13:03.818 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161656.log 2023-11-24T06:13:03.818 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144477.log.gz 2023-11-24T06:13:03.818 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148651.log 2023-11-24T06:13:03.818 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153245.log: /var/log/ceph/ceph-client.admin.161656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155764.log 2023-11-24T06:13:03.818 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153245.log.gz 2023-11-24T06:13:03.818 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148651.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.161656.log.gz 2023-11-24T06:13:03.819 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152904.log 2023-11-24T06:13:03.819 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148651.log.gz 2023-11-24T06:13:03.819 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164838.log 2023-11-24T06:13:03.819 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155764.log.gz 2023-11-24T06:13:03.819 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162879.log 2023-11-24T06:13:03.819 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152904.log.gz 2023-11-24T06:13:03.819 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133723.log 2023-11-24T06:13:03.820 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162879.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164838.log.gz 2023-11-24T06:13:03.820 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140348.log 2023-11-24T06:13:03.820 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159185.log 2023-11-24T06:13:03.820 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162879.log.gz 2023-11-24T06:13:03.820 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.133723.log.gz 2023-11-24T06:13:03.820 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147083.log 2023-11-24T06:13:03.820 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159185.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140348.log.gz 2023-11-24T06:13:03.820 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157460.log 2023-11-24T06:13:03.821 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159185.log.gz 2023-11-24T06:13:03.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151984.log 2023-11-24T06:13:03.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157460.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147083.log.gz 2023-11-24T06:13:03.821 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158254.log 2023-11-24T06:13:03.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151984.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157460.log.gz 2023-11-24T06:13:03.821 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156684.log 2023-11-24T06:13:03.821 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158254.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.151984.log.gz 2023-11-24T06:13:03.821 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139500.log 2023-11-24T06:13:03.822 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158254.log.gz 2023-11-24T06:13:03.822 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157006.log 2023-11-24T06:13:03.822 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139500.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156684.log.gz 2023-11-24T06:13:03.822 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150510.log 2023-11-24T06:13:03.822 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160281.log 2023-11-24T06:13:03.822 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139500.log.gz 2023-11-24T06:13:03.822 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150510.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157006.log.gz 2023-11-24T06:13:03.823 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144707.log 2023-11-24T06:13:03.823 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.150510.log.gz 2023-11-24T06:13:03.823 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165274.log 2023-11-24T06:13:03.823 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144707.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160281.log.gz 2023-11-24T06:13:03.823 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155614.log 2023-11-24T06:13:03.823 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144707.log.gz 2023-11-24T06:13:03.823 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.151634.log 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155614.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.165274.log.gz-5 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr:--verbose -- /var/log/ceph/ceph-client.admin.162621.log 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155614.log.gz 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.151634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155133.log 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.151634.log.gz 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162621.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144246.log 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.162621.log.gz 2023-11-24T06:13:03.824 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148591.log 2023-11-24T06:13:03.825 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155133.log.gz 2023-11-24T06:13:03.825 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144154.log 2023-11-24T06:13:03.825 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148591.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.144246.log.gz 2023-11-24T06:13:03.825 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157789.log 2023-11-24T06:13:03.825 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8%/var/log/ceph/ceph-client.admin.144154.log: -- replaced with /var/log/ceph/ceph-client.admin.148591.log.gz 2023-11-24T06:13:03.825 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148850.log 2023-11-24T06:13:03.825 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144154.log.gz 2023-11-24T06:13:03.825 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149425.log 2023-11-24T06:13:03.826 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.157789.log.gz-- 2023-11-24T06:13:03.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164746.log 2023-11-24T06:13:03.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148850.log: /var/log/ceph/ceph-client.admin.149425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147994.log 2023-11-24T06:13:03.826 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148850.log.gz 2023-11-24T06:13:03.826 INFO:teuthology.orchestra.run.smithi097.stderr:gzip /var/log/ceph/ceph-client.admin.164746.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.164424.log 2023-11-24T06:13:03.826 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149425.log.gz 2023-11-24T06:13:03.826 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147994.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164746.log.gz 2023-11-24T06:13:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149517.log 2023-11-24T06:13:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156776.log 2023-11-24T06:13:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164424.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.147994.log.gz 2023-11-24T06:13:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160189.log 2023-11-24T06:13:03.827 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.164424.log.gz 2023-11-24T06:13:03.827 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149517.log.gz 2023-11-24T06:13:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149563.log 2023-11-24T06:13:03.827 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144799.log 2023-11-24T06:13:03.828 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5%/var/log/ceph/ceph-client.admin.160189.log: -- replaced with /var/log/ceph/ceph-client.admin.156776.log.gz 2023-11-24T06:13:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149563.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.160189.log.gz 2023-11-24T06:13:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154971.log 2023-11-24T06:13:03.828 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149563.log.gz 2023-11-24T06:13:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156209.log 2023-11-24T06:13:03.828 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144799.log: /var/log/ceph/ceph-client.admin.154971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148545.log 2023-11-24T06:13:03.828 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144799.log.gz 2023-11-24T06:13:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156209.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154971.log.gz 2023-11-24T06:13:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146447.log 2023-11-24T06:13:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162925.log 2023-11-24T06:13:03.829 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156209.log.gz 2023-11-24T06:13:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146447.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.148545.log.gz 2023-11-24T06:13:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.163789.log 2023-11-24T06:13:03.829 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- 87.3%/var/log/ceph/ceph-client.admin.160854.log 2023-11-24T06:13:03.829 INFO:teuthology.orchestra.run.smithi097.stderr: -- replaced with /var/log/ceph/ceph-client.admin.146447.log.gz 2023-11-24T06:13:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162925.log: /var/log/ceph/ceph-client.admin.163789.log: gzip 89.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.162925.log.gz-- 2023-11-24T06:13:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133856.log 2023-11-24T06:13:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160854.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163789.log.gz 2023-11-24T06:13:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163743.log 2023-11-24T06:13:03.830 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133856.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T06:13:03.830 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160854.log.gz 2023-11-24T06:13:03.830 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133856.log.gz 2023-11-24T06:13:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146355.log 2023-11-24T06:13:03.831 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163743.log.gz 2023-11-24T06:13:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145110.log 2023-11-24T06:13:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.146355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160578.log 2023-11-24T06:13:03.831 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146355.log.gz 2023-11-24T06:13:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145428.log 2023-11-24T06:13:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164378.log 2023-11-24T06:13:03.831 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160578.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145110.log.gz 2023-11-24T06:13:03.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145428.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.160578.log.gz 2023-11-24T06:13:03.832 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141429.log 2023-11-24T06:13:03.832 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145428.log.gz/var/log/ceph/ceph-client.admin.164378.log: 2023-11-24T06:13:03.832 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146079.log 2023-11-24T06:13:03.832 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4%/var/log/ceph/ceph-client.admin.141429.log: -- replaced with /var/log/ceph/ceph-client.admin.164378.log.gz 2023-11-24T06:13:03.832 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146079.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141429.log.gz 2023-11-24T06:13:03.832 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146079.log.gz 2023-11-24T06:13:03.832 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146585.log 2023-11-24T06:13:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156822.log 2023-11-24T06:13:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163243.log 2023-11-24T06:13:03.833 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.146585.log.gz 2023-11-24T06:13:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148996.log 2023-11-24T06:13:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146309.log 2023-11-24T06:13:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163243.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156822.log.gz 2023-11-24T06:13:03.833 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157230.log 2023-11-24T06:13:03.834 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163243.log.gz 2023-11-24T06:13:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146309.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148996.log.gz 2023-11-24T06:13:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141682.log 2023-11-24T06:13:03.834 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146309.log.gz 2023-11-24T06:13:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164019.log 2023-11-24T06:13:03.834 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141682.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157230.log.gz 2023-11-24T06:13:03.834 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141682.log.gz 2023-11-24T06:13:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164019.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164019.log.gz 2023-11-24T06:13:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154083.log 2023-11-24T06:13:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154593.log 2023-11-24T06:13:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141057.log 2023-11-24T06:13:03.835 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154083.log.gz 2023-11-24T06:13:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154593.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T06:13:03.835 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141057.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154593.log.gz 2023-11-24T06:13:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139745.log 2023-11-24T06:13:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.3.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.141057.log.gz 2023-11-24T06:13:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139745.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139745.log.gz 2023-11-24T06:13:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.151630.log 2023-11-24T06:13:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152144.log 2023-11-24T06:13:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.151630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.151630.log.gz 2023-11-24T06:13:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152144.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.152144.log.gz 2023-11-24T06:13:03.836 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151005.log 2023-11-24T06:13:03.837 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162575.log 2023-11-24T06:13:03.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151005.log: /var/log/ceph/ceph-client.admin.162575.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151005.log.gz 2023-11-24T06:13:03.837 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.162575.log.gz 2023-11-24T06:13:03.837 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155568.log 2023-11-24T06:13:03.837 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162833.log 2023-11-24T06:13:03.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160097.log 2023-11-24T06:13:03.837 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162833.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155568.log.gz 2023-11-24T06:13:03.838 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160097.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.162833.log.gz 2023-11-24T06:13:03.838 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160097.log.gz 2023-11-24T06:13:03.838 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148804.log 2023-11-24T06:13:03.838 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145336.log 2023-11-24T06:13:03.838 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148804.log: /var/log/ceph/ceph-client.admin.145336.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.148804.log.gz 2023-11-24T06:13:03.838 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152030.log 2023-11-24T06:13:03.838 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145336.log.gz 2023-11-24T06:13:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153107.log 2023-11-24T06:13:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149222.log 2023-11-24T06:13:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153107.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.152030.log.gz 2023-11-24T06:13:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149222.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.153107.log.gz 2023-11-24T06:13:03.839 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.149222.log.gz 2023-11-24T06:13:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161978.log 2023-11-24T06:13:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161702.log 2023-11-24T06:13:03.839 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161978.log: /var/log/ceph/ceph-client.admin.161702.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.161978.log.gz 2023-11-24T06:13:03.840 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161702.log.gz 2023-11-24T06:13:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159360.log 2023-11-24T06:13:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150811.log 2023-11-24T06:13:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159360.log: /var/log/ceph/ceph-client.admin.150811.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159360.log.gz 2023-11-24T06:13:03.840 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.150811.log.gz--verbose 2023-11-24T06:13:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:-- /var/log/ceph/ceph-client.admin.160373.log 2023-11-24T06:13:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159668.log 2023-11-24T06:13:03.840 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157322.log 2023-11-24T06:13:03.841 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160373.log.gz 2023-11-24T06:13:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153475.log 2023-11-24T06:13:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157322.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159668.log.gz 2023-11-24T06:13:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153475.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157322.log.gz 2023-11-24T06:13:03.841 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153475.log.gz 2023-11-24T06:13:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165068.log 2023-11-24T06:13:03.841 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145520.log 2023-11-24T06:13:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165068.log: /var/log/ceph/ceph-client.admin.145520.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165068.log.gz 2023-11-24T06:13:03.842 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.145520.log.gz 2023-11-24T06:13:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147221.log 2023-11-24T06:13:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139631.log 2023-11-24T06:13:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147221.log: /var/log/ceph/ceph-client.admin.139631.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147221.log.gz 2023-11-24T06:13:03.842 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139631.log.gz 2023-11-24T06:13:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143903.log 2023-11-24T06:13:03.842 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144523.log 2023-11-24T06:13:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155476.log 2023-11-24T06:13:03.843 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143903.log.gz 2023-11-24T06:13:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144523.log: /var/log/ceph/ceph-client.admin.155476.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144523.log.gz 2023-11-24T06:13:03.843 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155476.log.gz 2023-11-24T06:13:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148950.log 2023-11-24T06:13:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149655.log 2023-11-24T06:13:03.843 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145669.log 2023-11-24T06:13:03.844 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148950.log.gz 2023-11-24T06:13:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149655.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.149655.log.gz 2023-11-24T06:13:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148407.log 2023-11-24T06:13:03.844 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.145669.log.gz 2023-11-24T06:13:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148407.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148407.log.gz 2023-11-24T06:13:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157184.log 2023-11-24T06:13:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142488.log 2023-11-24T06:13:03.844 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157184.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157184.log.gz 2023-11-24T06:13:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142488.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142488.log.gz 2023-11-24T06:13:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165364.log 2023-11-24T06:13:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150763.log 2023-11-24T06:13:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165364.log: /var/log/ceph/ceph-client.admin.150763.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165364.log.gz 2023-11-24T06:13:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147313.log 2023-11-24T06:13:03.845 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.150763.log.gz 2023-11-24T06:13:03.845 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139391.log 2023-11-24T06:13:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146723.log 2023-11-24T06:13:03.846 INFO:teuthology.orchestra.run.smithi097.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147313.log.gz 2023-11-24T06:13:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139391.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.139391.log.gz 2023-11-24T06:13:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146723.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146723.log.gz 2023-11-24T06:13:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149084.log 2023-11-24T06:13:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139837.log 2023-11-24T06:13:03.846 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149084.log: /var/log/ceph/ceph-client.admin.139837.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149084.log.gz 2023-11-24T06:13:03.847 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139837.log.gz 2023-11-24T06:13:03.847 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160808.log 2023-11-24T06:13:03.848 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T06:13:03.848 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133786.log 2023-11-24T06:13:03.848 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.133786.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133786.log.gz 2023-11-24T06:13:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141416.log 2023-11-24T06:13:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160808.log: /var/log/ceph/ceph-client.admin.141416.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160808.log.gz 2023-11-24T06:13:03.878 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141416.log.gz 2023-11-24T06:13:03.878 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151235.log 2023-11-24T06:13:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155522.log 2023-11-24T06:13:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143200.log 2023-11-24T06:13:03.879 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151235.log.gz 2023-11-24T06:13:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162024.log 2023-11-24T06:13:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143200.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155522.log.gz 2023-11-24T06:13:03.879 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133415.log 2023-11-24T06:13:03.879 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143200.log.gz 2023-11-24T06:13:03.880 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158024.log 2023-11-24T06:13:03.880 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.162024.log.gz 2023-11-24T06:13:03.880 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.133415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161886.log 2023-11-24T06:13:03.880 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.133415.log.gz 2023-11-24T06:13:03.880 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158024.log: /var/log/ceph/ceph-client.admin.161886.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158024.log.gz 2023-11-24T06:13:03.880 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161886.log.gz 2023-11-24T06:13:03.880 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152858.log 2023-11-24T06:13:03.880 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146217.log 2023-11-24T06:13:03.881 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152858.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T06:13:03.881 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152858.log.gz 2023-11-24T06:13:03.881 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147037.log 2023-11-24T06:13:03.881 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146217.log.gz 2023-11-24T06:13:03.881 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.147037.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147037.log.gz 2023-11-24T06:13:03.881 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145474.log 2023-11-24T06:13:03.881 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145474.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145474.log.gz 2023-11-24T06:13:03.882 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152234.log 2023-11-24T06:13:03.882 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149130.log 2023-11-24T06:13:03.882 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152234.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152234.log.gz 2023-11-24T06:13:03.882 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149130.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149130.log.gz 2023-11-24T06:13:03.882 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154639.log 2023-11-24T06:13:03.882 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150134.log 2023-11-24T06:13:03.882 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154639.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154639.log.gz 2023-11-24T06:13:03.883 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150134.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.150134.log.gz 2023-11-24T06:13:03.883 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150334.log 2023-11-24T06:13:03.883 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142058.log 2023-11-24T06:13:03.883 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150334.log: /var/log/ceph/ceph-client.admin.142058.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.150334.log.gz 2023-11-24T06:13:03.883 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142058.log.gz 2023-11-24T06:13:03.883 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140026.log 2023-11-24T06:13:03.883 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146033.log 2023-11-24T06:13:03.883 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140026.log: /var/log/ceph/ceph-client.admin.146033.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140026.log.gz 2023-11-24T06:13:03.884 INFO:teuthology.orchestra.run.smithi097.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.146033.log.gz 2023-11-24T06:13:03.884 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161840.log 2023-11-24T06:13:03.884 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153975.log 2023-11-24T06:13:03.884 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161840.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161840.log.gz 2023-11-24T06:13:03.884 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153975.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153975.log.gz 2023-11-24T06:13:03.884 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156546.log 2023-11-24T06:13:03.884 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158392.log 2023-11-24T06:13:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156546.log: /var/log/ceph/ceph-client.admin.158392.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156546.log.gz 2023-11-24T06:13:03.885 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158392.log.gz 2023-11-24T06:13:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141636.log 2023-11-24T06:13:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156297.log 2023-11-24T06:13:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141636.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.141636.log.gz 2023-11-24T06:13:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156297.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156297.log.gz 2023-11-24T06:13:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149793.log 2023-11-24T06:13:03.885 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160946.log 2023-11-24T06:13:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149793.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149793.log.gz 2023-11-24T06:13:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160946.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.160946.log.gz 2023-11-24T06:13:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163697.log 2023-11-24T06:13:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.142970.log 2023-11-24T06:13:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163697.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163697.log.gz 2023-11-24T06:13:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.142970.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.142970.log.gz 2023-11-24T06:13:03.886 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162725.log 2023-11-24T06:13:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149701.log 2023-11-24T06:13:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162725.log: /var/log/ceph/ceph-client.admin.149701.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162725.log.gz 2023-11-24T06:13:03.887 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149701.log.gz 2023-11-24T06:13:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160716.log 2023-11-24T06:13:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156163.log 2023-11-24T06:13:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160716.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160716.log.gz 2023-11-24T06:13:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156163.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156163.log.gz 2023-11-24T06:13:03.887 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159127.log 2023-11-24T06:13:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141426.log 2023-11-24T06:13:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159127.log: /var/log/ceph/ceph-client.admin.141426.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159127.log.gz 2023-11-24T06:13:03.888 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141426.log.gz 2023-11-24T06:13:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142150.log 2023-11-24T06:13:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140686.log 2023-11-24T06:13:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163835.log 2023-11-24T06:13:03.888 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142150.log.gz 2023-11-24T06:13:03.888 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140686.log: /var/log/ceph/ceph-client.admin.163835.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140686.log.gz 2023-11-24T06:13:03.889 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163835.log.gz 2023-11-24T06:13:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158116.log 2023-11-24T06:13:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142288.log 2023-11-24T06:13:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158116.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158116.log.gz 2023-11-24T06:13:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142288.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142288.log.gz 2023-11-24T06:13:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157928.log 2023-11-24T06:13:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153199.log 2023-11-24T06:13:03.889 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157928.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157928.log.gz 2023-11-24T06:13:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153199.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153199.log.gz 2023-11-24T06:13:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157658.log 2023-11-24T06:13:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159940.log 2023-11-24T06:13:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157658.log: /var/log/ceph/ceph-client.admin.159940.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157658.log.gz 2023-11-24T06:13:03.890 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159940.log.gz 2023-11-24T06:13:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.158837.log 2023-11-24T06:13:03.890 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160900.log 2023-11-24T06:13:03.891 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.158837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.158837.log.gz 2023-11-24T06:13:03.891 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160900.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.160900.log.gz 2023-11-24T06:13:03.891 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151097.log 2023-11-24T06:13:03.891 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151924.log 2023-11-24T06:13:03.891 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151097.log: /var/log/ceph/ceph-client.admin.151924.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.151097.log.gz 2023-11-24T06:13:03.891 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151924.log.gz 2023-11-24T06:13:03.891 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140689.log 2023-11-24T06:13:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161368.log 2023-11-24T06:13:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140689.log: /var/log/ceph/ceph-client.admin.161368.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.140689.log.gz 2023-11-24T06:13:03.892 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161368.log.gz 2023-11-24T06:13:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150865.log 2023-11-24T06:13:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162140.log 2023-11-24T06:13:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150865.log: /var/log/ceph/ceph-client.admin.162140.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.150865.log.gz 2023-11-24T06:13:03.892 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162140.log.gz 2023-11-24T06:13:03.892 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149176.log 2023-11-24T06:13:03.893 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160532.log 2023-11-24T06:13:03.893 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149176.log: /var/log/ceph/ceph-client.admin.160532.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149176.log.gz 2023-11-24T06:13:03.893 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160532.log.gz 2023-11-24T06:13:03.893 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157976.log 2023-11-24T06:13:03.893 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157743.log 2023-11-24T06:13:03.893 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157976.log: /var/log/ceph/ceph-client.admin.157743.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157976.log.gz 2023-11-24T06:13:03.893 INFO:teuthology.orchestra.run.smithi097.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157743.log.gz 2023-11-24T06:13:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:13:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147762.log 2023-11-24T06:13:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.147762.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147762.log.gz 2023-11-24T06:13:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150042.log 2023-11-24T06:13:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150042.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150042.log.gz 2023-11-24T06:13:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150623.log 2023-11-24T06:13:03.894 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150623.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:13:03.894 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150623.log.gz 2023-11-24T06:13:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164332.log 2023-11-24T06:13:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147359.log 2023-11-24T06:13:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164332.log: /var/log/ceph/ceph-client.admin.147359.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164332.log.gz 2023-11-24T06:13:03.895 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147359.log.gz 2023-11-24T06:13:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160486.log 2023-11-24T06:13:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161100.log 2023-11-24T06:13:03.895 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160486.log: /var/log/ceph/ceph-client.admin.161100.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.160486.log.gz 2023-11-24T06:13:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153383.log 2023-11-24T06:13:03.896 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161100.log.gz 2023-11-24T06:13:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141420.log 2023-11-24T06:13:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153383.log: /var/log/ceph/ceph-client.admin.141420.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.153383.log.gz 2023-11-24T06:13:03.896 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141420.log.gz 2023-11-24T06:13:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164700.log 2023-11-24T06:13:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164792.log 2023-11-24T06:13:03.896 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164700.log: /var/log/ceph/ceph-client.admin.164792.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.164700.log.gz 2023-11-24T06:13:03.897 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164792.log.gz 2023-11-24T06:13:03.897 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.148297.log 2023-11-24T06:13:03.897 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146539.log 2023-11-24T06:13:03.897 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.148297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.148297.log.gz 2023-11-24T06:13:03.897 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149268.log 2023-11-24T06:13:03.897 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146539.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146539.log.gz 2023-11-24T06:13:03.897 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149268.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149268.log.gz 2023-11-24T06:13:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153061.log 2023-11-24T06:13:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148453.log 2023-11-24T06:13:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153061.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153061.log.gz 2023-11-24T06:13:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148453.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148453.log.gz 2023-11-24T06:13:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140692.log 2023-11-24T06:13:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144339.log 2023-11-24T06:13:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140692.log: /var/log/ceph/ceph-client.admin.144339.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140692.log.gz 2023-11-24T06:13:03.898 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.155370.log 2023-11-24T06:13:03.899 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144339.log.gz 2023-11-24T06:13:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165022.log 2023-11-24T06:13:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.155370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.155370.log.gz 2023-11-24T06:13:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165022.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165022.log.gz 2023-11-24T06:13:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149379.log 2023-11-24T06:13:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161322.log 2023-11-24T06:13:03.899 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149379.log: /var/log/ceph/ceph-client.admin.161322.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149379.log.gz 2023-11-24T06:13:03.900 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161322.log.gz 2023-11-24T06:13:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144431.log 2023-11-24T06:13:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156481.log 2023-11-24T06:13:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144431.log: /var/log/ceph/ceph-client.admin.156481.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144431.log.gz 2023-11-24T06:13:03.900 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.156481.log.gz 2023-11-24T06:13:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163927.log 2023-11-24T06:13:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157138.log 2023-11-24T06:13:03.900 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163927.log: /var/log/ceph/ceph-client.admin.157138.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163927.log.gz 2023-11-24T06:13:03.901 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157138.log.gz 2023-11-24T06:13:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.148301.log 2023-11-24T06:13:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160235.log 2023-11-24T06:13:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.148301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.148301.log.gz 2023-11-24T06:13:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160235.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.160235.log.gz 2023-11-24T06:13:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141418.log 2023-11-24T06:13:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162667.log 2023-11-24T06:13:03.901 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141418.log: 88.4%/var/log/ceph/ceph-client.admin.162667.log: -- replaced with /var/log/ceph/ceph-client.admin.141418.log.gz 2023-11-24T06:13:03.902 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162667.log.gz 2023-11-24T06:13:03.902 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149333.log 2023-11-24T06:13:03.902 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147175.log 2023-11-24T06:13:03.902 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149333.log: /var/log/ceph/ceph-client.admin.147175.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149333.log.gz 2023-11-24T06:13:03.902 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147175.log.gz 2023-11-24T06:13:03.902 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151189.log 2023-11-24T06:13:03.902 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158070.log 2023-11-24T06:13:03.903 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151189.log: /var/log/ceph/ceph-client.admin.158070.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151189.log.gz 2023-11-24T06:13:03.903 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158070.log.gz 2023-11-24T06:13:03.903 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141874.log 2023-11-24T06:13:03.903 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146171.log 2023-11-24T06:13:03.903 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141874.log: /var/log/ceph/ceph-client.admin.146171.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141874.log.gz 2023-11-24T06:13:03.903 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146171.log.gz 2023-11-24T06:13:03.903 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157568.log 2023-11-24T06:13:03.903 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151327.log 2023-11-24T06:13:03.904 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157568.log: /var/log/ceph/ceph-client.admin.151327.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.157568.log.gz 2023-11-24T06:13:03.904 INFO:teuthology.orchestra.run.smithi097.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.151327.log.gz 2023-11-24T06:13:03.904 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155855.log 2023-11-24T06:13:03.904 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140118.log 2023-11-24T06:13:03.904 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155855.log: /var/log/ceph/ceph-client.admin.140118.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155855.log.gz 2023-11-24T06:13:03.904 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140118.log.gz 2023-11-24T06:13:03.904 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165318.log 2023-11-24T06:13:03.904 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139585.log 2023-11-24T06:13:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165318.log: /var/log/ceph/ceph-client.admin.139585.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165318.log.gz 2023-11-24T06:13:03.905 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.139585.log.gz 2023-11-24T06:13:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140690.log 2023-11-24T06:13:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163400.log 2023-11-24T06:13:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140690.log: /var/log/ceph/ceph-client.admin.163400.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140690.log.gz 2023-11-24T06:13:03.905 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163400.log.gz 2023-11-24T06:13:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142196.log 2023-11-24T06:13:03.905 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144292.log 2023-11-24T06:13:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142196.log: /var/log/ceph/ceph-client.admin.144292.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142196.log.gz 2023-11-24T06:13:03.906 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144292.log.gz 2023-11-24T06:13:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154879.log 2023-11-24T06:13:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152724.log 2023-11-24T06:13:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154879.log: /var/log/ceph/ceph-client.admin.152724.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.154879.log.gz 2023-11-24T06:13:03.906 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152724.log.gz 2023-11-24T06:13:03.906 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.142978.log 2023-11-24T06:13:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151143.log 2023-11-24T06:13:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.142978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.142978.log.gz 2023-11-24T06:13:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151143.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.151143.log.gz 2023-11-24T06:13:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151878.log 2023-11-24T06:13:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147576.log 2023-11-24T06:13:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151878.log: /var/log/ceph/ceph-client.admin.147576.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151878.log.gz 2023-11-24T06:13:03.907 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147576.log.gz 2023-11-24T06:13:03.907 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145941.log 2023-11-24T06:13:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163197.log 2023-11-24T06:13:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145941.log: /var/log/ceph/ceph-client.admin.163197.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.145941.log.gz 2023-11-24T06:13:03.908 INFO:teuthology.orchestra.run.smithi097.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.163197.log.gz 2023-11-24T06:13:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149950.log 2023-11-24T06:13:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146898.log 2023-11-24T06:13:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149950.log: /var/log/ceph/ceph-client.admin.146898.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.149950.log.gz 2023-11-24T06:13:03.908 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146898.log.gz 2023-11-24T06:13:03.908 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148904.log 2023-11-24T06:13:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152632.log 2023-11-24T06:13:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148904.log: /var/log/ceph/ceph-client.admin.152632.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.148904.log.gz 2023-11-24T06:13:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145895.log 2023-11-24T06:13:03.909 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152632.log.gz 2023-11-24T06:13:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164608.log 2023-11-24T06:13:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145895.log: /var/log/ceph/ceph-client.admin.164608.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145895.log.gz 2023-11-24T06:13:03.909 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164608.log.gz 2023-11-24T06:13:03.909 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160327.log 2023-11-24T06:13:03.910 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163881.log 2023-11-24T06:13:03.910 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160327.log: /var/log/ceph/ceph-client.admin.163881.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.160327.log.gz 2023-11-24T06:13:03.910 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163881.log.gz 2023-11-24T06:13:03.910 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142334.log 2023-11-24T06:13:03.910 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153929.log 2023-11-24T06:13:03.910 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142334.log: /var/log/ceph/ceph-client.admin.153929.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153929.log.gz 2023-11-24T06:13:03.910 INFO:teuthology.orchestra.run.smithi097.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.142334.log.gz 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150911.log 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153678.log 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150911.log: /var/log/ceph/ceph-client.admin.153678.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.150911.log.gz 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153678.log.gz 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145807.log 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150556.log 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145807.log: /var/log/ceph/ceph-client.admin.150556.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145807.log.gz 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150556.log.gz 2023-11-24T06:13:03.911 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T06:13:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144200.log 2023-11-24T06:13:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.144200.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.144200.log.gz 2023-11-24T06:13:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148064.log 2023-11-24T06:13:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148064.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148064.log.gz 2023-11-24T06:13:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.158833.log 2023-11-24T06:13:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.158833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.158833.log.gz 2023-11-24T06:13:03.912 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157835.log 2023-11-24T06:13:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157835.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157835.log.gz 2023-11-24T06:13:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148499.log 2023-11-24T06:13:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.148499.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148499.log.gz 2023-11-24T06:13:03.913 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157070.log 2023-11-24T06:13:03.914 INFO:teuthology.orchestra.run.smithi183.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T06:13:03.925 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157070.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157070.log.gz 2023-11-24T06:13:03.926 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164562.log 2023-11-24T06:13:03.926 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164562.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.164562.log.gz 2023-11-24T06:13:03.926 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141736.log 2023-11-24T06:13:03.926 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141736.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141736.log.gz 2023-11-24T06:13:03.926 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163289.log 2023-11-24T06:13:03.926 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163289.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.163289.log.gz 2023-11-24T06:13:03.926 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160440.log 2023-11-24T06:13:03.927 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160440.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160440.log.gz 2023-11-24T06:13:03.927 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153883.log 2023-11-24T06:13:03.927 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153883.log.gz 2023-11-24T06:13:03.927 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150226.log 2023-11-24T06:13:03.927 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.150226.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150226.log.gz 2023-11-24T06:13:03.927 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164516.log 2023-11-24T06:13:03.927 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164516.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164516.log.gz 2023-11-24T06:13:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142242.log 2023-11-24T06:13:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142242.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.142242.log.gz 2023-11-24T06:13:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152540.log 2023-11-24T06:13:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152540.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152540.log.gz 2023-11-24T06:13:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163151.log 2023-11-24T06:13:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163151.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163151.log.gz 2023-11-24T06:13:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164976.log 2023-11-24T06:13:03.928 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164976.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164976.log.gz 2023-11-24T06:13:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158989.log 2023-11-24T06:13:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158989.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158989.log.gz 2023-11-24T06:13:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154397.log 2023-11-24T06:13:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154397.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154397.log.gz 2023-11-24T06:13:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145064.log 2023-11-24T06:13:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145064.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145064.log.gz 2023-11-24T06:13:03.929 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158162.log 2023-11-24T06:13:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158162.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158162.log.gz 2023-11-24T06:13:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141059.log 2023-11-24T06:13:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141059.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141059.log.gz 2023-11-24T06:13:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158530.log 2023-11-24T06:13:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158530.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.158530.log.gz 2023-11-24T06:13:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144661.log 2023-11-24T06:13:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144661.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144661.log.gz 2023-11-24T06:13:03.930 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145761.log 2023-11-24T06:13:03.931 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145761.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145761.log.gz 2023-11-24T06:13:03.931 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144569.log 2023-11-24T06:13:03.931 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144569.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144569.log.gz 2023-11-24T06:13:03.931 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164654.log 2023-11-24T06:13:03.931 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164654.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164654.log.gz 2023-11-24T06:13:03.931 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159986.log 2023-11-24T06:13:03.931 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159986.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159986.log.gz 2023-11-24T06:13:03.932 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154350.log 2023-11-24T06:13:03.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154350.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154350.log.gz 2023-11-24T06:13:03.932 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144845.log 2023-11-24T06:13:03.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144845.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144845.log.gz 2023-11-24T06:13:03.932 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153429.log 2023-11-24T06:13:03.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153429.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153429.log.gz 2023-11-24T06:13:03.932 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147267.log 2023-11-24T06:13:03.932 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147267.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147267.log.gz 2023-11-24T06:13:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139931.log 2023-11-24T06:13:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139931.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139931.log.gz 2023-11-24T06:13:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159293.log 2023-11-24T06:13:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159293.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.159293.log.gz 2023-11-24T06:13:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149857.log 2023-11-24T06:13:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149857.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149857.log.gz 2023-11-24T06:13:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149471.log 2023-11-24T06:13:03.933 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149471.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149471.log.gz 2023-11-24T06:13:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142104.log 2023-11-24T06:13:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.142104.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.142104.log.gz 2023-11-24T06:13:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161460.log 2023-11-24T06:13:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161460.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.161460.log.gz 2023-11-24T06:13:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144016.log 2023-11-24T06:13:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144016.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.144016.log.gz 2023-11-24T06:13:03.934 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164065.log 2023-11-24T06:13:03.935 INFO:teuthology.orchestra.run.smithi097.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T06:13:03.935 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144385.log 2023-11-24T06:13:03.935 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.164065.log: /var/log/ceph/ceph-client.admin.144385.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164065.log.gz 2023-11-24T06:13:03.935 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144385.log.gz 2023-11-24T06:13:03.935 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159714.log 2023-11-24T06:13:03.935 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139979.log 2023-11-24T06:13:03.935 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159714.log: /var/log/ceph/ceph-client.admin.139979.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159714.log.gz 2023-11-24T06:13:03.935 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139979.log.gz 2023-11-24T06:13:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141268.log 2023-11-24T06:13:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142012.log 2023-11-24T06:13:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141268.log: /var/log/ceph/ceph-client.admin.142012.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141268.log.gz 2023-11-24T06:13:03.936 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142012.log.gz 2023-11-24T06:13:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141828.log 2023-11-24T06:13:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163973.log 2023-11-24T06:13:03.936 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141828.log: /var/log/ceph/ceph-client.admin.163973.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141828.log.gz 2023-11-24T06:13:03.936 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163973.log.gz 2023-11-24T06:13:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153837.log 2023-11-24T06:13:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154733.log 2023-11-24T06:13:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153837.log: /var/log/ceph/ceph-client.admin.154733.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.153837.log.gz 2023-11-24T06:13:03.937 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154733.log.gz 2023-11-24T06:13:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159760.log 2023-11-24T06:13:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152586.log 2023-11-24T06:13:03.937 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159760.log: /var/log/ceph/ceph-client.admin.152586.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159760.log.gz 2023-11-24T06:13:03.937 INFO:teuthology.orchestra.run.smithi097.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152586.log.gz 2023-11-24T06:13:03.938 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147808.log 2023-11-24T06:13:03.945 INFO:teuthology.orchestra.run.smithi183.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T06:13:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141054.log 2023-11-24T06:13:03.946 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147808.log: /var/log/ceph/ceph-client.admin.141054.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147808.log.gz 2023-11-24T06:13:03.946 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141054.log.gz 2023-11-24T06:13:03.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140164.log 2023-11-24T06:13:03.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155017.log 2023-11-24T06:13:03.947 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140164.log: /var/log/ceph/ceph-client.admin.155017.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140164.log.gz 2023-11-24T06:13:03.947 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.155017.log.gz 2023-11-24T06:13:03.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145156.log 2023-11-24T06:13:03.947 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146631.log 2023-11-24T06:13:03.947 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145156.log: /var/log/ceph/ceph-client.admin.146631.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145156.log.gz 2023-11-24T06:13:03.948 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146631.log.gz 2023-11-24T06:13:03.948 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145382.log 2023-11-24T06:13:03.948 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140486.log 2023-11-24T06:13:03.948 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145382.log: /var/log/ceph/ceph-client.admin.140486.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145382.log.gz 2023-11-24T06:13:03.948 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140486.log.gz 2023-11-24T06:13:03.948 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147856.log 2023-11-24T06:13:03.948 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146944.log 2023-11-24T06:13:03.949 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147856.log: /var/log/ceph/ceph-client.admin.146944.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147856.log.gz 2023-11-24T06:13:03.949 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146944.log.gz 2023-11-24T06:13:03.949 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141782.log 2023-11-24T06:13:03.949 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150424.log 2023-11-24T06:13:03.949 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141782.log: /var/log/ceph/ceph-client.admin.150424.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141782.log.gz 2023-11-24T06:13:03.949 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.150424.log.gz 2023-11-24T06:13:03.949 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158438.log 2023-11-24T06:13:03.949 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164884.log 2023-11-24T06:13:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158438.log: /var/log/ceph/ceph-client.admin.164884.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158438.log.gz 2023-11-24T06:13:03.950 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164884.log.gz 2023-11-24T06:13:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147948.log 2023-11-24T06:13:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145987.log 2023-11-24T06:13:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147948.log: /var/log/ceph/ceph-client.admin.145987.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.147948.log.gz 2023-11-24T06:13:03.950 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145987.log.gz 2023-11-24T06:13:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158300.log 2023-11-24T06:13:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152302.log 2023-11-24T06:13:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158300.log: /var/log/ceph/ceph-client.admin.152302.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158300.log.gz 2023-11-24T06:13:03.951 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152302.log.gz 2023-11-24T06:13:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158346.log 2023-11-24T06:13:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154173.log 2023-11-24T06:13:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158346.log: /var/log/ceph/ceph-client.admin.154173.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158346.log.gz 2023-11-24T06:13:03.951 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.154173.log.gz 2023-11-24T06:13:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163446.log 2023-11-24T06:13:03.952 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145715.log 2023-11-24T06:13:03.952 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163446.log: /var/log/ceph/ceph-client.admin.145715.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.163446.log.gz 2023-11-24T06:13:03.952 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145715.log.gz 2023-11-24T06:13:03.952 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152950.log 2023-11-24T06:13:03.952 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160762.log 2023-11-24T06:13:03.952 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152950.log: /var/log/ceph/ceph-client.admin.160762.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152950.log.gz 2023-11-24T06:13:03.952 INFO:teuthology.orchestra.run.smithi097.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.160762.log.gz 2023-11-24T06:13:03.952 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145623.log 2023-11-24T06:13:03.953 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144753.log 2023-11-24T06:13:03.953 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145623.log: /var/log/ceph/ceph-client.admin.144753.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145623.log.gz 2023-11-24T06:13:03.953 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144753.log.gz 2023-11-24T06:13:03.953 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.162373.log 2023-11-24T06:13:03.953 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151786.log 2023-11-24T06:13:03.953 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.0.162373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.162373.log.gz 2023-11-24T06:13:03.953 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151786.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.151786.log.gz 2023-11-24T06:13:03.953 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161414.log 2023-11-24T06:13:03.954 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159468.log 2023-11-24T06:13:03.954 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140693.log 2023-11-24T06:13:03.954 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159468.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161414.log.gz 2023-11-24T06:13:03.954 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159468.log.gz 2023-11-24T06:13:03.954 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151740.log 2023-11-24T06:13:03.954 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140693.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140693.log.gz 2023-11-24T06:13:03.954 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151740.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151740.log.gz 2023-11-24T06:13:03.955 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158484.log 2023-11-24T06:13:03.955 INFO:teuthology.orchestra.run.smithi183.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T06:13:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154825.log 2023-11-24T06:13:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158484.log: /var/log/ceph/ceph-client.admin.154825.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158484.log.gz 2023-11-24T06:13:03.961 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154825.log.gz 2023-11-24T06:13:03.961 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140210.log 2023-11-24T06:13:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156592.log 2023-11-24T06:13:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140210.log: /var/log/ceph/ceph-client.admin.156592.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140210.log.gz 2023-11-24T06:13:03.962 INFO:teuthology.orchestra.run.smithi097.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.156592.log.gz 2023-11-24T06:13:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158943.log 2023-11-24T06:13:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155660.log 2023-11-24T06:13:03.962 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158943.log: /var/log/ceph/ceph-client.admin.155660.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158943.log.gz 2023-11-24T06:13:03.962 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155660.log.gz 2023-11-24T06:13:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155063.log 2023-11-24T06:13:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150957.log 2023-11-24T06:13:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155063.log: /var/log/ceph/ceph-client.admin.150957.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150957.log.gz 2023-11-24T06:13:03.963 INFO:teuthology.orchestra.run.smithi097.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.155063.log.gz 2023-11-24T06:13:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163651.log 2023-11-24T06:13:03.963 INFO:teuthology.orchestra.run.smithi097.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T06:13:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140440.log 2023-11-24T06:13:03.963 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159894.log 2023-11-24T06:13:03.964 INFO:teuthology.orchestra.run.smithi097.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163651.log.gz 2023-11-24T06:13:03.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155718.log 2023-11-24T06:13:03.964 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140440.log.gz 2023-11-24T06:13:03.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160624.log 2023-11-24T06:13:03.964 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.155718.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159894.log.gz 2023-11-24T06:13:03.964 INFO:teuthology.orchestra.run.smithi097.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.155718.log.gz/var/log/ceph/ceph-client.admin.160624.log: 2023-11-24T06:13:03.964 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160624.log.gz 2023-11-24T06:13:03.965 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146125.log 2023-11-24T06:13:03.965 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141050.log 2023-11-24T06:13:03.965 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146125.log: /var/log/ceph/ceph-client.admin.141050.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146125.log.gz 2023-11-24T06:13:03.965 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141050.log.gz 2023-11-24T06:13:03.965 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147902.log 2023-11-24T06:13:03.965 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154925.log 2023-11-24T06:13:03.965 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147902.log: /var/log/ceph/ceph-client.admin.154925.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147902.log.gz 2023-11-24T06:13:03.965 INFO:teuthology.orchestra.run.smithi097.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154925.log.gz 2023-11-24T06:13:03.966 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151051.log 2023-11-24T06:13:03.966 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154258.log 2023-11-24T06:13:03.966 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151051.log: /var/log/ceph/ceph-client.admin.154258.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.151051.log.gz 2023-11-24T06:13:03.966 INFO:teuthology.orchestra.run.smithi097.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.154258.log.gz 2023-11-24T06:13:03.966 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.143631.log 2023-11-24T06:13:03.966 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158600.log 2023-11-24T06:13:03.966 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.2.143631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.143631.log.gz 2023-11-24T06:13:03.967 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158600.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158600.log.gz 2023-11-24T06:13:03.967 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139791.log 2023-11-24T06:13:03.967 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140687.log 2023-11-24T06:13:03.972 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139791.log: /var/log/ceph/ceph-client.admin.140687.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139791.log.gz 2023-11-24T06:13:03.972 INFO:teuthology.orchestra.run.smithi097.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.140687.log.gz 2023-11-24T06:13:03.973 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161932.log 2023-11-24T06:13:03.973 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163605.log 2023-11-24T06:13:03.973 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.161932.log: /var/log/ceph/ceph-client.admin.163605.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161932.log.gz 2023-11-24T06:13:03.973 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141920.log 2023-11-24T06:13:03.973 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163605.log.gz 2023-11-24T06:13:03.973 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163492.log 2023-11-24T06:13:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141920.log: /var/log/ceph/ceph-client.admin.163492.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.141920.log.gz 2023-11-24T06:13:03.974 INFO:teuthology.orchestra.run.smithi097.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163492.log.gz 2023-11-24T06:13:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156435.log 2023-11-24T06:13:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156868.log 2023-11-24T06:13:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156435.log: /var/log/ceph/ceph-client.admin.156868.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156435.log.gz 2023-11-24T06:13:03.974 INFO:teuthology.orchestra.run.smithi097.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.156868.log.gz 2023-11-24T06:13:03.974 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158208.log 2023-11-24T06:13:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150715.log 2023-11-24T06:13:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.158208.log: /var/log/ceph/ceph-client.admin.150715.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158208.log.gz 2023-11-24T06:13:03.975 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150715.log.gz 2023-11-24T06:13:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157368.log 2023-11-24T06:13:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150088.log 2023-11-24T06:13:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157368.log: /var/log/ceph/ceph-client.admin.150088.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157368.log.gz 2023-11-24T06:13:03.975 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.150088.log.gz 2023-11-24T06:13:03.975 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143857.log 2023-11-24T06:13:03.976 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141053.log 2023-11-24T06:13:03.976 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.143857.log: /var/log/ceph/ceph-client.admin.141053.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143857.log.gz 2023-11-24T06:13:03.976 INFO:teuthology.orchestra.run.smithi097.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141053.log.gz 2023-11-24T06:13:03.976 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149609.log 2023-11-24T06:13:03.976 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140302.log 2023-11-24T06:13:03.976 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149609.log: /var/log/ceph/ceph-client.admin.140302.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149609.log.gz 2023-11-24T06:13:03.976 INFO:teuthology.orchestra.run.smithi097.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140302.log.gz 2023-11-24T06:13:03.977 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165206.log 2023-11-24T06:13:03.977 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161508.log 2023-11-24T06:13:03.977 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.165206.log: /var/log/ceph/ceph-client.admin.161508.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165206.log.gz 2023-11-24T06:13:03.977 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161508.log.gz 2023-11-24T06:13:03.977 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139678.log 2023-11-24T06:13:03.977 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161748.log 2023-11-24T06:13:03.980 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139678.log: /var/log/ceph/ceph-client.admin.161748.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139678.log.gz 2023-11-24T06:13:03.981 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161748.log.gz 2023-11-24T06:13:03.981 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T06:13:03.981 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161190.log 2023-11-24T06:13:03.981 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.161190.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161190.log.gz 2023-11-24T06:13:03.981 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160992.log 2023-11-24T06:13:03.981 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164930.log 2023-11-24T06:13:03.981 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.160992.log: /var/log/ceph/ceph-client.admin.164930.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.160992.log.gz 2023-11-24T06:13:03.982 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164930.log.gz 2023-11-24T06:13:03.982 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153745.log 2023-11-24T06:13:03.982 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148696.log 2023-11-24T06:13:03.982 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153745.log: /var/log/ceph/ceph-client.admin.148696.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153745.log.gz 2023-11-24T06:13:03.982 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149747.log 2023-11-24T06:13:03.982 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.148696.log.gz 2023-11-24T06:13:03.982 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152372.log 2023-11-24T06:13:03.983 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.149747.log: /var/log/ceph/ceph-client.admin.152372.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.149747.log.gz 2023-11-24T06:13:03.983 INFO:teuthology.orchestra.run.smithi097.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152372.log.gz 2023-11-24T06:13:03.983 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156914.log 2023-11-24T06:13:03.983 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154539.log 2023-11-24T06:13:03.983 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.156914.log: /var/log/ceph/ceph-client.admin.154539.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156914.log.gz 2023-11-24T06:13:03.983 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154539.log.gz 2023-11-24T06:13:03.983 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151281.log 2023-11-24T06:13:03.983 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140072.log 2023-11-24T06:13:03.984 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.151281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156638.log 2023-11-24T06:13:03.986 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151281.log.gz 2023-11-24T06:13:03.986 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.140072.log: /var/log/ceph/ceph-client.admin.156638.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140072.log.gz 2023-11-24T06:13:03.986 INFO:teuthology.orchestra.run.smithi097.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156638.log.gz 2023-11-24T06:13:03.987 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145248.log 2023-11-24T06:13:03.987 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147670.log 2023-11-24T06:13:03.987 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.145248.log: /var/log/ceph/ceph-client.admin.147670.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145248.log.gz 2023-11-24T06:13:03.987 INFO:teuthology.orchestra.run.smithi097.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147670.log.gz 2023-11-24T06:13:03.987 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139295.log 2023-11-24T06:13:03.987 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.143635.log 2023-11-24T06:13:03.987 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.139295.log: /var/log/ceph/ceph-client.2.143635.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.139295.log.gz 2023-11-24T06:13:03.987 INFO:teuthology.orchestra.run.smithi097.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.143635.log.gz 2023-11-24T06:13:03.988 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146990.log 2023-11-24T06:13:03.988 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165160.log 2023-11-24T06:13:03.988 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146990.log: /var/log/ceph/ceph-client.admin.165160.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146990.log.gz 2023-11-24T06:13:03.988 INFO:teuthology.orchestra.run.smithi097.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165160.log.gz 2023-11-24T06:13:03.988 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153337.log 2023-11-24T06:13:03.988 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152678.log 2023-11-24T06:13:03.988 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.153337.log: /var/log/ceph/ceph-client.admin.152678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153337.log.gz 2023-11-24T06:13:03.989 INFO:teuthology.orchestra.run.smithi097.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152678.log.gz 2023-11-24T06:13:03.989 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157414.log 2023-11-24T06:13:03.989 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T06:13:03.989 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.157414.log: /var/log/ceph/ceph-mgr.y.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157414.log.gz 2023-11-24T06:13:03.990 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163017.log 2023-11-24T06:13:03.994 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159081.log 2023-11-24T06:13:03.995 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.163017.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163017.log.gz 2023-11-24T06:13:03.996 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.159081.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159081.log.gz 2023-11-24T06:13:03.996 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147129.log 2023-11-24T06:13:03.996 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154443.log 2023-11-24T06:13:03.997 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.147129.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147129.log.gz 2023-11-24T06:13:03.997 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154443.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154443.log.gz 2023-11-24T06:13:03.997 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154779.log 2023-11-24T06:13:03.998 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144937.log 2023-11-24T06:13:03.998 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.154779.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.154779.log.gz 2023-11-24T06:13:03.999 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144937.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144937.log.gz 2023-11-24T06:13:03.999 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146493.log 2023-11-24T06:13:03.999 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141425.log 2023-11-24T06:13:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.146493.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146493.log.gz 2023-11-24T06:13:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.141425.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141425.log.gz 2023-11-24T06:13:04.000 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152996.log 2023-11-24T06:13:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.152996.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.152996.log.gz 2023-11-24T06:13:04.001 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144108.log 2023-11-24T06:13:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.144108.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.144108.log.gz 2023-11-24T06:13:04.002 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162529.log 2023-11-24T06:13:04.003 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.162529.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162529.log.gz 2023-11-24T06:13:04.008 INFO:teuthology.orchestra.run.smithi183.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T06:13:04.023 INFO:teuthology.orchestra.run.smithi097.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T06:13:04.042 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T06:13:04.060 INFO:teuthology.orchestra.run.smithi097.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T06:13:04.072 INFO:teuthology.orchestra.run.smithi183.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T06:13:04.081 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T06:13:04.087 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T06:13:04.119 INFO:teuthology.orchestra.run.smithi183.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T06:13:04.119 INFO:teuthology.orchestra.run.smithi097.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T06:13:04.121 INFO:teuthology.orchestra.run.smithi183.stderr: 2023-11-24T06:13:04.121 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m0.372s 2023-11-24T06:13:04.121 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m1.796s 2023-11-24T06:13:04.121 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.077s 2023-11-24T06:13:04.137 INFO:teuthology.orchestra.run.smithi097.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T06:13:04.318 INFO:teuthology.orchestra.run.smithi097.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T06:13:04.320 INFO:teuthology.orchestra.run.smithi097.stderr: 2023-11-24T06:13:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m0.595s 2023-11-24T06:13:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m2.061s 2023-11-24T06:13:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.307s 2023-11-24T06:13:04.320 INFO:tasks.ceph:Archiving logs... 2023-11-24T06:13:04.321 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/remote/smithi097/log 2023-11-24T06:13:04.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:13:04.989 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/remote/smithi183/log 2023-11-24T06:13:04.990 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:13:05.212 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T06:13:05.237 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T06:13:05.237 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:13:05.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:13:05.320 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T06:13:05.329 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T06:13:05.330 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:13:05.332 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:13:05.346 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-11-24T06:13:05.349 INFO:teuthology.orchestra.run.smithi183.stderr:bash: ntpq: command not found 2023-11-24T06:13:05.350 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:13:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-11-24T06:13:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 60 +2158us[+2149us] +/- 69ms 2023-11-24T06:13:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -1670us[-1679us] +/- 57ms 2023-11-24T06:13:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -149us[ -157us] +/- 89ms 2023-11-24T06:13:05.351 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:13:05.352 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:13:05.352 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2023-11-24T06:13:05.353 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 59 +2174us[+2189us] +/- 69ms 2023-11-24T06:13:05.353 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv02.front.sepia.ceph.com 3 6 377 59 -1643us[-1628us] +/- 57ms 2023-11-24T06:13:05.353 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -144us[ -129us] +/- 89ms 2023-11-24T06:13:05.353 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:13:05.354 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T06:13:05.394 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T06:13:05.394 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T06:13:05.410 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:13:05.516 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:13:05.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:13:05.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:13:05.644 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:13:05.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:13:05.677 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T06:13:05.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T06:13:05.699 INFO:teuthology.task.internal:Duration was 1205.698343 seconds 2023-11-24T06:13:05.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T06:13:05.709 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T06:13:05.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:13:05.712 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:13:05.755 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:13:05.764 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:13:05.991 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T06:13:05.991 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2023-11-24T06:13:05.992 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:13:06.013 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2023-11-24T06:13:06.014 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:13:06.036 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T06:13:06.037 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:13:06.056 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:13:06.119 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T06:13:06.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:13:06.121 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:13:06.238 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T06:13:06.248 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T06:13:06.249 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:13:06.276 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:13:06.304 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T06:13:06.315 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:13:06.318 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:13:06.342 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2023-11-24T06:13:06.371 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2023-11-24T06:13:06.384 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:13:06.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:13:06.411 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:13:06.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:13:06.442 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T06:13:06.453 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T06:13:06.454 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/remote/smithi097 2023-11-24T06:13:06.454 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:13:06.493 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550/remote/smithi183 2023-11-24T06:13:06.493 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:13:06.531 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T06:13:06.532 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:13:06.534 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:13:06.583 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T06:13:06.595 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T06:13:06.595 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T06:13:06.667 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T06:13:06.667 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:13:06.670 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:13:06.684 INFO:teuthology.orchestra.run.smithi097.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:13 /home/ubuntu/cephtest 2023-11-24T06:13:06.688 INFO:teuthology.orchestra.run.smithi183.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:13 /home/ubuntu/cephtest 2023-11-24T06:13:06.689 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T06:13:06.702 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T06:13:06.733 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T06:13:06.759 DEBUG:teuthology.nuke:shortname: smithi097 2023-11-24T06:13:06.760 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:13:06.783 DEBUG:teuthology.nuke:shortname: smithi183 2023-11-24T06:13:06.783 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:13:06.786 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:47:46.624557', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T06:13:06.809 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465550', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:47:46.625801', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T06:13:06.852 INFO:teuthology.orchestra.console.smithi183:Power off 2023-11-24T06:13:06.853 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:13:06.875 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-11-24T06:13:06.875 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:13:06.889 INFO:teuthology.orchestra.console.smithi097:Power off 2023-11-24T06:13:06.890 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:13:06.899 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-11-24T06:13:06.911 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2023-11-24T06:13:06.912 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:13:06.933 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-11-24T06:13:10.901 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:13:10.934 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:13:11.011 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-11-24T06:13:11.044 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-11-24T06:13:15.016 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:13:15.045 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:13:15.127 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-11-24T06:13:15.154 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-11-24T06:13:15.229 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-11-24T06:13:19.158 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:13:19.271 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2023-11-24T06:13:19.372 INFO:teuthology.orchestra.console.smithi097:Power off completed 2023-11-24T06:13:19.477 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} duration: 1205.6983432769775 failure_reason: 'Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b3422b105f6e4d53b8ecf42136d28439 status: fail success: false 2023-11-24T06:13:19.477 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:13:19.564 INFO:teuthology.run:FAIL